Microsoft patents efficient way to send video to AI agents
This filing describes a method for a device to send video and audio to an automated agent while a user is speaking. It reduces data usage by lowering image quality and only sending the parts of the screen that change during the conversation.
In the filing’s own words
A client system interacts with a multi-modal automated agent via an agent interface in an efficient manner. The client system captures instance of image data from a camera or user-interface (UI) system at a prescribed image-capturing rate while a question is being asked. The client system sends the captured instances of query data and accompanying instances of query data to the automated agent via the agent interface in a series of content-bearing messages, and, in return, receives one or more replies from the automated agent via the agent interface. Other system features provide for the reduction in resolution of the instances of image data by the client system, the selective forwarding of only updated parts of the instances of image data that to the agent interface, and the static or dynamic insertion of prompt information into a system prompt by the agent interface.
What exactly they claim (claim 1)
1. A method, performed by a client system, for interacting with an automated agent, comprising: receiving an instance of query data from at least one input device, the receiving indicating that a question is being asked, wherein the instance of query data is audio data and/or text input data; receiving an instance of image data from an image source; sending a content-bearing message to an agent interface that includes the instance of image data and the instance of query data; receiving a reply from the automated agent via the agent interface, wherein the reply is generated by the automated agent using a language model based on the instance of image data and the instance of query data; generating output information based on the reply; and repeating the receiving an instance of query data, receiving an instance of image data, sending, receiving a reply, and generating output information during a span of time in which the question is being asked.
Facts
- Company
- Microsoft (Microsoft Technology Licensing, LLC)
- Official title
- Resource-Efficient Interaction with a Multi-Modal Automated Agent
- Publication number
- US20260289154A1
- Application number
- 19232898
- Filed
- Jun 10, 2025
- Published
- Sep 24, 2026
- Inventors
- 8
- Claims
- 20
Classified under
G06F 40/40 Physics & computingG06F 3/0481 Physics & computingG06F 3/0485 Physics & computingMore from Microsoft in AI & software
- Microsoft patents method to check AI answers against verified facts Oct 20, 2026
- Microsoft patents AI that presents PowerPoint slides for you Sep 3, 2026
- Microsoft patents method to check if LLM answers are based on real data Sep 10, 2026
- Microsoft patents system to blend multiple AI agents into one chat Oct 1, 2026
- Microsoft patents system that answers questions using your screen history Oct 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.