Microsoft patents chat summarization to keep long conversations on track
This filing describes a method for keeping long conversations with AI models from losing track of earlier details. When the model's memory limit is reached, the system creates a short summary of the past chat to replace the full history. The original messages are saved separately so the AI can retrieve specific details if needed during the conversation.
In the filing’s own words
A chat history between a user and a machine learning model is preserved despite the context window size constraint of the machine learning model to ensure an enduring understanding of the chat or conversation history. When the token constraint of the context window size is reached, a summary of the chat history is generated to replace the chat history. The original content of the summarized chat history is stored in a lookup table and indexed by keywords. Instructions are provided to the model that allow the model to ask for the original content of the chat history summary which is obtained from the lookup table and provided to the model.
What exactly they claim (claim 1)
1. A system comprising: a processor; and a memory that stores a program that is configured to be executed by the processor, wherein the program comprises instructions to perform acts that: obtain a user query for a machine learning model to perform a task, wherein the machine learning model is limited to a fixed-size context window; maintain, in the memory, a chat history with the machine learning model, wherein the chat history comprises an ordered sequence of prompts and model responses; in response to determining that the fixed-size of the context window has been exceeded: generate a summarized chat history; replace, in the memory, the chat history with the summarized chat history; and create a first prompt, wherein the first prompt comprises the summarized chat history and the user query; transmit the first prompt to the machine learning model; receive from the machine learning model a first response to the first prompt; and output the first response in a user interface.
Facts
- Company
- Microsoft (MICROSOFT TECHNOLOGY LICENSING, LLC)
- Official title
- SUMMARIZING CHAT SESSIONS TO MANAGE INFORMATION LOSS
- Publication number
- US20260281066A1
- Application number
- 19680969
- Filed
- May 18, 2026
- Published
- Sep 17, 2026
- Inventors
- 4
- Claims
- 20
Classified under
H04L 51/02 Electricity & electronicsG06F 16/345 Physics & computingH04L 51/216 Electricity & electronicsMore from Microsoft in AI & software
- Microsoft patents method to check AI answers against verified facts Oct 20, 2026
- Microsoft patents AI that presents PowerPoint slides for you Sep 3, 2026
- Microsoft patents method to check if LLM answers are based on real data Sep 10, 2026
- Microsoft patents system to blend multiple AI agents into one chat Oct 1, 2026
- Microsoft patents system that answers questions using your screen history Oct 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.