Microsoft patents method for AI models to share data to run faster
Microsoft has filed a patent application for a way to make multiple AI models work together more efficiently by sharing internal data. The system allows a second AI model to reuse parts of the data stored by a first model, rather than calculating everything from scratch, which can speed up processing. It determines which specific pieces of data can be safely shared based on how well the second model performs.
In the filing’s own words
Example solutions for context sharing between GAI models includes: identifying a pair of GAI models including a first model and a second model; generating a first key-value (KV) cache of the first model; populating a second KV cache of the second model with one or more layers of the first KV cache; recomputing one or more other layers of the second KV cache; computing an output performance score for the second model based on a baseline metric; and transmitting a reuse pattern to a host device for use in context sharing between an instance of first model and an instance of second model, the reuse pattern identifying the plurality of first KV cache layers to be reused, thereby causing the host device to generate a local KV cache for the instance of second model by reusing the plurality of first KV cache layers from a KV cache of first model.
What exactly they claim (claim 1)
1. A context sharing system comprising: a processor; and a memory comprising computer-readable instructions, the processor, the memory and the computer-readable instructions configured to cause the processor to: identify a first model and a second model; generate a first key-value (KV) cache of the first model, the first KV cache including a plurality of first KV cache layers; populate a second KV cache of the second model with one or more layers of the first KV cache layers; recompute one or more other layers of the second KV cache; generate an output performance score for the second model based on comparison between (a) an output generated by the second model while using the second KV cache and (b) a baseline metric; and configure a host device with a reuse pattern for context sharing between an instance of the first model and an instance of the second model, the reuse pattern identifying the plurality of first KV cache layers to be reused, thereby causing the host device to generate a local KV cache for the instance of the second model by reusing the plurality of first KV cache layers from a KV cache of the instance of the first model.
Facts
- Company
- Microsoft (Microsoft Technology Licensing, LLC)
- Official title
- EFFICIENT MODEL AGENT COMMUNICATION WITH CACHE SHARING
- Publication number
- US20260259830A1
- Application number
- 19067128
- Filed
- Feb 28, 2025
- Published
- Sep 3, 2026
- Inventors
- 4
- Claims
- 20
Classified under
G06F 12/084 Physics & computingG06F 16/3349 Physics & computingG06F 40/284 Physics & computingMore from Microsoft in AI & software
- Microsoft patents method to check AI answers against verified facts Oct 20, 2026
- Microsoft patents AI that presents PowerPoint slides for you Sep 3, 2026
- Microsoft patents method to check if LLM answers are based on real data Sep 10, 2026
- Microsoft patents system to blend multiple AI agents into one chat Oct 1, 2026
- Microsoft patents system that answers questions using your screen history Oct 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.