Microsoft patents shared memory for AI chips to cut data copying
Microsoft describes a system where multiple AI processors access a single copy of data or model parameters through a shared memory hub instead of each keeping its own separate copy. A dedicated hardware component translates requests from the processors so they think they are using their own private memory, while actually sharing the same stored information. This approach aims to speed up data access and reduce the amount of memory needed during AI training.
In the filing’s own words
Techniques for shared memory spaces in data and model parallelism are provided to improve memory efficiency and memory access speed. A shared memory space may be established at a host system or in a hardware memory agent. The shared memory may store training data or model parameters for an artificial intelligence model at a memory address in one or more memory circuits. Data for the artificial intelligence model may be processed across a plurality of artificial intelligence accelerators using the training data or the model parameters of the shared memory space. That is, multiple accelerators access one copy of the data from the shared memory space instead of accessing their own separate memory space.
What exactly they claim (claim 1)
1. A computer system comprising: one or more processors of a host device that stores model parameters or training data for an artificial intelligence model; a plurality of artificial intelligence accelerators that process data for the artificial intelligence model using the model parameters or the training data, each of the plurality of artificial intelligence accelerators comprising driver logic configured to issue memory requests to a corresponding one of multiple different dedicated host memory spaces within the host device; a memory agent device separate from the host device that serves as a communication interface between the one or more processors of the host device and the plurality of artificial intelligence accelerators, the memory agent device configured to: provide a shared memory space to the plurality of artificial intelligence accelerators, the shared memory space being mapped, via a many-to-one mapping, to the multiple different dedicated host memory spaces of the plurality of artificial intelligence accelerators, wherein the plurality of artificial intelligence accelerators are not configured to access the shared memory space; cache, at an address block of the shared memory space, a subset of the training data or model parameters obtained from the host device; translate requests from the plurality of artificial intelligence accelerators directed to the multiple different dedicated host memory spaces into accesses to the shared memory space; and return the subset of the training data or model parameters from the address block of the shared memory space when responding to the requests from the plurality of artificial intelligence accelerators that are directed to the multiple different dedicated host memory spaces within the host device such that the plurality of artificial intelligence accelerators operate as if they are accessing the multiple different dedicated host memory spaces while the subset of the training data or model parameters is shared via the shared memory space.
Facts
- Company
- Microsoft (Microsoft Technology Licensing, LLC)
- Official title
- Shared memory spaces in data and model parallelism
- Patent number
- US12770050B2
- Application number
- 17063329
- Filed
- Oct 5, 2020
- Granted
- Oct 20, 2026
- Inventors
- 4
- Claims
- 16
Classified under
G06N 20/00 Physics & computingG06F 13/28 Physics & computingMore from Microsoft in Chips & hardware
- Microsoft patents chips with liquid cooling built inside Oct 1, 2026
- Microsoft patents chip design that guesses code steps ahead Sep 8, 2026
- Microsoft patents 3D glass storage using laser light patterns Sep 8, 2026
- Microsoft patents hand-held radar reflectors for tracking movement Sep 8, 2026
- Microsoft patents video boost that reuses old frames to save power Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.