Nvidia stacks memory directly on chips to boost data speed
Nvidia describes a design where memory chips are stacked directly on top of a processor die. This vertical arrangement connects memory tiles directly to processing units, allowing data to move much faster and using less energy than traditional side-by-side layouts.
In the filing’s own words
Embodiments of the present disclosure relate to memory stacked on processor for high bandwidth. Systems and methods are disclosed for providing a one-level memory for a processing system by stacking bulk memory on a processor die. In an embodiment, one or more memory dies are stacked on the processor die. The processor die includes multiple processing tiles, where each tile includes a processing unit, mapper, and tile network. Each memory die includes multiple memory tiles. The processing tile is coupled to each memory tile that is above or below the processing tile. The vertically aligned memory tiles comprise the local memory block for the processing tile. The ratio of memory bandwidth (byte) to floating-point operation (B:F) may improve 50× for accessing the local memory block compared with conventional memory. Additionally, the energy consumed to transfer each bit may be reduced by 10×.
What exactly they claim (claim 1)
1. A device, comprising: a stack of dies including a processor die comprising a plurality of processing tiles and at least one memory die, each memory die comprising a plurality of memory tiles, wherein dies in the stack of dies are aligned in a first dimension and stacked in a second dimension; and conductive paths between each processing tile and a corresponding memory tile in each memory die of the at least one memory die, wherein the corresponding memory tile is stacked on the processing tile in the second dimension, the conductive paths comprise a through-die via structure in the second dimension that is fabricated within each one of the at least one memory die for communication between each processing tile and the corresponding memory tile, and the through-die via structure is fabricated between first contacts positioned within sense amps of each memory tile on a first surface of the at least one memory die and second contacts positioned on a second surface of the at least one memory die, wherein the first surface is a front surface of the at least one memory die and the second surface is a back surface of the at least one memory die or the first surface is the back surface and the second surface is the front surface.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- Memory stacked on processor for high bandwidth
- Patent number
- US12796536B2
- Application number
- 17683290
- Filed
- Feb 28, 2022
- Granted
- Nov 10, 2026
- Inventors
- 4
- Claims
- 19
Classified under
H10W 90/00 Electricity & electronicsH10B 41/00 Electricity & electronicsH10W 90/26 Electricity & electronicsMore from Nvidia in Chips & hardware
- Nvidia patents method to skip zero data in neural network math Oct 13, 2026
- Nvidia patents hardware to speed up motion blur in ray tracing Oct 13, 2026
- Nvidia patents method to sync data across split processor chips Sep 17, 2026
- Nvidia patents chip math trick to speed up AI calculations Oct 8, 2026
- Nvidia patents data center cooling that uses cryogenic fluids Oct 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.