NvidiaGranted patentChips & hardwareInterest 8/10

Nvidia stacks memory directly on chips to boost data speed

Nvidia describes a design where memory chips are stacked directly on top of a processor die. This vertical arrangement connects memory tiles directly to processing units, allowing data to move much faster and using less energy than traditional side-by-side layouts.

Why it mattersThis approach could make high-performance computing chips more efficient by reducing the distance data must travel, which is crucial for tasks like AI training and scientific simulations.

In the filing’s own words

Embodiments of the present disclosure relate to memory stacked on processor for high bandwidth. Systems and methods are disclosed for providing a one-level memory for a processing system by stacking bulk memory on a processor die. In an embodiment, one or more memory dies are stacked on the processor die. The processor die includes multiple processing tiles, where each tile includes a processing unit, mapper, and tile network. Each memory die includes multiple memory tiles. The processing tile is coupled to each memory tile that is above or below the processing tile. The vertically aligned memory tiles comprise the local memory block for the processing tile. The ratio of memory bandwidth (byte) to floating-point operation (B:F) may improve 50× for accessing the local memory block compared with conventional memory. Additionally, the energy consumed to transfer each bit may be reduced by 10×.

What exactly they claim (claim 1)

1. A device, comprising: a stack of dies including a processor die comprising a plurality of processing tiles and at least one memory die, each memory die comprising a plurality of memory tiles, wherein dies in the stack of dies are aligned in a first dimension and stacked in a second dimension; and conductive paths between each processing tile and a corresponding memory tile in each memory die of the at least one memory die, wherein the corresponding memory tile is stacked on the processing tile in the second dimension, the conductive paths comprise a through-die via structure in the second dimension that is fabricated within each one of the at least one memory die for communication between each processing tile and the corresponding memory tile, and the through-die via structure is fabricated between first contacts positioned within sense amps of each memory tile on a first surface of the at least one memory die and second contacts positioned on a second surface of the at least one memory die, wherein the first surface is a front surface of the at least one memory die and the second surface is a back surface of the at least one memory die or the first surface is the back surface and the second surface is the front surface.

Facts

Company
Nvidia (NVIDIA Corporation)
Official title
Memory stacked on processor for high bandwidth
Patent number
US12796536B2
Application number
17683290
Filed
Feb 28, 2022
Granted
Nov 10, 2026
Inventors
4
Claims
19

Classified under

H10W 90/00 Electricity & electronicsH10B 41/00 Electricity & electronicsH10W 90/26 Electricity & electronics

More from Nvidia in Chips & hardware

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.