Nvidia patents chip design that speeds up AI calculations by cutting data travel
Nvidia describes a computer chip layout designed to run AI models faster by keeping data close to where it is used. The design uses small, specialized blocks called compute tiles that work in parallel to handle math tasks, reducing the need to move data across the entire chip.
In the filing’s own words
A low-latency processing system reduces data movement, improves memory bandwidth utilization, and enables, e.g., scalable, low-latency neural network inference. Compute tiles are configured to efficiently execute matrix operations in parallel. Each compute tile includes an array of basic units equipped with local SRAM, vector registers, and multiply-accumulate (MAC) units, enabling localized computation to minimize data movement and reduce latency. IMLP tiles efficiently perform data format conversions and reduction operations as part of intermediate layer processing. A configurable network transmits data directly from source to destination register(s) within compute and IMLP tiles. Connections are, e.g., pre-configured for one or more cycles of neural network inference.
What exactly they claim (claim 1)
1. A method for performing a tensor operation, comprising: generating, at a plurality of compute tiles of a tile group, a partial result to provide a plurality of partial results; transmitting, via a gather network, the plurality of partial results to an inter-matrix layer processing (IMLP) tile; generating, by the IMLP tile and based on the plurality of partial results, a local reduction result; transmitting, via a reduction network, the local reduction result from the IMLP tile to a further IMLP tile; receiving, by the IMLP tile via the reduction network, a global reduction result; and distributing, via a distribution network, a final result to one or more further compute tiles, wherein the final result comprises one or more subvectors, values of the one or more subvectors being determined based on the global reduction result.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- LOW-LATENCY COMPUTE ARCHITECTURES AND USE FOR NEURAL NETWORK INFERENCE
- Publication number
- US20260329803A1
- Application number
- 19372189
- Filed
- Oct 28, 2025
- Published
- Oct 22, 2026
- Inventors
- 4
- Claims
- 20
Classified under
G06F 15/17381 Physics & computingG06F 15/17362 Physics & computingMore from Nvidia in Chips & hardware
- Nvidia patents method to skip zero data in neural network math Oct 13, 2026
- Nvidia patents hardware to speed up motion blur in ray tracing Oct 13, 2026
- Nvidia patents method to sync data across split processor chips Sep 17, 2026
- Nvidia patents chip math trick to speed up AI calculations Oct 8, 2026
- Nvidia patents data center cooling that uses cryogenic fluids Oct 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.