Nvidia patents method to split math tasks across 2D GPU grids
Nvidia describes a technique for performing weighted-averaging calculations on machine learning models by distributing data blocks across a two-dimensional grid of GPUs. The system processes these blocks in steps to calculate local contributions and normalization values, which are then combined to produce a final output.
In the filing’s own words
Apparatuses, systems, and techniques to perform distributed weighted-averaging computations for machine-learning models. In at least one embodiment, aligned distributed blocks including value blocks derived from a sequence-alignment representation and weight blocks derived from a pair-representation tensor are used to compute a local weighted-averaging contribution and local normalization information, and a local normalization state and a local output block are updated based on the local weighted-averaging contribution and the local normalization information.
What exactly they claim (claim 1)
1. One or more processors, comprising: circuitry to: receive, during successive communication steps on a GPU of a two-dimensional GPU mesh, aligned distributed value blocks derived from a sequence-alignment representation and aligned distributed weight blocks derived from a pair-representation tensor, the aligned distributed value blocks and the aligned distributed weight blocks being for the same weighted-averaging computation over a shared sequence-position axis and being distributed according to different sharding arrangements on the two-dimensional GPU mesh; compute, from currently aligned ones of the aligned distributed value blocks and the aligned distributed weight blocks, a local weighted-averaging contribution and local normalization information; update a local normalization state across the successive communication steps based on the local normalization information; and generate a local output block for the weighted-averaging computation based on the local weighted-averaging contribution and the local normalization state.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- DISTRIBUTED PAIR WEIGHTED AVERAGING
- Publication number
- US20260359924A1
- Application number
- 19672628
- Filed
- May 9, 2026
- Published
- Nov 12, 2026
- Inventors
- 8
- Claims
- 20
Classified under
G06T 1/20 Physics & computingMore from Nvidia in Chips & hardware
- Nvidia stacks memory directly on chips to boost data speed Nov 10, 2026
- Nvidia patents method to skip zero data in neural network math Oct 13, 2026
- Nvidia patents hardware to speed up motion blur in ray tracing Oct 13, 2026
- Nvidia patents method to sync data across split processor chips Sep 17, 2026
- Nvidia patents chip math trick to speed up AI calculations Oct 8, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.
