NvidiaPatent applicationChips & hardwareInterest 3/10

Nvidia patents method to split math tasks across 2D GPU grids

Patent drawing
Patent drawing from the official filing.

Nvidia describes a technique for performing weighted-averaging calculations on machine learning models by distributing data blocks across a two-dimensional grid of GPUs. The system processes these blocks in steps to calculate local contributions and normalization values, which are then combined to produce a final output.

Why it mattersThis filing details a specific way to organize data processing on high-end graphics cards, which could help improve the efficiency of training large AI models.

In the filing’s own words

Apparatuses, systems, and techniques to perform distributed weighted-averaging computations for machine-learning models. In at least one embodiment, aligned distributed blocks including value blocks derived from a sequence-alignment representation and weight blocks derived from a pair-representation tensor are used to compute a local weighted-averaging contribution and local normalization information, and a local normalization state and a local output block are updated based on the local weighted-averaging contribution and the local normalization information.

What exactly they claim (claim 1)

1. One or more processors, comprising: circuitry to: receive, during successive communication steps on a GPU of a two-dimensional GPU mesh, aligned distributed value blocks derived from a sequence-alignment representation and aligned distributed weight blocks derived from a pair-representation tensor, the aligned distributed value blocks and the aligned distributed weight blocks being for the same weighted-averaging computation over a shared sequence-position axis and being distributed according to different sharding arrangements on the two-dimensional GPU mesh; compute, from currently aligned ones of the aligned distributed value blocks and the aligned distributed weight blocks, a local weighted-averaging contribution and local normalization information; update a local normalization state across the successive communication steps based on the local normalization information; and generate a local output block for the weighted-averaging computation based on the local weighted-averaging contribution and the local normalization state.

Facts

Company
Nvidia (NVIDIA Corporation)
Official title
DISTRIBUTED PAIR WEIGHTED AVERAGING
Publication number
US20260359924A1
Application number
19672628
Filed
May 9, 2026
Published
Nov 12, 2026
Inventors
8
Claims
20

Classified under

G06T 1/20 Physics & computing

More from Nvidia in Chips & hardware

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.