NvidiaPatent applicationAI & softwareInterest 4/10

Nvidia patents faster math for AI models that compare pairs of items

Patent drawing
Patent drawing from the official filing.

Nvidia describes a method for speeding up a specific type of neural network calculation known as a triangular multiplicative update. The system uses specialized processor code to rearrange data in memory while performing math, avoiding extra steps that usually slow down processing.

Why it mattersThis improves the efficiency of running certain AI models on Nvidia chips, potentially making them faster and less power-hungry for tasks involving complex data comparisons.

In the filing’s own words

Apparatuses, systems, and techniques are to perform a triangular multiplicative update using one or more processors that execute fused accelerator kernels. An input tensor representing pairwise features is received in a first ordered memory layout that includes a first pair-index axis, a second pair-index axis, and a feature-channel axis. A fused accelerator kernel computes a first tensor and a second tensor from the input tensor and stores the first tensor and the second tensor in a second ordered memory layout in which the feature-channel axis precedes the first pair-index axis and the second pair-index axis. The storing performs a transpose by writing tensor values directly to memory addresses corresponding to the second ordered memory layout. A tensor contraction then computes a triangular multiplicative update tensor by summing over an intermediate index while preserving the feature-channel axis. In some embodiments, an output fused layer-normalization-and-transpose kernel restores a pair-feature layout after the tensor contraction.

What exactly they claim (claim 1)

1. One or more processors, comprising: circuitry to: receive an input tensor representing features and stored in memory according to a first ordered memory layout, the input tensor including a first pair-index axis, a second pair-index axis, and a feature-channel axis; execute a fused accelerator kernel configured to perform a plurality of neural network operations including computing, from the input tensor, a first tensor and a second tensor, and storing the first tensor and the second tensor in memory according to a second ordered memory layout, wherein the storing performs a transpose operation that reorders axes from the first ordered memory layout to the second ordered memory layout, and wherein, in the second ordered memory layout, the feature-channel axis precedes the first pair-index axis and the second pair-index axis; and compute a triangular multiplicative update tensor by performing a tensor contraction between the first tensor and the second tensor that sums over an intermediate index, wherein the feature-channel axis is preserved by the tensor contraction and is not summed over the intermediate index.

Facts

Company
Nvidia (NVIDIA Corporation)
Official title
TRIANGULAR UPDATE NEURAL NETWORK
Publication number
US20260359641A1
Application number
19672609
Filed
May 9, 2026
Published
Nov 12, 2026
Inventors
6
Claims
20

Classified under

G06N 3/084 Physics & computingG06N 3/048 Physics & computing

More from Nvidia in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.