Nvidia patents faster math for AI models that compare pairs of items
Nvidia describes a method for speeding up a specific type of neural network calculation known as a triangular multiplicative update. The system uses specialized processor code to rearrange data in memory while performing math, avoiding extra steps that usually slow down processing.
In the filing’s own words
Apparatuses, systems, and techniques are to perform a triangular multiplicative update using one or more processors that execute fused accelerator kernels. An input tensor representing pairwise features is received in a first ordered memory layout that includes a first pair-index axis, a second pair-index axis, and a feature-channel axis. A fused accelerator kernel computes a first tensor and a second tensor from the input tensor and stores the first tensor and the second tensor in a second ordered memory layout in which the feature-channel axis precedes the first pair-index axis and the second pair-index axis. The storing performs a transpose by writing tensor values directly to memory addresses corresponding to the second ordered memory layout. A tensor contraction then computes a triangular multiplicative update tensor by summing over an intermediate index while preserving the feature-channel axis. In some embodiments, an output fused layer-normalization-and-transpose kernel restores a pair-feature layout after the tensor contraction.
What exactly they claim (claim 1)
1. One or more processors, comprising: circuitry to: receive an input tensor representing features and stored in memory according to a first ordered memory layout, the input tensor including a first pair-index axis, a second pair-index axis, and a feature-channel axis; execute a fused accelerator kernel configured to perform a plurality of neural network operations including computing, from the input tensor, a first tensor and a second tensor, and storing the first tensor and the second tensor in memory according to a second ordered memory layout, wherein the storing performs a transpose operation that reorders axes from the first ordered memory layout to the second ordered memory layout, and wherein, in the second ordered memory layout, the feature-channel axis precedes the first pair-index axis and the second pair-index axis; and compute a triangular multiplicative update tensor by performing a tensor contraction between the first tensor and the second tensor that sums over an intermediate index, wherein the feature-channel axis is preserved by the tensor contraction and is not summed over the intermediate index.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- TRIANGULAR UPDATE NEURAL NETWORK
- Publication number
- US20260359641A1
- Application number
- 19672609
- Filed
- May 9, 2026
- Published
- Nov 12, 2026
- Inventors
- 6
- Claims
- 20
Classified under
G06N 3/084 Physics & computingG06N 3/048 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents chip that turns text into video using noise reduction Nov 5, 2026
- Nvidia patents system that translates specific speakers in group conversations Nov 12, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.
