/Patent Tracker
NvidiaPatent applicationChips & hardwareInterest 6/10

Nvidia patents compiler trick to skip useless math in AI chips

Nvidia describes a software tool that helps computer chips work faster by ignoring parts of data they don't need. The system figures out which sections of a large data grid can be skipped, saving time and energy during processing.

Why it mattersThis could make AI calculations run more efficiently by reducing unnecessary work, which is a common goal in chip design.

In the filing’s own words

In various examples, apparatuses, systems, processors, methods, and/or machine-readable mediums are used to generate bounding functions for a mask predicate, and to generate instructions to skip computation of values corresponding to one or more portions of an attention matrix masked by the mask predicate. In at least one embodiment, a compiler is used to analyze arbitrary and/or user-defined masks, and to generate bounding functions that may be used by an executing kernel to identify which tiles of an attention matrix can be skipped at runtime.

What exactly they claim (claim 1)

1. One or more processors comprising processing circuitry to: implement a compiler to generate instructions that, when executed by at least one processor, cause the at least one processor to skip computation of one or more masked values in one or more masked portions of at least one data structure representing one or more attention matrices that are masked by a mask predicate, the compiler generating the instructions by at least: deriving a plurality of elements based at least on the mask predicate; determining pairs of upper and lower bounding expressions corresponding to the plurality of elements; generating left and right bounding functions using the pairs of upper and lower bounding expressions corresponding to the plurality of elements; and using the left and right bounding functions to generate the instructions.

Facts

Company
Nvidia (NVIDIA Corporation)
Official title
COMPILER TO DETERMINE BOUNDS OF AN ATTENTION MASK FOR TILE-LEVEL COMPUTATION SKIPPING
Publication number
US20260259730A1
Application number
19206908
Filed
May 13, 2025
Published
Sep 3, 2026
Inventors
2
Claims
20

Classified under

G06F 9/30032 Physics & computingG06F 9/30029 Physics & computingG06F 9/30038 Physics & computingG06F 9/30069 Physics & computing
Read the full filing (official PDF) ↗ Open in the Patent Tracker

More from Nvidia in Chips & hardware

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.