Nvidia patents method to skip zero data in neural network math
This patent describes a way for processors to handle neural networks more efficiently by focusing only on the non-zero numbers in large data tables. It uses a specific pattern of empty spots to shrink the data before calculations, reducing the work the chip has to do.
In the filing’s own words
Approaches in accordance with various embodiments provide for the processing of sparse matrices for mathematical and programmatic operations. In particular, various embodiments enforce sparsity constraints for performing sparse matrix multiply-add instruction (MMA) operations. Deep neural networks can exhibit significant sparsity in the data used in operations, both in the activations and weights. The computational load can be reduced by excluding zero-valued data elements. A sparsity constraint is applied across all submatrices of a sparse matrix, providing fine-grained structured sparsity that is evenly distributed across the matrix. The matrix may then be compressed since a minimum number of elements of the matrix are known to have zero value. Matrix operations are then performed using these matrices.
What exactly they claim (claim 1)
1. A processor, comprising: one or more circuits to; identify a matrix multiplication operation subject to a sparsity constraint indicating a required minimum number of zero-values for a sparsity pattern repeated across one or more matrix operands of the matrix multiplication operation; and responsive to the one or more matrix operands satisfying the sparsity constraint, compress the one or more matrix operands of the matrix multiplication operation to exclude a plurality of zero-values from the one or more matrix operands according to the sparsity pattern.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- Managing data sparsity for neural networks
- Patent number
- US12762394B1
- Application number
- 17866031
- Filed
- Jul 15, 2022
- Granted
- Oct 13, 2026
- Inventors
- 9
- Claims
- 20
Classified under
G06N 3/08 Physics & computingG06F 17/16 Physics & computingG06N 3/04 Physics & computingMore from Nvidia in Chips & hardware
- Nvidia patents hardware to speed up motion blur in ray tracing Oct 13, 2026
- Nvidia patents method to sync data across split processor chips Sep 17, 2026
- Nvidia patents chip math trick to speed up AI calculations Oct 8, 2026
- Nvidia patents data center cooling that uses cryogenic fluids Oct 15, 2026
- Nvidia patents chip design that speeds up AI calculations by keeping data local Oct 22, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.