Nvidia patents method to trim neural networks during training
Nvidia describes a technique for removing unnecessary parts of a neural network while it is still being trained. The system calculates how important each part is and checks if the network is stable enough to safely cut those parts before training is fully finished.
In the filing’s own words
Apparatuses, systems, and techniques to remove one or more nodes of a neural network. In at least one embodiment, one or more nodes of a neural network are removed, based on, for example, whether the one or more nodes are likely to affect performance of the neural network.
What exactly they claim (claim 1)
1. A processor, comprising: one or more circuits to: identify a sub-network of a neural network by at least: calculating a set of importance scores for a plurality of nodes of the neural network; and determining that importance scores for a plurality of nodes of a sub-network of the neural network satisfies one or more criteria; calculate, for the sub-network of the neural network, a stability value that indicates a measure of potential change to parameters of the plurality of nodes of the sub-network as a result of one or more subsequent training epochs for the neural network; determine to perform early pruning of the neural network before completing the one or more subsequent training epochs for the neural network based, at least in part, on a determination that the stability value of the sub-network is above a pre-defined threshold; and remove one or more nodes of the neural network based, at least in part, on respective importance scores of the one or more nodes of the neural network to perform the early pruning of the neural network before completing the one or more subsequent training epochs for the neural network.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- Pruning neural networks
- Patent number
- US12792915B2
- Application number
- 17201768
- Filed
- Mar 15, 2021
- Granted
- Nov 10, 2026
- Inventors
- 4
- Claims
- 25
Classified under
G06N 3/082 Physics & computingG06F 11/302 Physics & computingG06N 3/045 Physics & computingG06N 3/063 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents chip that turns text into video using noise reduction Nov 5, 2026
- Nvidia patents system that translates specific speakers in group conversations Nov 12, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.