Microsoft patents method to shrink AI models by linking data trimming and rounding
This patent describes a way to make deep neural networks smaller and faster by combining two techniques: sparsity and quantization. The system removes some connections (sparsity) and reduces the number of bits needed to store the remaining numbers (quantization), adjusting the rounding precision based on how many connections were removed.
In the filing’s own words
A computing system is configured to implement a deep neural network comprising an input layer for receiving inputs applied to the deep neural network, an output layer for outputting inferences based on the received inputs, and a plurality of hidden layers interposed between the input layer and the output layer. A plurality of nodes selectively operate on the inputs to generate and cause outputting of the inferences, wherein operation of the nodes is controlled based on parameters of the deep neural network. A sparsity controller is configured to selectively apply a plurality of different sparsity states to control parameter density of the deep neural network. A quantization controller is configured to selectively quantize the parameters of the deep neural network in a manner that is sparsity-dependent, such that quantization applied to each parameter is based on which of the plurality of different sparsity states applies to the parameter.
What exactly they claim (claim 1)
1. A computing system, comprising: a logic subsystem of one or more logic devices; a storage subsystem of one or more storage devices having instructions stored thereon executable by the one or more logic devices of the logic subsystem to implement a deep neural network that includes: an input layer for receiving inputs applied to the deep neural network; an output layer for outputting inferences based on the received inputs; a plurality of hidden layers interposed between the input layer and the output layer; a plurality of nodes disposed within and interconnecting the input layer, output layer, and hidden layers, wherein the nodes selectively operate on the inputs to generate and cause outputting of the inferences, and wherein operation of the nodes is controlled based on a set of parameters of the deep neural network; a sparsity controller configured to selectively apply a plurality of different sparsity states to parameter tensors of the deep neural network to control parameter density of the set of parameters within the parameter tensors, wherein each sparsity state differs from one or more other sparsity states of the plurality of different sparsity states with respect to a proportion of the set of parameters that are sparsified within each parameter tensor of the deep neural network to which that sparsity state is applied by the sparsity controller; and a quantization controller configured to selectively quantize the set of parameters of the deep neural network in a manner that is sparsity-dependent, such that a degree of quantization applied by the quantization controller to at least one or more non-sparsified parameters of the set of parameters within a parameter tensor of the deep neural network is based on which of the plurality of different sparsity states is applied by the sparsity controller to that parameter tensor; wherein for each of the parameter tensors to which a sparsity state of the plurality of different sparsity states is applied by the sparsity controller, the degree of quantization applied by the quantization controller to the one or more non-sparsified parameters of that parameter tensor is inversely proportional to the proportion of the set of parameters that are sparsified within that parameter tensor; and wherein for each non-sparsified parameter that is quantized by the quantization controller, the degree of quantization defines a magnitude of a reduction in a quantity of bits that encode that non-sparsified parameter within memory of the computing system.
Facts
- Company
- Microsoft (Microsoft Technology Licensing, LLC)
- Official title
- Sparsity and quantization for deep neural networks
- Patent number
- US12801890B2
- Application number
- 17664616
- Filed
- May 23, 2022
- Granted
- Nov 17, 2026
- Inventors
- 6
- Claims
- 20
Classified under
G06N 3/04 Physics & computingG06N 5/04 Physics & computingMore from Microsoft in AI & software
- Microsoft patents method to check AI answers against verified facts Oct 20, 2026
- Microsoft patents AI that presents PowerPoint slides for you Sep 3, 2026
- Microsoft patents method to check if LLM answers are based on real data Sep 10, 2026
- Microsoft patents system to blend multiple AI agents into one chat Oct 1, 2026
- Microsoft patents system that answers questions using your screen history Oct 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.
