MicrosoftGranted patentAI & softwareInterest 6/10

Microsoft patents method to shrink AI models by linking data trimming and rounding

Patent drawing
Patent drawing from the official filing.

This patent describes a way to make deep neural networks smaller and faster by combining two techniques: sparsity and quantization. The system removes some connections (sparsity) and reduces the number of bits needed to store the remaining numbers (quantization), adjusting the rounding precision based on how many connections were removed.

Why it mattersThis could help developers run large AI models on devices with less memory or power, making advanced features more accessible on everyday hardware.

In the filing’s own words

A computing system is configured to implement a deep neural network comprising an input layer for receiving inputs applied to the deep neural network, an output layer for outputting inferences based on the received inputs, and a plurality of hidden layers interposed between the input layer and the output layer. A plurality of nodes selectively operate on the inputs to generate and cause outputting of the inferences, wherein operation of the nodes is controlled based on parameters of the deep neural network. A sparsity controller is configured to selectively apply a plurality of different sparsity states to control parameter density of the deep neural network. A quantization controller is configured to selectively quantize the parameters of the deep neural network in a manner that is sparsity-dependent, such that quantization applied to each parameter is based on which of the plurality of different sparsity states applies to the parameter.

What exactly they claim (claim 1)

1. A computing system, comprising: a logic subsystem of one or more logic devices; a storage subsystem of one or more storage devices having instructions stored thereon executable by the one or more logic devices of the logic subsystem to implement a deep neural network that includes: an input layer for receiving inputs applied to the deep neural network; an output layer for outputting inferences based on the received inputs; a plurality of hidden layers interposed between the input layer and the output layer; a plurality of nodes disposed within and interconnecting the input layer, output layer, and hidden layers, wherein the nodes selectively operate on the inputs to generate and cause outputting of the inferences, and wherein operation of the nodes is controlled based on a set of parameters of the deep neural network; a sparsity controller configured to selectively apply a plurality of different sparsity states to parameter tensors of the deep neural network to control parameter density of the set of parameters within the parameter tensors, wherein each sparsity state differs from one or more other sparsity states of the plurality of different sparsity states with respect to a proportion of the set of parameters that are sparsified within each parameter tensor of the deep neural network to which that sparsity state is applied by the sparsity controller; and a quantization controller configured to selectively quantize the set of parameters of the deep neural network in a manner that is sparsity-dependent, such that a degree of quantization applied by the quantization controller to at least one or more non-sparsified parameters of the set of parameters within a parameter tensor of the deep neural network is based on which of the plurality of different sparsity states is applied by the sparsity controller to that parameter tensor; wherein for each of the parameter tensors to which a sparsity state of the plurality of different sparsity states is applied by the sparsity controller, the degree of quantization applied by the quantization controller to the one or more non-sparsified parameters of that parameter tensor is inversely proportional to the proportion of the set of parameters that are sparsified within that parameter tensor; and wherein for each non-sparsified parameter that is quantized by the quantization controller, the degree of quantization defines a magnitude of a reduction in a quantity of bits that encode that non-sparsified parameter within memory of the computing system.

Facts

Company
Microsoft (Microsoft Technology Licensing, LLC)
Official title
Sparsity and quantization for deep neural networks
Patent number
US12801890B2
Application number
17664616
Filed
May 23, 2022
Granted
Nov 17, 2026
Inventors
6
Claims
20

Classified under

G06N 3/04 Physics & computingG06N 5/04 Physics & computing

More from Microsoft in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.