MicrosoftGranted patentChips & hardwareInterest 4/10

Microsoft patents chip design that splits neural network tasks across units

Patent drawing
Patent drawing from the official filing.

Microsoft describes a processor design that divides neural network calculations between a matrix vector unit and a chain of multifunction units. The system routes specific instructions to the matrix unit for heavy math or to the multifunction units for operations like adding or multiplying numbers.

Why it mattersThis design aims to make hardware more efficient at running AI models by assigning specific math tasks to dedicated parts of the chip.

In the filing’s own words

Processors and methods for neural network processing are provided. A method in a processor including a pipeline having a matrix vector unit (MVU), a first multifunction unit connected to receive an input from the MVU, a second multifunction unit connected to receive an output from the first multifunction unit, and a third multifunction unit connected to receive an output from the second multifunction unit is provided. The method includes decoding instructions including a first type of instruction for processing by only the MVU and a second type of instruction for processing by only one of the multifunction units. The method includes mapping a first instruction for processing by the matrix vector unit or to any one of the first multifunction unit, the second multifunction unit, or the third multifunction unit depending on whether the first instruction is the first type of instruction or the second type of instruction.

What exactly they claim (claim 1)

1. A method in a processor including a pipeline for processing instructions, the pipeline including a matrix vector circuitry and at least one multifunction circuitry, wherein the at least one multifunction circuitry is connected to receive an input from the matrix vector circuitry, and wherein the at least one multifunction circuitry comprises at least two blocks from among a pointwise addition block, a pointwise multiplication block, a sigmoid block, a hyperbolic tangent block, or a no-operation block, the method comprising: decoding received instructions, wherein a subset of the received instructions comprises a set of instructions including a first type of instruction for processing by only the matrix vector circuitry and a second type of instruction for processing by only the at least one multifunction circuitry; and mapping a first instruction for processing by the matrix vector circuitry or the at least one multifunction circuitry, depending on whether the first instruction is the first type of instruction or the second type of instruction.

Facts

Company
Microsoft (Microsoft Technology Licensing, LLC)
Official title
Multi-function unit for programmable hardware nodes for neural network processing
Patent number
US12801905B2
Application number
17409039
Filed
Aug 23, 2021
Granted
Nov 17, 2026
Inventors
3
Claims
20

Classified under

G06N 3/063 Physics & computingG06F 9/30098 Physics & computingG06F 9/3836 Physics & computingG06F 9/3867 Physics & computingG06N 3/04 Physics & computingG06F 17/16 Physics & computing

More from Microsoft in Chips & hardware

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.