/Patent Tracker
MicrosoftPatent applicationChips & hardwareInterest 6/10

Microsoft patents splitting AI tasks between secure and memory chips

Microsoft describes a method for running artificial intelligence models by splitting the work between a standard processor and a specialized memory chip. The system keeps the most critical calculations on the secure processor while moving the rest to the memory chip to improve efficiency.

Why it mattersThis approach could help companies run complex AI models more securely and efficiently by balancing performance with data protection.

In the filing’s own words

The technology described herein is related to a hybrid neural network that divides operations of a neural network layer between a trusted environment, which may be a processor, and an untrusted environment, which may be a processor in memory (PIM). In an aspect, a first portion of matrix components in a layer operate on a processor and a second portion of matrix components in the layer operate on the PIM. The layer output is generated by combining the result produced by the matrix components in the trusted and untrusted environments. The technology described herein identifies matrix components in a layer that provide the largest contribution to an accurate model result. The matrix components that make the largest contribution may be described as the significant components and be deployed for use in the trusted processor environment. The rest of the layer may be deployed to the PIM.

What exactly they claim (claim 1)

1. One or more computer storage media comprising computer-executable instructions that when executed by computing device performs a method of operating a neural network in hybrid deployment, the method comprising: receiving, at a partial processor-side neural network operating on a processor, a processor in memory (PIM)-side output for a first layer of the neural network generated based on a layer input, wherein the PIM-side output is generated by a partial PIM-side neural network that includes less than 100% of all matrix components associated with the first layer, wherein the PIM-side neural network operates on a PIM that does not include the processor; receiving, at the partial processor-side neural network, the layer input; generating, at the partial processor-side neural network, a processor-side output for the first layer using the layer input, wherein the processor-side output is generated by the partial processor-side neural network that includes less than 20% of all matrix components associated with the first layer; generating, at the partial processor-side neural network, a final output for the first layer based on the processor-side output and the PIM-side output; and communicating the final output for the first layer to the partial PIM-side neural network.

Facts

Company
Microsoft (Microsoft Technology Licensing, LLC)
Official title
HYBRID PROCESSING IN MEMORY MACHINE-LEARNING MODEL WITH LAYER DECOMPOSITION
Publication number
US20260267512A1
Application number
19069815
Filed
Mar 4, 2025
Published
Sep 10, 2026
Inventors
4
Claims
20

Classified under

G06F 3/0611 Physics & computingG06F 3/0659 Physics & computingG06F 3/0673 Physics & computingG06N 3/0455 Physics & computing
Read the full filing (official PDF) ↗ Open in the Patent Tracker

More from Microsoft in Chips & hardware

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.