MicrosoftPatent applicationAI & softwareInterest 6/10

Microsoft patents method to tweak AI models without retraining them

Patent drawing
Patent drawing from the official filing.

Microsoft describes a way to adjust large language models for specific tasks by adding small, adjustable layers to the existing system. Instead of changing the main model's core settings, this method uses smaller mathematical shortcuts that are easier to update and merge back into the original design.

Why it mattersThis technique could make it faster and cheaper for companies to customize AI tools for specific jobs without starting the training process from scratch.

In the filing’s own words

A computer implemented method obtains neural network-based model base model weight matrices for each of multiple neural network layers. First low-rank factorization matrices are added to corresponding base model weight matrices to form a first domain model. The low-rank factorization matrices are treated as trainable parameters. The first domain model is trained with first domain specific training data without modifying base model weight matrices.

What exactly they claim (claim 1)

1. A computer-implemented method comprising: accessing a transformer-based, general-domain-trained language model having a plurality of layers, each layer comprising a frozen base weight matrix of dimension d; obtaining, for each layer, a pair of low-rank adaptation matrices previously trained on domain-specific data to adapt the language model, the pair having an intermediate dimension r less than d; merging each frozen base weight matrix with a product of its corresponding pair to form an adapted weight matrix; and performing inference on domain input using the adapted weight matrices.

Facts

Company
Microsoft (Microsoft Technology Licensing, LLC)
Official title
Low-Rank Adaptation of Neural Network Models
Publication number
US20260359642A1
Application number
19748900
Filed
Jul 21, 2026
Published
Nov 12, 2026
Inventors
5
Claims
20

Classified under

G06N 3/084 Physics & computingG06N 3/10 Physics & computingG06N 5/04 Physics & computing

More from Microsoft in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.