GooglePatent applicationAI & softwareInterest 6/10

Google patents faster AI attention that cuts computing costs

Patent drawing
Patent drawing from the official filing.

This filing describes a method for machine learning models to process data more efficiently by changing how they calculate attention. Instead of comparing every single piece of data to every other piece, the system groups similar data points together and calculates a summary for each group. This allows the model to consider the full context of the input while using less memory and processing power.

Why it mattersThis could make large language models and other AI tools run faster and cheaper, potentially allowing them to run on smaller devices or handle longer texts without slowing down.

In the filing’s own words

The present disclosure is directed to machine learning model architectures which provide full attention capability in each attention head while maintaining low computation and memory complexity. Specifically, according to one aspect of the present disclosure, example attention models provided herein can treat the self-attention mechanism as a conditional expectation over embeddings at each location and approximate the conditional distribution with a structured factorization. Each location can attend to all other locations, either via direct attention, or through indirect attention to group representations, which are again conditional expectations of embeddings from corresponding local regions.

What exactly they claim (claim 1)

1. A computing system for performing an attention mechanism with reduced computational requirements, the computing system comprising: one or more processors; and one or more non-transitory computer-readable media that collectively store a machine-learned attention model configured to receive and process a model input to generate a model output, wherein the machine-learned attention model comprises one or more attention layers, wherein at least one of the attention layers comprises one or more attention heads, and wherein at least one of the attention heads is configured to: receive a sequence of input data elements; and apply a structured attention pattern to the sequence of input data elements to generate a sequence of output data elements; wherein, for each input data element in the sequence of input data elements, the structured attention pattern specifies one or more locations of direct expectation and one or more groups of locations of local expectation; and wherein, for each of the one or more groups of locations with local expectation, the at least one of the attention heads is configured to: determine a single group probability for the group of locations; and determine an individual local expectation for each location in the group of locations.

Facts

Company
Google (Google LLC)
Official title
Full Attention with Sparse Computation Cost
Publication number
US20260359561A1
Application number
19733438
Filed
Jul 7, 2026
Published
Nov 12, 2026
Inventors
6
Claims
20

Classified under

G06F 40/20 Physics & computingG06N 20/00 Physics & computing

More from Google in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.