Google patents faster AI attention that cuts computing costs
This filing describes a method for machine learning models to process data more efficiently by changing how they calculate attention. Instead of comparing every single piece of data to every other piece, the system groups similar data points together and calculates a summary for each group. This allows the model to consider the full context of the input while using less memory and processing power.
In the filing’s own words
The present disclosure is directed to machine learning model architectures which provide full attention capability in each attention head while maintaining low computation and memory complexity. Specifically, according to one aspect of the present disclosure, example attention models provided herein can treat the self-attention mechanism as a conditional expectation over embeddings at each location and approximate the conditional distribution with a structured factorization. Each location can attend to all other locations, either via direct attention, or through indirect attention to group representations, which are again conditional expectations of embeddings from corresponding local regions.
What exactly they claim (claim 1)
1. A computing system for performing an attention mechanism with reduced computational requirements, the computing system comprising: one or more processors; and one or more non-transitory computer-readable media that collectively store a machine-learned attention model configured to receive and process a model input to generate a model output, wherein the machine-learned attention model comprises one or more attention layers, wherein at least one of the attention layers comprises one or more attention heads, and wherein at least one of the attention heads is configured to: receive a sequence of input data elements; and apply a structured attention pattern to the sequence of input data elements to generate a sequence of output data elements; wherein, for each input data element in the sequence of input data elements, the structured attention pattern specifies one or more locations of direct expectation and one or more groups of locations of local expectation; and wherein, for each of the one or more groups of locations with local expectation, the at least one of the attention heads is configured to: determine a single group probability for the group of locations; and determine an individual local expectation for each location in the group of locations.
Facts
- Company
- Google (Google LLC)
- Official title
- Full Attention with Sparse Computation Cost
- Publication number
- US20260359561A1
- Application number
- 19733438
- Filed
- Jul 7, 2026
- Published
- Nov 12, 2026
- Inventors
- 6
- Claims
- 20
Classified under
G06F 40/20 Physics & computingG06N 20/00 Physics & computingMore from Google in AI & software
- Google patents video search that answers questions using AI descriptions Sep 15, 2026
- Google patents photo tool that removes people and their gear Sep 15, 2026
- Google patents AI that writes live meeting notes and summaries Sep 15, 2026
- Google patents method for LLMs to self-correct and learn from mistakes Oct 27, 2026
- Google patents search results that mix real clips with AI-made video Nov 10, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.
