/Patent Tracker
GoogleGranted patentAI & softwareInterest 4/10

Google patents method to compress and load parts of AI memory tables

This patent describes a way to store large lists of data used in machine learning models by compressing each entry individually. It allows a computer to find and unpack only the specific pieces of data it needs for a task, rather than loading the entire massive list into memory at once.

Why it mattersThis technique could help AI systems run more efficiently by using less memory, which is useful for developers building large-scale models.

In the filing’s own words

The present disclosure provides systems, methods, and computer program products for providing efficient embedding table storage and lookup in machine-learning models. A computer-implemented method may include obtaining an embedding table comprising a plurality of embeddings respectively associated with a corresponding index of the embedding table, compressing each particular embedding of the embedding table individually allowing each respective embedding of the embedding table to be decompressed independent of any other embedding in the embedding table, packing the embedding table comprising individually compressed embeddings with a machine-learning model, receiving an input to use for locating an embedding in the embedding table, determining a lookup value based on the input to search indexes of the embedding table, locating the embedding based on searching the indexes of the embedding table for the determined lookup value, and decompressing the located embedding independent of any other embedding in the embedding table.

What exactly they claim (claim 1)

1. A computer-implemented method, comprising: receiving, by a computing system comprising one or more processors and a memory, an input for processing using a machine-learned model; accessing, by the computing system, an embedding table associated with the machine-learned model, the embedding table comprising a plurality of embeddings; locating, by the computing system based on one or more lookup values associated with the input, one or more embeddings in the embedding table that respectively correspond to the one or more lookup values; loading, by the computing system and into the memory, and based on the locating of the one or more embeddings, the one or more embeddings without loading the entirety of the embedding table into the memory; and processing, by the computing system, the one or more embeddings in the memory using the machine-learned model.

Facts

Company
Google (Google LLC)
Official title
Efficient embedding table storage and lookup
Patent number
US12737341B2
Application number
19029909
Filed
Jan 17, 2025
Granted
Sep 15, 2026
Inventors
1
Claims
20

Classified under

G06F 16/2282 Physics & computingG06F 16/215 Physics & computingG06F 16/2255 Physics & computingG06F 16/3347 Physics & computingG06F 30/27 Physics & computingG06N 3/04 Physics & computing
Read the full filing (official PDF) ↗ Open in the Patent Tracker

More from Google in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.