/Patent Tracker
GooglePatent applicationChips & hardwareInterest 4/10

Google patents chip trick to save power when AI guesses text

Google describes a method where a specialized chip checks if a piece of data is empty before running an AI model. If the data is empty, it skips the calculation and sends a signal, saving energy. If not, it calculates the most likely characters and sends only the top guesses to the main processor.

Why it mattersThis improves battery life in devices by preventing the AI chip from wasting energy on blank inputs.

In the filing’s own words

A hardware accelerator can receive, from a host processor, a slice of input data at a time-step. The hardware accelerator can process the input data using a machine learning model deployed on the hardware accelerator to compute a respective probability among multiple probabilities for each of multiple classes. The respective probability for each class being a likelihood that content in the slice belongs to the class. The hardware accelerator can determine, from the multiple probabilities, a preset number of highest probabilities for the slice of input data. The hardware accelerator can transmit the preset number of highest probabilities for the slice to the host processor. Related apparatus, systems, techniques and articles are also described.

What exactly they claim (claim 1)

1. A method comprising: receiving, by a hardware accelerator from a host processor, a slice of input data; determining, by the hardware accelerator, whether the slice of input data is an empty data slice; in response to determining that the slice of input data is an empty data slice, transmitting, by the hardware accelerator, a token to the host processor, wherein transmitting the token prevents the hardware accelerator from computing probabilities for the empty data slice; and in response to determining that the slice of input data is not an empty data slice: processing, by the hardware accelerator, the input data using a machine learning model deployed on the hardware accelerator to compute a respective probability of a plurality of probabilities for each of a plurality of classes, wherein the plurality of classes comprises a plurality of characters, wherein the respective probability for each class is a likelihood that content in the slice belongs to the class; determining, by the hardware accelerator and from the plurality of probabilities, a preset number of highest probabilities for the slice of input data; and transmitting, by the hardware accelerator, the preset number of highest probabilities for the slice to the host processor.

Facts

Company
Google (Google LLC)
Official title
REDUCING POWER CONSUMPTION BY HARDWARE ACCELERATOR DURING GENERATION AND TRANSMISSION OF MACHINE LEARNING INFERENCES
Publication number
US20260288468A1
Application number
19679012
Filed
May 15, 2026
Published
Sep 24, 2026
Inventors
2
Claims
18

Classified under

G06F 9/3877 Physics & computingG06F 9/544 Physics & computing
Read the full filing (official PDF) ↗ Open in the Patent Tracker

More from Google in Chips & hardware

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.