GoogleGranted patentAI & softwareInterest 3/10

Google patents method to split large data tasks for faster processing

Patent drawing
Patent drawing from the official filing.

This patent describes a technique for breaking down complex mathematical calculations into smaller, manageable pieces. It identifies specific points in a data processing map where the work can be divided, allowing the computer to handle smaller chunks of data separately before combining the results. This approach helps manage memory usage more efficiently when running large-scale computations.

Why it mattersIt offers a specific way to optimize how computers handle large datasets, which is a standard engineering improvement for running complex software models.

In the filing’s own words

A method includes obtaining a compute graph for computing a first tensor, identifying in the graph a reduction operation in at least one dimension of the first tensor, locating, at the operation, a cut point that cuts the graph into first and second portions, and determining a plurality of slices of the first tensor. The method also includes backpropagating the cut point through the graph to define a plurality of first graph pieces for the first portion, each particular first graph piece representing a computation of a particular slice of the plurality of slices based on a particular portion of a plurality of portions of a second tensor. The method further includes defining one or more second graph pieces to combine outputs of the first graph pieces, and executing the first graph pieces and the second graph pieces to execute the first portion of the compute graph.

What exactly they claim (claim 1)

1. A method comprising: obtaining, by data processing hardware, a compute graph for computing a result based on a first tensor, the compute graph comprising a plurality of nodes, each node representing a computation operation and connected to one or more other nodes via edges, each edge representing a computational dependency between two connected nodes; identifying, by the data processing hardware and from the plurality of nodes, a reduction operation in at least one dimension of the first tensor; cutting, by the data processing hardware and at the reduction operation in the compute graph, the compute graph into a first portion and a second portion; determining, by the data processing hardware and based on the reduction operation, a plurality of slices of the first tensor; defining, by the data processing hardware, a plurality of first graph pieces for the first portion of the compute graph by at least propagating a cut backward from the reduction operation to inputs of preceding operations, each first graph piece of the plurality of first graph pieces representing a computation of a respective slice of the plurality of slices of the first tensor based on a respective portion of a plurality of portions of a second tensor; defining, by the data processing hardware, one or more second graph pieces to combine outputs of the plurality of first graph pieces; executing, by the data processing hardware, the plurality of first graph pieces and the one or more second graph pieces to compute the first tensor; and after executing the first portion of the compute graph, executing, by the data processing hardware, the second portion of the compute graph, using the first tensor, to compute the result.

Facts

Company
Google (Google LLC)
Official title
Automatic memory management for compute graphs
Patent number
US12801925B2
Application number
18063804
Filed
Dec 9, 2022
Granted
Nov 17, 2026
Inventors
6
Claims
20

Classified under

G06N 3/084 Physics & computing

More from Google in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.