/Patent Tracker
GooglePatent applicationAI & softwareInterest 4/10

Google patents offline method to speed up robot learning

This filing describes a way to make reinforcement learning faster by using pre-recorded data instead of live trial and error. It uses two models: one that summarizes the current situation into a compact code, and another that picks the next action based on that code and the original state.

Why it mattersThis could help robots or autonomous systems learn complex tasks more efficiently by reducing the need for expensive real-time testing.

In the filing’s own words

A method includes determining a first state associated with a particular task, and determining, by a task policy model, a latent space representation of the first state. The task policy model may have been trained to define, for each respective state of a plurality of possible states associated with the particular task, a corresponding latent space representation of the respective state. The method also includes determining, by a primitive policy model and based on the first state and the latent space representation of the first state, an action to take as part of the particular task. The primitive policy model may have been trained to define a space of primitive policies for the plurality of possible states associated with the particular task and a plurality of possible latent space representations. The method further includes executing the action to reach a second state associated with the particular task.

What exactly they claim (claim 1)

1. A computer-implemented method comprising: determining a first state associated with a particular task; determining, by a task policy model, a latent space representation of the first state, wherein the task policy model has been trained to define, for each respective state of a plurality of possible states associated with the particular task, a corresponding latent space representation of the respective state; determining, by a primitive policy model and based on (i) the first state and (ii) the latent space representation of the first state, an action to take as part of the particular task, wherein the primitive policy model has been trained to define a space of primitive policies for the plurality of possible states associated with the particular task and a plurality of possible latent space representations, and wherein the primitive policy model has been trained by a process comprising: determining an undirected experience dataset comprising one or more sub-trajectories each representing a corresponding transition through a corresponding plurality of states of a training task; determining, by an encoder model, for each respective sub-trajectory of the one or more sub-trajectories, one or more parameters of a corresponding latent space distribution; generating, by a prior model and for each respective sub-trajectory of the one or more sub-trajectories, second one or more parameters of the corresponding latent space distribution based on an initial state associated with the respective sub-trajectory; and training the primitive policy model based on a comparison of (i) the one or more parameters of the corresponding latent space distribution and (ii) the second one or more parameters of the corresponding latent space distribution; and executing the action to reach a second state associated with the particular task.

Facts

Company
Google (Google LLC)
Official title
Offline Primitive Discovery For Accelerating Data-Driven Reinforcement Learning
Publication number
US20260289242A1
Application number
19701512
Filed
Jun 8, 2026
Published
Sep 24, 2026
Inventors
4
Claims
20

Classified under

G06N 3/0455 Physics & computingG06N 3/092 Physics & computing
Read the full filing (official PDF) ↗ Open in the Patent Tracker

More from Google in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.