/Patent Tracker
GooglePatent applicationRobots & automationInterest 8/10

Google patents robot control via natural language instructions

Patent drawing
Patent drawing from the official filing.

The filing describes a system where a robot receives free-form spoken or written commands and uses a large language model to break them down into specific physical actions. It combines this with real-time sensor data to check if the task is safe and possible before executing the movement.

Why it mattersThis approach could make robots easier for ordinary people to use by allowing them to give simple, everyday commands instead of complex programming codes.

In the filing’s own words

Implementations process, using a large language model, a free-form natural language (NL) instruction to generate to generate LLM output. Those implementations generate, based on the LLM output and a NL skill description of a robotic skill, a task-grounding measure that reflects a probability of the skill description in the probability distribution of the LLM output. Those implementations further generate, based on the robotic skill and current environmental state data, a world-grounding measure that reflects a probability of the robotic skill being successful based on the current environmental state data. Those implementations further determine, based on both the task-grounding measure and the world-grounding measure, whether to implement the robotic skill.

What exactly they claim (claim 1)

1. A method implemented by one or more processors, the method comprising: identifying a high-level instruction, the high-level instruction being a free-form natural language instruction; identifying state data that is based on sensor data captured by one or more sensor components of a robot; generating a prompt that includes the high-level instruction and the state data; processing the prompt using a sequence-to-sequence model to generate a sequence of steps that are in natural language and that describe a sequence of low-level skills that can be executed by the robot to accomplish the high-level instruction; in response to generating the sequence of steps: processing, using a policy model, (i) the natural language for a first low-level skill, of the sequence of low-level skills, and (ii) the sensor data or additional sensor data captured by one or more of the sensor components of the robot, to generate action output; and controlling the robot based on the action output generated based on the processing using the policy model.

Facts

Company
Google (GOOGLE LLC)
Official title
NATURAL LANGUAGE CONTROL OF A ROBOT
Publication number
US20260316289A1
Application number
19700942
Filed
Jun 8, 2026
Published
Oct 15, 2026
Inventors
6
Claims
20

Classified under

B25J 13/003 Operations & transportB25J 9/161 Operations & transportB25J 9/163 Operations & transportB25J 11/0005 Operations & transportG06F 40/40 Physics & computing
Read the full filing (official PDF) ↗ Open in the Patent Tracker

More from Google in Robots & automation

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.