GoogleGranted patentRobots & automationInterest 7/10

Google patents robot vision that predicts how objects will move before acting

This patent describes a system where a robot uses a neural network to predict how objects in its surroundings will shift if it moves. The robot tests different potential movements virtually by applying them to a current camera image to see which action is safest or most effective before executing it.

Why it mattersThis helps robots navigate crowded spaces by letting them 'imagine' the consequences of their actions, reducing the risk of knocking over items or colliding with people.

In the filing’s own words

Some implementations of this specification are directed generally to deep machine learning methods and apparatus related to predicting motion(s) (if any) that will occur to object(s) in an environment of a robot in response to particular movement of the robot in the environment. Some implementations are directed to training a deep neural network model to predict at least one transformation (if any), of an image of a robot's environment, that will occur as a result of implementing at least a portion of a particular movement of the robot in the environment. The trained deep neural network model may predict the transformation based on input that includes the image and a group of robot movement parameters that define the portion of the particular movement.

What exactly they claim (claim 1)

1. A method implemented by one or more processors, comprising: generating candidate robot movement parameters, the candidate robot movement parameters defining at least a portion of a candidate movement performable in an environment of a robot by one or more components of the robot; identifying a current image captured by a vision sensor associated with the robot, the current image capturing at least a portion of the environment of the robot; applying the current image and the candidate robot movement parameters as input to a trained neural network; generating at least one predicted transformation of the current image, the predicted transformation indicating pixel motion predictions and being generated based on the application of the current image and the candidate robot movement parameters to the trained neural network; transforming the current image to generate at least one predicted image, wherein transforming the current image comprises applying, to the current image, the at least one predicted transformation generated based on the application of the current image and the candidate robot movement parameters to the trained neural network, the predicted image predicting the portion of the environment of the robot were the at least the portion of the candidate movement performed in the environment by the components of the robot; determining, based on the predicted image, to perform a second candidate movement in lieu of the candidate movement; generating second candidate robot movement parameters, the second candidate robot movement parameters defining at least a portion of the second candidate movement; applying the current image and the second candidate robot movement parameters as input to the trained neural network; generating at least one second predicted transformation of the current image, the second predicted transformation being generated based on the application of the current image and the second candidate robot movement parameters to the trained neural network; transforming one or more pixels of the current image based on the second predicted transformation to generate at least one second predicted image, the second predicted image predicting the portion of the environment of the robot if the at least the portion of the second candidate movement is performed in the environment by the components of the robot; selecting the second candidate movement based on the second predicted image; and providing one or more control commands to one or more actuators of the robot to perform the second candidate movement.

Facts

Company
Google (GOOGLE LLC)
Official title
Machine learning methods and apparatus related to predicting motion(s) of object(s) in a robot's environment based on image(s) capturing the object(s) and based on parameter(s) for future robot movement in the environment
Patent number
US12793402B2
Application number
17524185
Filed
Nov 11, 2021
Granted
Nov 10, 2026
Inventors
3
Claims
20

Classified under

G06V 10/82 Physics & computingB25J 9/161 Operations & transportB25J 9/1656 Operations & transportB25J 9/1697 Operations & transportG05B 13/027 Physics & computingG06N 3/008 Physics & computing

More from Google in Robots & automation

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.