/Patent Tracker
GoogleGranted patentRobots & automationInterest 7/10

Google patents robot that finds objects you point at using a stored map

Patent drawing
Patent drawing from the official filing.

This patent describes a system where a robot uses its sensors to detect a human gesture pointing toward an area it cannot currently see. It then checks a pre-stored map of the environment to identify the object in that specific direction and performs an action on it.

Why it mattersThis allows robots to understand human pointing gestures to interact with objects outside their immediate camera view, making them more intuitive to control in complex spaces.

In the filing’s own words

Methods, systems, and apparatus for receiving a command for controlling a robot, the command referencing an object, receiving sensor data for a portion of an environment of the robot, identifying, from the sensor data, a gesture of a human that indicates a spatial region located outside of the portion of the environment described by the sensor data, searching map data for the object, determining, based at least on searching the map data for the object referenced in the command, that the object referenced in the command is present in the spatial region, and in response to determining that the object referenced in the command is present in the spatial region, controlling the robot to perform an action with respect to the object referenced in the command.

What exactly they claim (claim 1)

1. A computer-implemented method comprising: storing information indicative of objects in an environment of a robot; detecting, based on sensor data that is captured by the robot and that describes a first region, a user in the environment; determining, based on a location of the detected user, a second region outside the first region described by the sensor data; accessing a portion of the stored information indicating locations of one or more objects in the second region; identifying, based at least on the accessed portion of the stored information, an object that that is located in the second region; and causing the robot to perform an action associated with the identified object that is located in the second region.

Facts

Company
Google (Google LLC)
Official title
Multimodal object identification
Patent number
US12746686B2
Application number
18888112
Filed
Sep 17, 2024
Granted
Sep 29, 2026
Inventors
2
Claims
20

Classified under

B25J 9/1697 Operations & transportG05B 19/048 Physics & computingB25J 19/026 Operations & transportG05B 2219/23021 Physics & computing
Read the full filing (official PDF) ↗ Open in the Patent Tracker

More from Google in Robots & automation

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.