/Patent Tracker
NvidiaGranted patentRobots & automationInterest 6/10

Nvidia patents robot system that learns tasks from visual data

Nvidia describes a system where robots use neural networks to watch tasks being performed and learn how to do them. The robot identifies objects and their relationships in images, creates a plan of actions, and executes those actions to complete the task.

Why it mattersThis approach could make it easier to teach robots new skills by showing them examples rather than programming every step.

In the filing’s own words

Various embodiments enable a robot, or other autonomous or semi-autonomous device or system, to receive data involving the performance of a task in the physical world. The data can be provided as input to a perception network to infer a set of percepts about the task, which can correspond to relationships between objects observed during the performance. The percepts can be provided as input to a plan generation network, which can infer a set of actions as part of a plan. Each action can correspond to one of the observed relationships. The plan can be reviewed and any corrections made, either manually or through another demonstration of the task. Once the plan is verified as correct, the plan (and any related data) can be provided as input to an execution network that can infer instructions to cause the robot, and/or another robot, to perform the task.

What exactly they claim (claim 1)

1. One or more processors, comprising: circuitry to use one or more neural networks to: infer probabilities of one or more locations of two or more objects within one or more images; determine, based at least in part on the inferred probabilities of the one or more locations, one or more relationships of the two or more objects within the one or more images; and generate one or more motion trajectories to cause one or more robotic devices to perform one or more tasks based, at least in part, on the one or more relationships of the two or more objects within the one or more images and the inferred probabilities of the one or more locations of the two or more objects within the one or more images.

Facts

Company
Nvidia (NVIDIA Corporation)
Official title
Learning robotic tasks using one or more neural networks
Patent number
US12749138B1
Application number
18437862
Filed
Feb 9, 2024
Granted
Sep 29, 2026
Inventors
6
Claims
20

Classified under

G06T 1/0014 Physics & computingB25J 9/161 Operations & transportB25J 9/1661 Operations & transportB25J 9/1697 Operations & transportG05B 13/00 Physics & computingG06N 3/08 Physics & computing
Read the full filing (official PDF) ↗ Open in the Patent Tracker

More from Nvidia in Robots & automation

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.