Nvidia patents method to teach humanoid robots to use hands via simulation
This filing describes a way to train humanoid robots with multi-fingered hands to perform complex tasks. It uses computer simulations that mimic the physical robot's movements to find the best virtual settings for learning.
In the filing’s own words
Mechanisms for configuring a humanoid robot comprising a plurality of multi-fingered graspers to perform dexterous manipulation tasks, whereby a plurality of simulated environments are generated representing a distribution of kinematic parameter combinations corresponding to a physical robot. In each simulated environment, a target trajectory sequence recorded from the physical robot is executed. For each simulated environment, a tracking error relative to execution of the target trajectory sequence by the physical robot is calculated. One or more simulated environments are selected based on a minimization of an average squared difference between simulated tracking error and real-world tracking error, and a manipulation policy for the physical robot is then trained within the selected simulated environments using reinforcement learning.
What exactly they claim (claim 1)
1. A computer-implemented method for configuring a humanoid robot comprising a plurality of multi-fingered graspers to perform dexterous manipulation tasks, the method comprising: generating a plurality of simulated environments representing a distribution of kinematic parameter combinations corresponding to a physical robot; executing, in each simulated environment, a target trajectory sequence recorded from the physical robot; computing, for each simulated environment, a tracking error relative to execution of the target trajectory sequence by the physical robot; selecting one or more simulated environments based on a minimization of an average difference between simulated tracking error and real-world tracking error; training a manipulation policy within the selected simulated environments using reinforcement learning; and configuring the physical robot with the manipulation policy.
Facts
- Company
- Nvidia (NVIDIA Corp.)
- Official title
- Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids
- Publication number
- US20260316276A1
- Application number
- 19567726
- Filed
- Mar 16, 2026
- Published
- Oct 15, 2026
- Inventors
- 4
- Claims
- 20
Classified under
B25J 9/1671 Operations & transportB25J 9/163 Operations & transportG06N 3/092 Physics & computingB25J 9/1612 Operations & transportMore from Nvidia in Robots & automation
- Nvidia patents robot control using language models to predict 3D positions Sep 10, 2026
- Nvidia patents 3D scene mapping that lets robots understand objects by name Oct 15, 2026
- Nvidia patents robot system that learns tasks from visual data Sep 29, 2026
- Nvidia patents method for robots to estimate object positions from images Oct 13, 2026
- Nvidia patents method to create new robot training videos from existing footage Sep 10, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.