Nvidia patents method to track 3D body poses from 2D camera images
Nvidia describes a system that trains computer models to predict a person's three-dimensional body position using only standard 2D camera footage. The model learns by comparing its guesses against accurate depth data, adjusting itself until it can accurately map body movements from multiple camera angles.
In the filing’s own words
In various examples, systems and methods for pose detection model training for predicting three-dimensional pose estimates using two-dimensional image data are provided. The occupant pose detection model may be trained using multi-view image sensor training data that includes image frames that capture a pose of a training subject within a machine interior using multiple synchronized optical image sensors placed around the machine interior that produce a set of captured image frames of the training subject from different viewpoints. Based on the multi-view image sensor training data, the occupant pose detection model may generate a set of individual, predicted 3D pose estimates for the training subject from a captured image frame from each of the respective optical image sensors. To adjust the occupant pose detection model during training, a loss feedback may be generated that comprises a pose alignment loss, a pose depth loss, and/or a ground truth kinematic loss.
What exactly they claim (claim 1)
1. A system comprising: one or more processing units to: apply to at least one pose detection model, multi-view image sensor training data comprising two-dimensional optical image data representing an occupant of a machine, the two-dimensional optical image data captured from a plurality of viewpoints by a plurality of synchronized optical image sensors; input reference depth data representing the occupant derived from a depth sensor synchronized with the plurality of synchronized optical image sensors; generate a plurality of three-dimensional pose estimates of the occupant based on the two-dimensional optical image data captured from the plurality of viewpoints; generate a loss feedback comprising at least an alignment loss representing an alignment between the plurality of three-dimensional pose estimates for one or more kinematic elements of the occupant and a pose depth loss representing a difference between a depth of the one or more kinematic elements represented by the plurality of three-dimensional pose estimates and a depth of the one or more kinematic elements represented by the reference depth data; and adjust the at least one pose detection model based on the loss feedback.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- Three-dimensional pose estimation using two-dimensional images
- Patent number
- US12749214B2
- Application number
- 18349842
- Filed
- Jul 10, 2023
- Granted
- Sep 29, 2026
- Inventors
- 3
- Claims
- 18
Classified under
G06T 7/70 Physics & computingG06T 7/50 Physics & computingG06T 7/60 Physics & computingG06T 7/80 Physics & computingG06T 15/205 Physics & computingG06T 2207/20044 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.