Nvidia patents video tracking that locks onto objects using heatmaps
Nvidia describes a method for tracking the position and orientation of objects in video by analyzing multiple frames. The system uses neural networks to create 'heatmaps' that highlight key points on an object, allowing it to calculate precise movements between images. This data is then used to build a 3D box around the object to track its path.
In the filing’s own words
Apparatuses, systems, and techniques to determined a pose of an object from a plurality of images. In at least one embodiment, the pose of an object is determined from at least two images of a video sequence using one or more neural networks, in which the neural network produces a distribution of pose information that is filtered to determine the current pose.
What exactly they claim (claim 1)
1. A computer-implemented method comprising: capturing one or more first images of one or more objects; generating one or more first heatmaps based, at least in part, on a centroid of the one or more objects and a plurality of keypoints of the one or more objects; capturing one or more second images of the one or more objects; identifying one or more pixel offsets for the centroid and the plurality of keypoints based, at least in part, on the one or more second images; generating one or more second heatmaps based, at least in part, on the one or more pixel offsets; and generating a tracking cuboid having dimensions centered around the one or more objects.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- OBJECT POSE TRACKING FROM VIDEO IMAGES
- Publication number
- US20260268515A1
- Application number
- 19663046
- Filed
- Apr 29, 2026
- Published
- Sep 10, 2026
- Inventors
- 4
- Claims
- 20
Classified under
G06T 7/70 Physics & computingG06T 7/277 Physics & computingG06T 2207/10016 Physics & computingG06T 2207/10024 Physics & computingG06T 2207/20081 Physics & computingG06T 2207/20084 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.