Nvidia patents system that tracks objects across multiple cameras
Nvidia describes a machine learning framework that combines images from several cameras to track objects in 3D space. It uses neural networks to align the camera feeds and create a unique digital fingerprint for each object, allowing the system to follow that object even when it moves between different camera views.
In the filing’s own words
In various examples, the embodiments disclosed herein describe a 3 D perception-based machine learning framework for generating behavior data for detected objects (e.g., objects, persons, animals, machines, etc.) using multi-view optical image data. The framework processes multi-view optical image data from multiple camera sensors, and neural network-based spatial-temporal processing, to generate object behavior data that may be used to facilitate accurate real-time multi-target multi-camera (MTMC) object tracking across a monitored environment. The framework may comprise one or more machine learning models that input multi-view image sensor data and infer behavior data that may include, but is not limited to, 3 D bounding shapes, instance features, and/or 3 D Re-Identification (ReID) feature embeddings that may be used for assigning an object ID and for extending tracking of detected objects within the monitored environment. To generate an ReID feature embedding, an ReID module may aggregate features from different camera views based on visibility scores.
What exactly they claim (claim 1)
1. One or more processors comprising processing circuitry to: obtain image data captured from a plurality of image sensors and representing a combined field of view of a first portion of a monitored environment, wherein the image data comprises individual sensor image data corresponding to respective image sensors of the plurality of image sensors; determine a temporal alignment of the individual sensor image data based at least on an alignment threshold to define a batch of image data; and process the batch of image data using one or more neural networks to infer behavior data associated with one or more detected objects from the batch of image data, wherein the behavior data includes at least a feature embedding corresponding to a first detected object of the one or more detected objects, the feature embedding computed based at least on an aggregation of feature vectors representing the first detected object as extracted from different viewpoints represented by the individual sensor image data from the batch of image data.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- 3D MULTI-SENSOR PERCEPTION AND OBJECT TRACKING
- Publication number
- US20260279058A1
- Application number
- 19079809
- Filed
- Mar 14, 2025
- Published
- Sep 17, 2026
- Inventors
- 13
- Claims
- 20
Classified under
G06V 20/52 Physics & computingG06N 3/0455 Physics & computingG06V 10/765 Physics & computingG06V 10/82 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.