Nvidia patents 3D camera system that tracks moving objects in real time
Nvidia describes a system that uses groups of cameras to create 3D maps of an area and track objects moving through it. It combines data from multiple cameras using machine learning models to keep track of objects as they move over time.
In the filing’s own words
In various embodiments, 3D perception and multi-sensor object tracking for monitored environments using machine learning are provided. A 3D perception-based real-time location system is disclosed that tracks multiple objects within a monitored area based on 3D behavior data from multiple image sensors, producing 3D tracking solutions for detected entities moving within the environment. The system may leverage synchronized 3D behavior data computed from optical image sensors grouped into sensor pods. The system may include a machine learning-based tracking framework that includes a spatial aggregation model and a temporal tracking model. The spatial aggregation model aggregates 3D perception data across the sensor pods, while the temporal tracking model refines object associations across time. The spatial aggregation model and/or the temporal tracking model may comprise a graph neural network (GNN). In some embodiments, a machine learning-based tracking framework may comprise a state management system to augment inputs into the temporal tracking model.
What exactly they claim (claim 1)
1. One or more processors comprising processing circuitry to: obtain image data captured from a plurality of sensors, wherein the plurality of sensors are grouped based at least on proximity into a plurality of sensor pods; compute, for individual sensor pods of the plurality of sensor pods and based at least on the image data, behavior data comprising one or more three-dimensional (3D) representations of a behavior of one or more objects within a region of a monitored environment within a field of view of the individual sensor pods; synchronize contemporaneous metadata of the behavior data across the plurality of sensor pods to generate synchronized behavior data; compute an aggregated 3D behavior dataset based at least on applying the synchronized behavior data to a spatial aggregation model; and generate, based at least on processing the aggregated 3D behavior dataset over time with a temporal tracking model, 3D object tracking data corresponding to one or more objects tracked across the individual sensor pods of the plurality of sensor pods.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- 3D PERCEPTION AND MULTI-SENSOR OBJECT TRACKING FOR MONITORED ENVIRONMENTS USING MACHINE LEARNING
- Publication number
- US20260278806A1
- Application number
- 19310767
- Filed
- Aug 26, 2025
- Published
- Sep 17, 2026
- Inventors
- 14
- Claims
- 20
Classified under
G06T 7/20 Physics & computingG06V 10/82 Physics & computingG06V 20/52 Physics & computingG06V 40/20 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.