Nvidia patents method to sync camera and LIDAR data using motion tracking
Nvidia describes a system that uses optical flow to detect movement between video frames from sensors like cameras or LIDAR. It creates a new, intermediate frame to align data from different sensors that capture images at slightly different times. This helps combine data from multiple sensors for tasks like 3D reconstruction.
In the filing’s own words
In various examples, systems and methods are disclosed that perform motion detection across image frames, such as optical flow determination, to synchronize an asynchronous frame with respect to a target time for the asynchronous frame. For example, image frames from a sensor can be processed by an optical flow accelerator to detect displacement across the image frames, and the displacement can be used to interpolate a modified frame at the target time. This can be used to perform data collection and combining operations such as stitching and/or reconstruction. The synchronization can be performed from sensor data from sensors such as cameras, LIDAR sensors, and/or RADAR sensors.
What exactly they claim (claim 1)
1. One or more processors comprising: one or more circuits to: detect an asynchronous condition of sensor data from a sensor; determine, responsive to detecting the asynchronous condition, motion associated with a first frame of the sensor data and a second frame of the sensor data; generate a third frame based at least on the motion and the first frame; and perform one or more operations associated with a machine based at least on the generated third frame.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- Optical flow-based frame interpolation to synchronize between sensors
- Patent number
- US12762719B2
- Application number
- 18783218
- Filed
- Jul 24, 2024
- Granted
- Oct 13, 2026
- Inventors
- 5
- Claims
- 20
Classified under
G06T 7/248 Physics & computingG01S 13/931 Physics & computingH04N 7/0127 Electricity & electronicsH04N 7/0137 Electricity & electronicsH04N 17/002 Electricity & electronicsG01S 2013/9315 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.