Nvidia patents method to help self-driving cars spot objects using LiDAR
Nvidia describes a system that uses a neural network to identify objects in data from LiDAR sensors, which use laser pulses to map surroundings. The technology combines information from camera images and LiDAR scans to create accurate 3D models of objects without needing extensive manual labeling. This data helps autonomous vehicles understand their environment for safer navigation.
In the filing’s own words
In various examples, a deep neural network (DNN) may be used to detect and classify animate objects and/or parts of an environment. The DNN may be trained using camera-to-LiDAR cross injection to generate reliable ground truth data for LiDAR range images. For example, annotations generated in the image domain may be propagated to the LiDAR domain to increase the accuracy of the ground truth data in the LiDAR domain—e.g., without requiring manual annotation in the LiDAR domain. Once trained, the DNN may output instance segmentation masks, class segmentation masks, and/or bounding shape proposals corresponding to two-dimensional (2D) LiDAR range images, and the outputs may be fused together to project the outputs into three-dimensional (3D) LiDAR point clouds. This 2D and/or 3D information output by the DNN may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.
What exactly they claim (claim 1)
1. A method comprising: processing, using a neural network, a converted representation of sensor data generated from an initial representation of the sensor data to determine: one or more segmentation masks representing one or more portions of the converted representation of the sensor data that correspond to one or more object classes; and one or more first locations within the converted representation of the sensor data of one or more first bounding shapes corresponding to one or more objects; correlating the one or more first bounding shapes with the one or more segmentation masks; generating, based at least on the correlating and using the one or more first bounding shapes, one or more second locations within the initial representation of the sensor data of one or more second bounding shapes corresponding to the one or more objects; and performing one or more operations using a machine based at least on the one or more second locations of the one or more second bounding shapes.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- Object detection and classification
- Patent number
- US12730190B2
- Application number
- 18531103
- Filed
- Dec 6, 2023
- Granted
- Sep 8, 2026
- Inventors
- 6
- Claims
- 20
Classified under
G01S 7/4802 Physics & computingG01S 7/481 Physics & computingG01S 17/894 Physics & computingG01S 17/931 Physics & computingG06V 10/764 Physics & computingG06V 10/80 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.