Nvidia patents blending real and simulated images for self-driving cars
Nvidia describes a method for combining photos taken by a vehicle's cameras with computer-generated images of the same scene. The system mixes these two types of images, adjusting for differences in lighting, to help the vehicle plan its route and control its movement.
In the filing’s own words
In various examples, techniques for blending simulated and real-world data for semi-autonomous and/or autonomous systems and applications are described herein. Systems and methods described herein may obtain real images captured using one or more image sensors of a machine navigating within an environment along with simulated images captured using a simulation of the environment. As described herein, in some examples, the real images may be associated with a first level of lighting while the simulated images are associated with a second, greater level of lighting. Systems and methods are then described that generate blended images by at least blending the real images with the simulated images. In some examples, different weights may be used to perform the blending, such as first weights that are applied to the real images and second weights that are applied to the simulated images. A machine may then perform operations using the blended images.
What exactly they claim (claim 1)
1. A method comprising: obtaining image data representing a real-world image of a portion of an environment, the image data obtained using one or more image sensors of a machine and associated with a first level of light in the portion of the environment; obtaining, based at least on one or more simulations associated with the environment, simulation data representing a simulated image corresponding to at least the portion of the environment, the simulated image being associated with a second level of light that is greater than the first level of light; and causing, based at least on the real-world image and the simulated image, the machine to perform one or more planning, navigation, or control operations.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- FUSING REAL-WORLD AND SIMULATED DATA
- Publication number
- US20260359939A1
- Application number
- 19746364
- Filed
- Jul 17, 2026
- Published
- Nov 12, 2026
- Inventors
- 2
- Claims
- 20
Classified under
G06T 5/50 Physics & computingB60T 7/12 Operations & transportB60W 30/09 Operations & transportB60W 30/095 Operations & transportB60T 2201/022 Operations & transportMore from Nvidia in Cars & transport
- Nvidia patents robotaxi system with AI dispatcher and virtual rails Sep 15, 2026
- Nvidia patents car drowsiness check that uses your body clock Oct 13, 2026
- Nvidia patents 3D camera system to measure passengers in vehicles Oct 13, 2026
- Nvidia patents cruise control that predicts driver's next move Oct 20, 2026
- Nvidia patents system to rate how much a car's sensors can trust their own data Nov 3, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.
