Nvidia patents blending real and simulated images for self-driving cars
Nvidia describes a method for combining actual camera footage from a vehicle with computer-generated images of the same environment. The system assigns specific weights to each image type to create a single blended picture that the car's software uses to guide its movement.
In the filing’s own words
In various examples, techniques for blending simulated and real-world data for semi-autonomous and/or autonomous systems and applications are described herein. Systems and methods described herein may obtain real images captured using one or more image sensors of a machine navigating within an environment along with simulated images captured using a simulation of the environment. As described herein, in some examples, the real images may be associated with a first level of lighting while the simulated images are associated with a second, greater level of lighting. Systems and methods are then described that generate blended images by at least blending the real images with the simulated images. In some examples, different weights may be used to perform the blending, such as first weights that are applied to the real images and second weights that are applied to the simulated images. A machine may then perform operations using the blended images.
What exactly they claim (claim 1)
1. A method comprising: receiving image data obtained using one or more image sensors of a machine, the image data representing a real-world image of a portion of an environment; obtaining, using a digital twin simulation associated with the environment, simulation data representing a simulated image corresponding to at least the portion of the environment; determining a first weight associated with the real-world image and a second weight associated with the simulated image; generating, based at least on blending the real-world image with the simulated image using the first weight and the second weight, a blended image of the environment; processing the blended image using one or more machine learning models of a machine; and causing, based at least on the processing the blended image, the machine to navigate within the environment.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- Fusing simulated and real-world data
- Patent number
- US12737850B2
- Application number
- 18949345
- Filed
- Nov 15, 2024
- Granted
- Sep 15, 2026
- Inventors
- 2
- Claims
- 20
Classified under
G06T 5/50 Physics & computingB60T 7/12 Operations & transportB60W 30/09 Operations & transportB60W 30/095 Operations & transportB60T 2201/022 Operations & transportMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.