NvidiaPatent applicationCars & transportInterest 6/10

Nvidia patents blending real and simulated images for self-driving cars

Patent drawing
Patent drawing from the official filing.

Nvidia describes a method for combining photos taken by a vehicle's cameras with computer-generated images of the same scene. The system mixes these two types of images, adjusting for differences in lighting, to help the vehicle plan its route and control its movement.

Why it mattersThis approach could help self-driving cars see and react to their surroundings more reliably by supplementing real camera feeds with simulated data.

In the filing’s own words

In various examples, techniques for blending simulated and real-world data for semi-autonomous and/or autonomous systems and applications are described herein. Systems and methods described herein may obtain real images captured using one or more image sensors of a machine navigating within an environment along with simulated images captured using a simulation of the environment. As described herein, in some examples, the real images may be associated with a first level of lighting while the simulated images are associated with a second, greater level of lighting. Systems and methods are then described that generate blended images by at least blending the real images with the simulated images. In some examples, different weights may be used to perform the blending, such as first weights that are applied to the real images and second weights that are applied to the simulated images. A machine may then perform operations using the blended images.

What exactly they claim (claim 1)

1. A method comprising: obtaining image data representing a real-world image of a portion of an environment, the image data obtained using one or more image sensors of a machine and associated with a first level of light in the portion of the environment; obtaining, based at least on one or more simulations associated with the environment, simulation data representing a simulated image corresponding to at least the portion of the environment, the simulated image being associated with a second level of light that is greater than the first level of light; and causing, based at least on the real-world image and the simulated image, the machine to perform one or more planning, navigation, or control operations.

Facts

Company
Nvidia (NVIDIA Corporation)
Official title
FUSING REAL-WORLD AND SIMULATED DATA
Publication number
US20260359939A1
Application number
19746364
Filed
Jul 17, 2026
Published
Nov 12, 2026
Inventors
2
Claims
20

Classified under

G06T 5/50 Physics & computingB60T 7/12 Operations & transportB60W 30/09 Operations & transportB60W 30/095 Operations & transportB60T 2201/022 Operations & transport

More from Nvidia in Cars & transport

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.