Nvidia patents method to pull 3D material data from video AI
Nvidia describes a system that uses video AI to generate a moving view of a 3D object from a few images and a text prompt. It then analyzes that video to determine the object's physical surface properties, such as how shiny or rough it is.
In the filing’s own words
Systems and methods are disclosed that extracting material information using one or more video diffusion models. For example, embodiments of the present disclosure may process one or more images and a text prompt using a first video diffusion model to generate a rendered video of an object. For instance, the one or more images may be based on a three-dimensional (3D) geometry of the object and lighting conditions and the rendered image may include frames that show different viewpoints of the object. Subsequently, a differentiable renderer may be used to process the rendered video to generate a material data structure for the object, which indicates material information for points from the 3D geometry of the object. The material information may indicate physical-based rendering (PBR) characteristics such as an albedo, roughness, and/or metallic characteristic of the points on the object.
What exactly they claim (claim 1)
1. A computer-implemented method for extracting material information using one or more video diffusion models, comprising: processing one or more images and a text prompt using a first video diffusion model, of the one or more video diffusion models, to generate a rendered video of an object, wherein the one or more images are based on a three-dimensional (3D) geometry of the object and one or more lighting conditions images, and wherein the rendered video comprises a plurality of frames that show different viewpoints of the object; and processing the rendered video of the object using a differentiable renderer to generate a material data structure for the object, wherein the material data structure indicates the material information for one or more points from the 3D geometry.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- EXTRACTING PHYSICALLY-BASED RENDERING (PBR) MATERIALS USING VIDEO DIFFUSION MODELS
- Publication number
- US20260311422A1
- Application number
- 19333549
- Filed
- Sep 19, 2025
- Published
- Oct 8, 2026
- Inventors
- 5
- Claims
- 20
Classified under
G06T 15/506 Physics & computingG06T 13/20 Physics & computingG06T 15/04 Physics & computingG06T 15/20 Physics & computingG06T 17/20 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.