Nvidia patents video tech to build 3D scenes from casual clips
Nvidia describes a method for creating 3D models from standard video footage without needing precise camera tracking data. The system uses a grid-based structure to organize 3D points, allowing it to generate new views of a scene from angles not captured in the original video.
In the filing’s own words
Systems and methods are disclosed for novel view synthesis (NVS) using a three-dimensional (3D) Gaussian Spatting (3DGS) framework. For example, embodiments of the present disclosure may obtain an input video comprising a plurality of frames and train the 3DGS framework to obtain training outputs comprising estimated camera poses for each of the plurality of frames, octree anchor formations comprising anchor points indicating a center of each of a plurality of octree anchor formation voxels that are based on a voxel density of the corresponding octree anchor formation voxel, and a multi-layer perceptron (MLP) that determines 3DGS parameters based on the octree anchor formations. Subsequently, an inference input indicating an inference camera pose may be obtained and an inference output from a viewpoint indicated by the inference camera pose may be generated.
What exactly they claim (claim 1)
1. A computer-implemented method for novel view synthesis (NVS) using a three-dimensional (3D) Gaussian Spatting (3DGS) framework, comprising: obtaining an input video comprising a plurality of frames; training the 3DGS framework to obtain training outputs comprising estimated camera poses for each of the plurality of frames and octree anchor formations comprising anchor points indicating a center of each of a plurality of octree anchor formation voxels that are based on a voxel density of the corresponding octree anchor formation voxel; obtaining an inference input indicating an inference camera pose; and generating an inference output from a viewpoint indicated by the inference camera pose based on the estimated camera poses and the octree anchor formations.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- ROBUST UNPOSED THREE-DIMENSIONAL GAUSSIAN SPLATTING FOR CASUAL LONG VIDEOS USING OCTREE ANCHOR FORMATIONS
- Publication number
- US20260320906A1
- Application number
- 19324259
- Filed
- Sep 10, 2025
- Published
- Oct 15, 2026
- Inventors
- 6
- Claims
- 22
Classified under
G06T 15/205 Physics & computingG06T 7/70 Physics & computingG06T 2207/10016 Physics & computingG06T 2207/10024 Physics & computingG06T 2207/10028 Physics & computingG06T 2207/20081 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.