Netflix patents method to estimate video depth in real time
This filing describes a computer system that creates depth maps for video frames by combining high-resolution details with lower-resolution semantic information. It uses a stateful model to align these depth features across consecutive frames, ensuring consistent depth scaling without needing to look at future frames.
In the filing’s own words
A computer system may generate spatially detailed depth-related features from a higher-resolution representation of a video frame. The computer system may generate semantically informative depth-related features from a lower-resolution representation of a video frame. The computer system may generate refined spatially detailed depth-related features by selectively incorporating the semantically informative depth-related features into the spatially detailed depth-related features. The computer system may generate a dense feature representation for each video frame from the refined spatially detailed depth-related features. The computer system may temporally align the dense feature representation for a current video frame using a stateful temporal alignment model that maintains a state derived from previously processed video frames. The computer system may generate a depth map for the current video frame from the temporally aligned dense feature representation.
What exactly they claim (claim 1)
1. A computer-implemented method comprising: receiving a sequence of video frames; generating spatially detailed depth-related features from a higher-resolution representation of a current video frame selected from the sequence of video frames using a first transformer-based encoder; generating semantically informative depth-related features from a lower-resolution representation of the current video frame using a second transformer-based encoder; generating refined spatially detailed depth-related features by selectively incorporating, via a feature-interaction module, the semantically informative depth-related features into the spatially detailed depth-related features; generating a dense feature representation for the current video frame from the refined spatially detailed depth-related features using one or more transformer-based depth prediction modules; temporally aligning the dense feature representation for the current video frame using a stateful temporal alignment model that maintains a state derived from previously processed video frames, the temporally aligning including generating one or more alignment parameters configured to enforce depth-scale consistency across frames and applying the one or more alignment parameters to the dense feature representation or to depth values derived therefrom, without access to future video frames; and generating a depth map for the current video frame from the temporally aligned dense feature representation.
Facts
- Company
- Netflix (Netflix, Inc.)
- Official title
- REAL-TIME ONLINE VIDEO DEPTH ESTIMATION AT HIGH RESOLUTION
- Publication number
- US20260289804A1
- Application number
- 19572675
- Filed
- Mar 19, 2026
- Published
- Sep 24, 2026
- Inventors
- 4
- Claims
- 20
Classified under
G06T 7/55 Physics & computingG06T 7/33 Physics & computingG06T 11/00 Physics & computingG06T 2207/10016 Physics & computingG06T 2207/20016 Physics & computingG06T 2207/20081 Physics & computingMore from Netflix in AI & software
- Netflix patents method to train AI for better video scaling Sep 15, 2026
- Netflix patents method to bundle game frames for smoother streaming Sep 22, 2026
- Netflix patents search system that explains why a show is missing Sep 29, 2026
- Netflix patents tech to match video scenes across different languages Oct 13, 2026
- Netflix patents on-device AI to identify objects in streaming video Oct 1, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.