Netflix patents on-device AI to identify objects in streaming video
The filing describes a method where a device runs AI directly on the video feed to spot specific objects. It then compares these objects to a local database to pull up related information, which is displayed on the screen while the content plays.
In the filing’s own words
In some embodiments, a computer-implemented method for performing artificial intelligence (AI) processing in a streaming environment on an endpoint device comprises: obtaining media data associated with playback of content by a streaming application executing on the endpoint device; identifying, via at least one AI inference operation executed on the media data, an object represented within the media data; generating refined object data corresponding to the object; generating feature data associated with the object based on the refined object data; obtaining item information associated with the object based on a comparison between the feature data and reference data stored on the endpoint device; and outputting presentation data associated with the item information for display in association with the playback of the content.
What exactly they claim (claim 1)
1. A computer-implemented method for performing artificial intelligence (AI) processing in a streaming environment on an endpoint device, the method comprising: obtaining media data associated with playback of content by a streaming application executing on the endpoint device; identifying, via at least one AI inference operation executed on the media data, an object represented within the media data; generating refined object data corresponding to the object; generating feature data associated with the object based on the refined object data; obtaining item information associated with the object based on a comparison between the feature data and reference data stored on the endpoint device; and outputting presentation data associated with the item information for display in association with the playback of the content.
Facts
- Company
- Netflix (NETFLIX, INC.)
- Official title
- SCALABLE AND OPTIMIZED EDGE AI ARCHITECTURE FOR INTERACTIVE STREAMING APPLICATIONS
- Publication number
- US20260303926A1
- Application number
- 19570808
- Filed
- Mar 18, 2026
- Published
- Oct 1, 2026
- Inventors
- 4
- Claims
- 20
Classified under
H04N 21/6581 Electricity & electronicsG06V 10/25 Physics & computingH04N 21/44008 Electricity & electronicsH04N 21/4532 Electricity & electronicsH04N 21/8586 Electricity & electronicsG06V 2201/07 Physics & computingMore from Netflix in AI & software
- Netflix patents method to train AI for better video scaling Sep 15, 2026
- Netflix patents method to bundle game frames for smoother streaming Sep 22, 2026
- Netflix patents search system that explains why a show is missing Sep 29, 2026
- Netflix patents tech to match video scenes across different languages Oct 13, 2026
- Netflix patents method to estimate video depth in real time Sep 24, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.