Nvidia patents AI tool that auto-labels app states for streaming
Nvidia describes a system where AI models analyze an app's screenshots or documentation to automatically create unique labels for its different states. These labels, stored as key-value pairs, allow the system to track what part of the application is currently active during runtime.
In the filing’s own words
In various examples, generative artificial intelligence (AI) models may be used to automatically generate state keys (or “identifiers”) associated with an interactive application. For instance, the systems and methods of the present disclosure may use the AI models to generate the state keys corresponding to essential features of an application. The generated state keys may be structured as key-value pairs, enabling automated application state tracking by mapping values of the essential features to their respective state identifier keys during application runtime. In some examples, the systems and methods of the present disclosure may generate the state keys in a number of ways, such as by using a pretrained AI model with intrinsic knowledge of the application, using a generic AI model to infer the essential features by analyzing a plurality of screenshots captured during application runtime and/or by processing application documentation, wikis, or manuals.
What exactly they claim (claim 1)
1. A method comprising: applying input data to one or more language models; determining, based at least on the one or more language models processing the input data, one or more features of an interactive application that contribute to one or more states associated with the interactive application; generating, using the one or more language models, text data representing one or more identifiers corresponding to the one or more features; and storing one or more data structures including the one or more identifiers corresponding to the one or more features, wherein, during runtime of the interactive application, one or more values associated with the one or more features are mapped to the one or more identifiers of the one or more data structures to indicate the one or more states associated with the interactive application.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- ARTIFICIAL INTELLIGENCE-BASED GENERATION OF APPLICATION STATE KEYS FOR CONTENT STREAMING SYSTEMS AND APPLICATION
- Publication number
- US20260278249A1
- Application number
- 19089628
- Filed
- Mar 25, 2025
- Published
- Sep 17, 2026
- Inventors
- 5
- Claims
- 20
Classified under
G06F 40/166 Physics & computingA63F 13/67 Human necessitiesG06V 10/25 Physics & computingG06V 10/70 Physics & computingG06V 30/1448 Physics & computingG06V 30/191 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.