Nvidia patents method for smooth character transitions in neural rendering
Nvidia describes a technique for rendering digital characters that switches between pre-recorded video and AI-generated frames to keep movements smooth. The system tracks the character's state, using recorded clips for idle moments and generating new frames when the character begins to interact or speak.
In the filing’s own words
In various examples, techniques for neural rendering of characters using operating states are described herein. Systems and methods described herein may use various techniques with regard to neural rendering (and/or any other rendering techniques) to render smooth, stable, and/or quality interactive characters. For instance, a presentation of a character may operate in different operating states, such as an idle state where the character is not interacting, a transition state where the character is preparing to interact, and an interactive state where the character is interacting. To maintain a smooth transition between the operating states, poses associated with the character may be maintained, even when transitioning switches from recorded frames to newly generated frames. In some examples, the idle state and the transition state may use the recorded frames while the interactive state uses the newly generated frames to improve the performance of the presentation.
What exactly they claim (claim 1)
1. A method comprising: causing, during a first period of time, a presentation of one or more first frames of a recorded video, the one or more first frames representing a character using one or more first poses; determining that one or more second frames of the recorded video, which are subsequent to the one or more first frames of the recorded video, are associated with one or more second poses for the character; generating, during at least a portion the first period of time and based at least on one or more neural networks processing first data representing the one or more second poses and second data associated with speech for the character, one or more third frames representing the character using the one or more second poses; and causing, during a second period of time, a presentation of the one or more third frames.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- NEURAL RENDERING OF CHARACTERS USING OPERATING STATES
- Publication number
- US20260301289A1
- Application number
- 19174667
- Filed
- Apr 9, 2025
- Published
- Oct 1, 2026
- Inventors
- 10
- Claims
- 20
Classified under
G06T 13/40 Physics & computingG06T 7/70 Physics & computingG06T 17/00 Physics & computingG06V 10/82 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.