Nvidia patents method to animate faces from fast speech audio
Nvidia describes a technique that creates facial animations from audio recordings. When the speech rate is fast, the system adjusts the data size to help a machine learning model generate accurate mouth and face movements.
In the filing’s own words
In various examples, a technique for audio-driven facial animation with adaptive speech includes determining that a rate of speech associated with an audio segment exceeds a threshold. The technique also includes based at least on the rate of speech exceeding the threshold, upsampling a first set of features associated with the audio segment into a second set of features that is different in size than the first set of features. The technique further includes generating, using one or more machine learning models and based at least on at least a subset of the second set of features, a facial animation output corresponding to the audio segment.
What exactly they claim (claim 1)
1. A method, comprising: determining that a rate of speech associated with an audio segment exceeds a threshold; based at least on the rate of speech exceeding the threshold, upsampling a first set of features associated with the audio segment into a second set of features that is different in size than the first set of features; and generating, using one or more machine learning models and based at least on at least a subset of the second set of features, a facial animation output corresponding to the audio segment.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- AUDIO-DRIVEN FACIAL ANIMATION WITH ADAPTIVE SPEECH RATE
- Publication number
- US20260360063A1
- Application number
- 19741043
- Filed
- Jul 14, 2026
- Published
- Nov 12, 2026
- Inventors
- 8
- Claims
- 1
Classified under
G06T 13/40 Physics & computingG06T 13/205 Physics & computingG10L 15/02 Physics & computingG10L 15/04 Physics & computingG10L 15/16 Physics & computingG10L 2015/025 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents chip that turns text into video using noise reduction Nov 5, 2026
- Nvidia patents system that translates specific speakers in group conversations Nov 12, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.
