Nvidia patents method to animate digital faces from voice and style
Nvidia describes a system that uses audio and a specific speaking style to create movements for a digital face, known as a mesh. It uses a machine-learning model to calculate how the face's points should move based on the sound and the chosen style.
In the filing’s own words
In various examples, systems and methods are disclosed relating to animating virtual or digital actors or avatars using audio-driven animation. A system can identify an animation for a mesh corresponding to audio data and an indication of a speaking style. The system can generate a plurality of vertex deltas using the animation and a neutral pose for the mesh. The system can update, using the plurality of vertex deltas, the audio data, and the indication of the speaking style, a machine-learning model to generate output vertex deltas for the mesh given an input speaking style and input audio data.
What exactly they claim (claim 1)
1. One or more processors comprising: one or more circuits to: identify an animation for a mesh corresponding to audio data and an indication of a speaking style; generate a plurality of vertex deltas using the animation and a neutral pose for the mesh; and using the plurality of vertex deltas, the audio data, and the indication of the speaking style, update one or more parameters of a machine-learning model such that the machine-learning model generates output vertex deltas for the mesh given an input speaking style and input audio data.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- Audio-driven facial animation supporting varying identities and speaking styles
- Patent number
- US12793308B2
- Application number
- 18917641
- Filed
- Oct 16, 2024
- Granted
- Nov 10, 2026
- Inventors
- 4
- Claims
- 20
Classified under
G06T 13/205 Physics & computingG06N 3/0475 Physics & computingG06T 13/40 Physics & computingG06T 17/205 Physics & computingG06T 2200/24 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents chip that turns text into video using noise reduction Nov 5, 2026
- Nvidia patents system that translates specific speakers in group conversations Nov 12, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.