Meta patents VR avatars that mimic facial expressions from voice
This patent describes a system that analyzes the voice of a person in a virtual room to predict their facial expressions. It then displays these expressions on a digital avatar, making the virtual character look like the real person is speaking.
In the filing’s own words
An audio system in a local area providing an audio signal to a headset of a remote user is presented herein. The audio system identifies sounds from a human sound source in the local area, based in part on sounds detected within the local area. The audio system generates an audio signal for presentation to a remote user within a virtual representation of the local area based in part on a location of the remote user within the virtual representation of the local area relative to a virtual representation of the human sound source within the virtual representation of the local area. The audio system provides the audio signal to a headset of the remote user, wherein the headset presents the audio signal as part of the virtual representation of the local area to the remote user.
What exactly they claim (claim 1)
1. A method comprising: receiving audio data captured, by a first computing system, from a human sound source; predicting a facial expression based at least in part on the audio data; causing a second computing system, remote from the first computing system, to play audio, based on the audio data, in relation to a virtual representation of the human sound source; and causing the second computing system to provide, in a virtual environment and in synchronization with the audio that is played, an additional virtual representation of the predicted expression in relation to the virtual representation of the human sound source, the additional virtual representation being provided responsive to an occluded portion of a face of the human sound source being within a threshold angle of a field of view of the second computing system.
Facts
- Company
- Meta (Meta Platforms Technologies, LLC)
- Official title
- Spatial audio and avatar control at headset using audio signals
- Patent number
- US12737954B2
- Application number
- 18658463
- Filed
- May 8, 2024
- Granted
- Sep 15, 2026
- Inventors
- 2
- Claims
- 20
Classified under
G06T 13/205 Physics & computingG02B 27/0172 Physics & computingG06N 20/00 Physics & computingG06T 13/40 Physics & computingG06T 19/006 Physics & computingG10L 21/10 Physics & computingMore from Meta in AR, VR & displays
- Meta patents spatial mouse for pointing at and controlling real-world objects Sep 24, 2026
- Meta patents AR notifications that grow when you look at them Sep 1, 2026
- Meta patents AR glasses with antennas built into the lenses Sep 8, 2026
- Meta patents auto-pause for VR headsets when you take them off Oct 6, 2026
- Meta patents AR glasses that merge multiple cameras for sharper views Sep 17, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.