Meta patents hand gestures and voice combos for assistant commands
Meta describes a system that combines hand gestures and spoken words to control digital assistants. It uses a personalized model to recognize specific gesture-and-speech pairs, or 'tuples,' to determine what the user intends to do.
In the filing’s own words
In one embodiment, a method includes receiving a user request from a first user from a client system associated with a first user, wherein the user request comprise a gesture-input from the first user and a speech-input from the first user, determining an intent corresponding to the user request based on the gesture-input by a personalized gesture-classification model associated with the first user, executing one or more tasks based on the determined intent and the speech-input, and sending instructions for presenting execution results of the one or more tasks to the client system responsive the user request.
What exactly they claim (claim 1)
1. A method comprising: receiving an input from a user that includes (i) a gesture input or (ii) a speech input; in accordance with receiving the input from the user, identifying a respective input tuple from a set of input tuples of known gesture inputs and corresponding known speech inputs associated with a particular user intent; determining whether the input from the user satisfies a confidence score for a respective known gesture input or a respective known speech input of the respective input tuple; and in accordance with determining that the input from the user satisfies the confidence score, providing an input to an assistant system corresponding to the particular user intent.
Facts
- Company
- Meta (Meta Platforms Technologies, LLC)
- Official title
- Techniques for recognizing alternative input via gesture recognition tuples for user interactions with assistant systems
- Patent number
- US12783967B2
- Application number
- 19392028
- Filed
- Nov 17, 2025
- Granted
- Nov 3, 2026
- Inventors
- 4
- Claims
- 20
Classified under
G06Q 10/00 Physics & computingG06F 3/011 Physics & computingG06F 3/013 Physics & computingG06F 3/017 Physics & computingG06F 3/167 Physics & computingG06F 7/14 Physics & computingMore from Meta in AI & software
- Meta patents AI summaries for notifications on glasses and watches Sep 29, 2026
- Meta patents avatar editing that keeps your face while changing the rest Sep 10, 2026
- Meta patents group video effects that change when you interact with them Oct 20, 2026
- Meta patents AI camera that frames avatars based on body movement Nov 17, 2026
- Meta patents AI tool that remixes low-performing posts into new content Nov 12, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.
