MetaGranted patentAI & softwareInterest 6/10

Meta patents hand gestures and voice combos for assistant commands

Patent drawing
Patent drawing from the official filing.

Meta describes a system that combines hand gestures and spoken words to control digital assistants. It uses a personalized model to recognize specific gesture-and-speech pairs, or 'tuples,' to determine what the user intends to do.

Why it mattersThis could make voice assistants more accurate by using hand movements to clarify commands, reducing errors from background noise or ambiguous speech.

In the filing’s own words

In one embodiment, a method includes receiving a user request from a first user from a client system associated with a first user, wherein the user request comprise a gesture-input from the first user and a speech-input from the first user, determining an intent corresponding to the user request based on the gesture-input by a personalized gesture-classification model associated with the first user, executing one or more tasks based on the determined intent and the speech-input, and sending instructions for presenting execution results of the one or more tasks to the client system responsive the user request.

What exactly they claim (claim 1)

1. A method comprising: receiving an input from a user that includes (i) a gesture input or (ii) a speech input; in accordance with receiving the input from the user, identifying a respective input tuple from a set of input tuples of known gesture inputs and corresponding known speech inputs associated with a particular user intent; determining whether the input from the user satisfies a confidence score for a respective known gesture input or a respective known speech input of the respective input tuple; and in accordance with determining that the input from the user satisfies the confidence score, providing an input to an assistant system corresponding to the particular user intent.

Facts

Company
Meta (Meta Platforms Technologies, LLC)
Official title
Techniques for recognizing alternative input via gesture recognition tuples for user interactions with assistant systems
Patent number
US12783967B2
Application number
19392028
Filed
Nov 17, 2025
Granted
Nov 3, 2026
Inventors
4
Claims
20

Classified under

G06Q 10/00 Physics & computingG06F 3/011 Physics & computingG06F 3/013 Physics & computingG06F 3/017 Physics & computingG06F 3/167 Physics & computingG06F 7/14 Physics & computing

More from Meta in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.