Amazon patents voice assistant that detects user frustration
This patent describes a system that listens for signs of frustration in a user's voice, such as repeating a command. Instead of blindly executing the command again, the system uses machine learning to interpret the user's intent differently or asks for clarification to resolve the issue.
In the filing’s own words
Described herein is a system for responding to a frustrated user with a response determined based on spoken language understanding (SLU) processing of a user input. The system detects user frustration and responds to a repeated user input by confirming an action to be performed or presenting an alternative action, instead of performing the action responsive to the user input. The system also detects poor audio quality of the captured user input, and responds by requesting the user to repeat the user input. The system processes sentiment data and signal quality data to respond to user inputs.
What exactly they claim (claim 1)
1. A computer-implemented method comprising: receiving first audio data representing a first utterance; determining an automatic speech recognition (ASR) confidence score corresponding to an ASR hypothesis representing the first utterance; determining first natural language understanding (NLU) data corresponding to the first utterance; causing performance of a first action based on the first NLU data; receiving second audio data representing a second utterance; determining, using a trained machine learning model, sentiment data corresponding to the second audio data, wherein the sentiment data indicates frustration; based on the sentiment data and the ASR confidence score, determining second NLU data as a modified interpretation of the first utterance that differs from the first NLU data, wherein the second NLU data is determined using: (i) stored utterance rephrase pairs corresponding to historic dialog sessions separate from a current dialog session including the first utterance and the second utterance, and (ii) feedback corresponding to the stored utterance rephrase pairs, wherein the feedback indicates output of desired system responses; and in response to the second utterance, causing performance of a second action based on the second NLU data, wherein the second action is different from the first action.
Facts
- Company
- Amazon (Amazon Technologies, Inc.)
- Official title
- Sentiment aware voice user interface
- Patent number
- US12793710B2
- Application number
- 17974677
- Filed
- Oct 27, 2022
- Granted
- Nov 10, 2026
- Inventors
- 14
- Claims
- 15
Classified under
G10L 15/22 Physics & computingG06V 10/764 Physics & computingG10L 15/1815 Physics & computingG10L 15/26 Physics & computingG10L 2015/225 Physics & computingG10L 2015/227 Physics & computingMore from Amazon in Audio & voice
- Amazon patents speech system that listens while talking Nov 10, 2026
- Amazon patents smart audio detection that only listens when needed Sep 1, 2026
- Amazon patents system to pick best audio from multiple devices Sep 8, 2026
- Amazon patents speech system that works even when internet drops Sep 8, 2026
- Amazon patents TV remote that helps pick the clearest audio channel Sep 8, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.