AmazonGranted patentAudio & voiceInterest 6/10

Amazon patents voice assistant that detects user frustration

This patent describes a system that listens for signs of frustration in a user's voice, such as repeating a command. Instead of blindly executing the command again, the system uses machine learning to interpret the user's intent differently or asks for clarification to resolve the issue.

Why it mattersThis could make smart speakers feel less robotic by allowing them to adapt their responses when a user is clearly unhappy with the previous result.

In the filing’s own words

Described herein is a system for responding to a frustrated user with a response determined based on spoken language understanding (SLU) processing of a user input. The system detects user frustration and responds to a repeated user input by confirming an action to be performed or presenting an alternative action, instead of performing the action responsive to the user input. The system also detects poor audio quality of the captured user input, and responds by requesting the user to repeat the user input. The system processes sentiment data and signal quality data to respond to user inputs.

What exactly they claim (claim 1)

1. A computer-implemented method comprising: receiving first audio data representing a first utterance; determining an automatic speech recognition (ASR) confidence score corresponding to an ASR hypothesis representing the first utterance; determining first natural language understanding (NLU) data corresponding to the first utterance; causing performance of a first action based on the first NLU data; receiving second audio data representing a second utterance; determining, using a trained machine learning model, sentiment data corresponding to the second audio data, wherein the sentiment data indicates frustration; based on the sentiment data and the ASR confidence score, determining second NLU data as a modified interpretation of the first utterance that differs from the first NLU data, wherein the second NLU data is determined using: (i) stored utterance rephrase pairs corresponding to historic dialog sessions separate from a current dialog session including the first utterance and the second utterance, and (ii) feedback corresponding to the stored utterance rephrase pairs, wherein the feedback indicates output of desired system responses; and in response to the second utterance, causing performance of a second action based on the second NLU data, wherein the second action is different from the first action.

Facts

Company
Amazon (Amazon Technologies, Inc.)
Official title
Sentiment aware voice user interface
Patent number
US12793710B2
Application number
17974677
Filed
Oct 27, 2022
Granted
Nov 10, 2026
Inventors
14
Claims
15

Classified under

G10L 15/22 Physics & computingG06V 10/764 Physics & computingG10L 15/1815 Physics & computingG10L 15/26 Physics & computingG10L 2015/225 Physics & computingG10L 2015/227 Physics & computing

More from Amazon in Audio & voice

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.