Amazon patents speech system that listens while talking
Amazon has been granted a patent for a speech-processing system that can listen to a user while it is still speaking. The technology uses a language model to distinguish between a user actually trying to interrupt and a user making small sounds like "mm-hmm" to show they are listening. If the system detects these small sounds, it continues its response without pausing, allowing for a smoother conversation flow.
In the filing’s own words
A speech-processing system may include a language model configured to interact with a user in a dual mode in which the system processes an input while continuing to generate an output, or vice versa. Dual-mode operation may allow the user and system to engage in backchannel communication in which the user and/or system interjects speech while the other is speaking, but without an intent to interrupt or take a turn. For example, if the system is outputting speech and detects user speech, the system may continue generating/outputting speech while it processes the user speech to determine whether it likely represents backchanneling or an interruption. If the user speech likely represents backchanneling, the system may continue generating speech without yielding the turn. Similarly, the system may receive user speech and identify an opportunity for backchanneling before the user has finished speaking. The system may generate backchannel speech without taking a turn.
What exactly they claim (claim 1)
1. A computer-implemented method comprising: generating, using a first language model (LM), first LM output data representing a first portion of first synthesized speech; causing a user device to present first output audio corresponding to the first LM output data; receiving first input audio data; determining that at least a first plurality of frames of the first input audio data represent human speech; in response to determining that at least the first plurality of frames of the first input audio data represent human speech, processing the first input audio data using the first LM; after beginning processing of the first input audio data using the first LM, generating, using the first LM, second LM output data representing a second portion of the first synthesized speech; and after beginning generating of the second LM output data, determining, based on processing of the first input audio data using the first LM, to perform a first action comprising at least one of continuing to process input audio data using the first LM or continuing to generate LM output data representing the first synthesized speech.
Facts
- Company
- Amazon (Amazon Technologies, Inc.)
- Official title
- Language model-based conversational modeling
- Patent number
- US12793713B1
- Application number
- 18824385
- Filed
- Sep 4, 2024
- Granted
- Nov 10, 2026
- Inventors
- 10
- Claims
- 20
Classified under
G10L 15/22 Physics & computingG10L 13/02 Physics & computingG10L 25/30 Physics & computingG10L 25/78 Physics & computingMore from Amazon in Audio & voice
- Amazon patents smart audio detection that only listens when needed Sep 1, 2026
- Amazon patents system to pick best audio from multiple devices Sep 8, 2026
- Amazon patents speech system that works even when internet drops Sep 8, 2026
- Amazon patents TV remote that helps pick the clearest audio channel Sep 8, 2026
- Amazon patents speaker design that points away from mic for better voice control Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.