AppleGranted patentAI & softwareInterest 7/10

Apple patents voice assistant that pauses when you interrupt

Patent drawing
Patent drawing from the official filing.

Apple has been granted a patent for a system that manages how voice assistants handle user interruptions. When a user speaks while the assistant is talking, the system stops the audio and uses the new input to generate a fresh response. It also distinguishes between quick interruptions and longer ones to decide whether to pause or ask the user to continue.

Why it mattersThis makes voice conversations feel more natural by ensuring the assistant listens to you immediately instead of finishing its current sentence.

In the filing’s own words

A method or system for managing interruptions during oral interactions between users and Large Language Models (LLMs). Initially, a user's spoken input is received and converted to text, which forms a prompt for the LLM. Upon generating a text response by the LLM, the text response is then converted back into speech and played to the user. If the user interrupts while the response is being played, the playback stops, and the interruption is captured as a new spoken input. This interruption is used to generate a new prompt for the LLM. Subsequently, the LLM generates a second text response based on the interruption, which is converted to speech and played back to the user. This process ensures that user interruptions are effectively managed, allowing for a more dynamic and interactive conversation with the LLM and enhancing the user's experience by adapting the conversation flow to real-time inputs.

What exactly they claim (claim 1)

1. A method for managing interruptions during a verbal conversation between users and Large Language Models (LLMs), comprising: receiving a first speech input from a client device of a user; converting the first speech input into first text; generating a first prompt for an LLM based on the first text; transmitting the first prompt to the LLM, causing the LLM to generate a first text response; receiving the first text response from the LLM; converting the first text response into a first speech response; transmitting the first speech response back to the client device, causing the client device to play the first speech response to the user; detecting a second speech input from the user during the play of the first speech response; and in response to detecting an intent to interrupt the first speech response with the second speech input, causing the client device to stop playing the first speech response; and in accordance with a determination that the second speech input is a first type of interruption, wherein the first type of interruption occurs after a threshold time of the first speech response playing: generating a second prompt to the LLM based on the second speech input; transmitting the second prompt to the LLM, causing the LLM to generate a second text response; receiving the second text response from the LLM; converting the second text response into a second speech response; and transmitting the second speech response back to the client device, causing the client device to play the second speech response to the user; and in accordance with a determination that the second speech input is a second type of interruption, wherein the second type of interruption occurs prior to the threshold time of the first speech response playing, generating a continuer response to signal the user to continue speaking.

Facts

Company
Apple (Apple Inc.)
Official title
Real-time interactive voice conversation state management in large language models
Patent number
US12802686B2
Application number
18634800
Filed
Apr 12, 2024
Granted
Nov 17, 2026
Inventors
2
Claims
20

Classified under

G10L 15/22 Physics & computingG10L 13/08 Physics & computingG10L 15/26 Physics & computing

More from Apple in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.