Google patents speech recognition that uses past context to improve accuracy
This patent describes a method for automatic speech recognition (ASR) that combines audio data with previous transcriptions to better understand what is being said. It uses a 'joint network' to fuse acoustic features, context from past sentences, and text predictions into a single decision for the most likely words.
In the filing’s own words
A method includes receiving a sequence of acoustic frames characterizing an input utterance and generating a higher order feature representation for a corresponding acoustic frame in the sequence of acoustic frames by an audio encoder of an automatic speech recognition (ASR) model. The method also includes generating a context embedding corresponding to one or more previous transcriptions output by the ASR model by a context encoder of the ASR model and generating, by a prediction network of the ASR model, a dense representation based on a sequence of non-blank symbols output by a final Softmax layer. The method also includes generating, by a joint network of the ASR model, a probability distribution over possible speech recognition hypotheses based on the context embeddings generated by the context encoder, the higher order feature representation generated by the audio encoder, and the dense representation generated by the prediction network.
What exactly they claim (claim 1)
1. A system comprising data processing hardware and memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to implement an automated speech recognition (ASR) model comprising: an audio encoder configured to: receive, as input, a sequence of acoustic frames characterizing an input utterance; generate, at each of a plurality of output steps, a higher order feature representation for a corresponding acoustic frame in the sequence of acoustic frames; a context encoder configured to: receive, as input, one or more previous transcriptions output by the ASR model, each previous transcription corresponding to a respective previous utterance comprising one or more words; generate, at each of the plurality of output steps, a context embedding corresponding to the one or more previous transcriptions; a prediction network configured to: receive, as input, a sequence of non-blank symbols output by a final Softmax layer; and generate, at each of the plurality of output steps, a dense representation; and a joint network configured to: receive, as input, the context embedding generated by the context encoder at each of the plurality of output steps, the higher order feature representation generated by the audio encoder at each of the plurality of output steps, and the dense representation generated by the prediction network at each of the plurality of output steps; fuse, at each of the plurality of output steps, using a 3-way tensor fusion mechanism, the context embedding, the higher order feature representation, and the dense representation into a fused representation; and generate, at each of the plurality of output steps, a probability distribution over possible speech recognition hypotheses based on the fused representation.
Facts
- Company
- Google (Google LLC)
- Official title
- Context-aware end-to-end ASR fusion of context, acoustic and text presentations
- Patent number
- US12770614B2
- Application number
- 18489970
- Filed
- Oct 19, 2023
- Granted
- Oct 20, 2026
- Inventors
- 1
- Claims
- 22
Classified under
G10L 15/18 Physics & computingG10L 15/183 Physics & computingMore from Google in AI & software
- Google patents video search that answers questions using AI descriptions Sep 15, 2026
- Google patents photo tool that removes people and their gear Sep 15, 2026
- Google patents AI that writes live meeting notes and summaries Sep 15, 2026
- Google patents AI tool that turns messy emergency notes into clear messages Sep 3, 2026
- Google patents AI that clicks through apps to finish tasks for you Sep 24, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.