Google patents speech recognition training method using best-guess corrections
This patent describes a way to improve speech-to-text models by comparing their top guess against the most accurate option in a list of possibilities. It calculates a combined error score based on both the model's initial prediction and its best possible alternative to guide the learning process.
In the filing’s own words
A method includes, for each training sample of a plurality of training samples, processing, using an RNN-T model, a corresponding sequence of acoustic frames to obtain an n-best list of speech recognition hypotheses, and, for each speech recognition hypothesis of the n-best list, determining a corresponding number of word errors relative to a corresponding ground-truth transcription. For a top-ranked hypothesis from the n-best list, the method includes determining a first loss based on the corresponding ground-truth transcription. The method includes identifying, as an oracle hypothesis, the speech recognition hypothesis from the n-best list having the smallest corresponding number of word errors relative to the corresponding ground-truth transcription, and determining a second loss for the oracle hypothesis based on the corresponding ground-truth transcription. The method includes determining a corresponding self-training combined loss based on the first and second losses, and training the model based on the corresponding self-training combined loss.
What exactly they claim (claim 1)
1. A computer-implemented method executed on data processing hardware that causes the data processing hardware to perform operations comprising: receiving a plurality of training samples, each training sample of the plurality of training samples comprising: a corresponding sequence of acoustic frames characterizing a corresponding training utterance; and a corresponding ground-truth transcription of the corresponding training utterance; for each particular training sample of the plurality of training samples: processing, using an automatic speech recognition (ASR) model, the corresponding sequence of acoustic frames to obtain an n-best list of speech recognition hypotheses for the corresponding training utterance; for each speech recognition hypothesis of the n-best list of speech recognition hypotheses for the corresponding training utterance, determining a corresponding number of word errors relative to the corresponding ground-truth transcription; for a top-ranked hypothesis from the n-best list of speech recognition hypotheses, determining a first loss based on the corresponding ground-truth transcription; identifying, as an oracle hypothesis, the speech recognition hypothesis from the n-best list of speech recognition hypotheses having the smallest corresponding number of word errors relative to the corresponding ground-truth transcription; determining a second loss for the oracle hypothesis based on the corresponding ground-truth transcription; and determining a corresponding combined loss based on the first and second losses; and training the ASR model based on the corresponding combined losses determined for the plurality of training samples.
Facts
- Company
- Google (Google LLC)
- Official title
- Self-training with oracle and top-ranked hypotheses
- Patent number
- US12738265B2
- Application number
- 18590918
- Filed
- Feb 28, 2024
- Granted
- Sep 15, 2026
- Inventors
- 3
- Claims
- 21
Classified under
G10L 15/063 Physics & computingG10L 15/01 Physics & computingG10L 15/16 Physics & computingG10L 15/197 Physics & computingMore from Google in AI & software
- Google patents video search that answers questions using AI descriptions Sep 15, 2026
- Google patents photo tool that removes people and their gear Sep 15, 2026
- Google patents AI that writes live meeting notes and summaries Sep 15, 2026
- Google patents AI tool that turns messy emergency notes into clear messages Sep 3, 2026
- Google patents AI that clicks through apps to finish tasks for you Sep 24, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.