/Patent Tracker
GoogleGranted patentAudio & voiceInterest 5/10

Google patents speech tech that links letters and sounds for better text-to-speech

This patent describes a method for creating computer-generated speech by processing text as both letters (graphemes) and sounds (phonemes). The system matches each sound token with its corresponding letter tokens to create a combined data structure that helps a speech model produce more accurate audio.

Why it mattersThis improves the accuracy of text-to-speech systems by ensuring the model correctly aligns written words with their spoken sounds.

In the filing’s own words

A method includes receiving a text input including a sequence of words represented as an input encoder embedding. The input encoder embedding includes a plurality of tokens, with the plurality of tokens including a first set of grapheme tokens representing the text input as respective graphemes and a second set of phoneme tokens representing the text input as respective phonemes. The method also includes, for each respective phoneme token of the second set of phoneme tokens: identifying a respective word of the sequence of words corresponding to the respective phoneme token and determining a respective grapheme token representing the respective word of the sequence of words corresponding to the respective phoneme token. The method also includes generating an output encoder embedding based on a relationship between each respective phoneme token and the corresponding grapheme token determined to represent a same respective word as the respective phoneme token.

What exactly they claim (claim 1)

1. A computer-implemented method executed by data processing hardware causes the data processing hardware to perform operations comprising: receiving, at an encoder of a speech synthesis model, a text input comprising a sequence of words represented as an input encoder embedding, the input encoder embedding comprising set of grapheme tokens representing the text input as respective graphemes and a set of phoneme tokens representing the text input as respective phonemes, wherein the input encoder embedding represents, for each respective word in the sequence of words, respective sub-word level positions for both one or more of the grapheme tokens from the set of grapheme tokens that correspond to the respective word and one or more of the phoneme tokens from the set of phoneme tokens that correspond to the respective word; for each respective phoneme token of the set of phoneme tokens: identifying, by the encoder, a respective word of the sequence of words corresponding to the respective phoneme token; and determining, by the encoder, one or more respective grapheme tokens representing the same respective word of the sequence of words corresponding to the respective phoneme token based on the respective sub-word level position for the respective phoneme token that corresponds to the respective word and the respective sub-word level position for each of the one or more respective grapheme tokens that correspond to the same respective word; and generating, by the encoder, an output encoder embedding for use by a decoder of the speech synthesis model to synthesize speech from the text input, the output encoder embedding based on a relationship between each respective phoneme token and the one or more respective grapheme tokens determined to represent a same respective word as the respective phoneme token.

Facts

Company
Google (Google LLC)
Official title
Phonemes and graphemes for neural text-to-speech
Patent number
US12744029B2
Application number
18746809
Filed
Jun 18, 2024
Granted
Sep 22, 2026
Inventors
5
Claims
20

Classified under

G10L 13/086 Physics & computingG06F 40/263 Physics & computingG06F 40/279 Physics & computingG06N 3/08 Physics & computingG10L 13/047 Physics & computing
Read the full filing (official PDF) ↗ Open in the Patent Tracker

More from Google in Audio & voice

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.