NvidiaPatent applicationAI & softwareInterest 8/10

Nvidia patents system that translates specific speakers in group conversations

Illustration of the invention
Illustration based on the patent drawing. Not a photo of a real product.
Patent drawing
Patent drawing from the official filing.

Nvidia describes a single machine learning model that identifies a specific person's voice in a noisy group conversation and translates their words directly into another language. The system uses a short audio sample to create a voice profile, allowing it to ignore other speakers and skip the usual steps of first transcribing the audio and then translating the text.

Why it mattersThis could make live translation in meetings or social gatherings more accurate by focusing only on the person you want to hear, rather than trying to process everyone talking at once.

In the filing’s own words

In various examples, AI-based techniques may be used to translate speech of specific speakers in multi-speaker environments. For example, the systems and methods of the present disclosure may integrate target speaker extraction with automatic speech translation (AST) into a single AST model. Based on this integration, the AST model may be used to extract the speech of a specified speaker from mixed audio and directly translate the speech into text of a target language without intermediate transcription. For instance, the AST model may receive a speech sample for the speaker and use the sample to identify and extract the target speaker’s voice from a dialogue. The AST model may then process the extracted speech and generate a direct translation without requiring separate automatic speech recognition (ASR) and machine translation (MT) stages.

What exactly they claim (claim 1)

1. A method comprising: generating, using one or more first encoders of a machine learning model and based at least on first audio data representing speech of a target speaker, a voice profile associated with the target speaker; determining, using one or more second encoders of the machine learning model and based at least on the voice profile and second audio data representing a multi-speaker dialogue, one or more portions of the multi-speaker dialogue that correspond to the target speaker; and generating, using one or more decoders of the machine learning model, output data representing text or speech corresponding to one or more translations of the one or more portions of the multi-speaker dialogue from a first language into one or more second languages.

Facts

Company
Nvidia (NVIDIA Corporation)
Official title
IDENTITY-BASED SPEAKER TARGETING FOR AUTOMATIC SPEECH TRANSLATION SYSTEMS AND APPLICATIONS
Publication number
US20260360368A1
Application number
19205716
Filed
May 12, 2025
Published
Nov 12, 2026
Inventors
5
Claims
20

Classified under

G10L 17/26 Physics & computingG06F 40/58 Physics & computingG10L 15/005 Physics & computingG10L 17/18 Physics & computingG10L 21/0208 Physics & computingG10L 2021/02087 Physics & computing

More from Nvidia in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.