NvidiaGranted patentAudio & voiceInterest 6/10

Nvidia patents audio tech that groups similar microphones to identify speakers

Nvidia describes a system that uses machine learning to figure out who is speaking in a room with many microphones. It groups similar audio signals together to reduce data volume before a neural network identifies the specific speakers.

Why it mattersThis could improve how smart speakers and conference systems distinguish between multiple people talking at once.

In the filing’s own words

Disclosed are apparatuses, systems, and techniques that may use machine learning for implementing speaker recognition, verification, and/or diarization. The techniques include receiving a first set of audio data channels (ADCs) jointly capturing a speech produced by one or more speakers and obtaining, using the first set of ADCs, a second set of one or more ADCs. Individual ADCs of the second set of ADCs represent one or more channels of the first set of ADCs, and at least one channel of the second set of ADCs represents a cluster of two or more ADCs of the first set of ADCs, the two of more ADCs being selected based on similarity of audio data of the two or more ADCs. The techniques further include processing, using an audio processing neural network model, the second set of ADCs to obtain an association of the speech to the one or more speakers.

What exactly they claim (claim 1)

1. A method comprising: receiving a first set of audio data channels (ADCs) jointly capturing a speech produced by one or more speakers; obtaining, using the first set of ADCs, a second set of ADCs, at least one ADC of the second set of ADCs representing a cluster of two or more ADCs of the first set of ADCs, the two or more ADCs of the first set of ADCs being selected based at least on a similarity of audio data of the two or more ADCs of the first set of ADCs; processing the second set of ADCs using a voice detection model to determine a set of voice activity likelihoods (VALs) that individual ADCs of the second set of ADCs comprise the speech; combining a set of embeddings associated with the second set of ADCs to obtain one or more mixed embeddings, wherein combining the set of embeddings comprises multiplying each embedding of the set of embeddings by a respective VAL of the set of VALs; and processing, using an audio processing neural network (NN) model, the third set of one or more mixed embeddings to obtain one or more associations of the speech to the one or more speakers.

Facts

Company
Nvidia (NVIDIA Corporation)
Official title
Audio processing in multi-speaker multi-channel audio environments
Patent number
US12793720B2
Application number
18410794
Filed
Jan 11, 2024
Granted
Nov 10, 2026
Inventors
9
Claims
20

Classified under

G10L 17/18 Physics & computingG10L 17/02 Physics & computingG10L 17/20 Physics & computingG10L 25/30 Physics & computingG10L 25/78 Physics & computingG10L 2025/783 Physics & computing

More from Nvidia in Audio & voice

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.