Amazon patents tech to swap words in movies using actor's voice
This patent describes a method for changing specific words spoken by actors in a video. The system identifies the original word, removes it, and generates a new word using a digital profile of the actor's voice to fit the same time slot.
In the filing’s own words
A method for modifying recorded audio of a video title may include: identifying a video title available for presentation; identifying a portion of audio data, of the video title, representing dialogue of an actor of the video title; identifying a first utterance of the dialogue to be replaced in the video title; selecting, based on a replaceable amount of the audio data, a second utterance to replace the utterance word in the portion of the audio data; generating, based on a voice profile of the actor, first speech signals representing the actor uttering the second utterance; removing second speech signals representing the actor uttering the first utterance from the dialogue; adding the first speech signals into the dialogue using at least a portion of the replaceable amount; and generating a modified version of the video title comprising the dialogue with the speech signals added.
What exactly they claim (claim 1)
1. A method for modifying recorded audio of a video title, the method comprising: identifying, by at least one processor of a device, a video title available for presentation; identifying, by the at least one processor, a portion of audio data, of the video title, representing dialogue spoken by an actor of the video title; identifying, by the at least one processor, the actor who spoke the dialogue; dividing, by the at least one processor, the portion of the audio data into a first portion representing the dialogue and a second portion representing background noise during the dialogue; identifying, by the at least one processor, a first lexical word of the dialogue to be replaced in the video title; identifying, by the at least one processor, a first time at which the actor begins to speak the first lexical word and a second time at which the actor finishes speaking the first lexical word; selecting, by the at least one processor, based on a time duration between the first time and the second time, a second lexical word different than the first lexical word to replace the first lexical word in the portion of the audio data; identifying, by the at least one processor, a voice profile of the actor, wherein the voice profile represents audio samples of a voice of the actor from at least one of the video title or an additional video title; generating, by the at least one processor, based on the voice profile, first speech signals simulating the actor speaking the second lexical word; removing, by the at least one processor, second speech signals representing the actor speaking the first lexical word from the dialogue; adding, by the at least one processor, the first speech signals into the dialogue using at least a portion of the time duration; combining, by the at least one processor, the background noise with the dialogue into which the first speech signals were combined; and generating, by the at least one processor, a modified version of the video title comprising the background noise with the dialogue into which the first speech signals were added.
Facts
- Company
- Amazon (Amazon Technologies, Inc.)
- Official title
- Enhanced spoken dialogue modification
- Patent number
- US12731573B2
- Application number
- 18325516
- Filed
- May 30, 2023
- Granted
- Sep 8, 2026
- Inventors
- 4
- Claims
- 20
Classified under
G10L 13/027 Physics & computingG10L 21/0208 Physics & computingH04N 21/4307 Electricity & electronicsG10L 2013/021 Physics & computingMore from Amazon in AI & software
- Amazon patents system that cites sources for AI answers Sep 29, 2026
- Amazon patents ML tool to add graphics to live sports from moving cameras Sep 15, 2026
- Amazon patents shopping assistant that remembers your search filters Oct 6, 2026
- Amazon patents system where AI agents split tasks and talk to each other Oct 20, 2026
- Amazon patents system that picks the right AI model for your request Oct 1, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.