MicrosoftPatent applicationAI & softwareInterest 8/10

Microsoft patents AI that edits live video and audio in real time

Patent drawing
Patent drawing from the official filing.

Microsoft describes a system that uses AI to instantly change what people say in live video or AR streams. It swaps specific words with new ones and smooths the audio and video transitions so the changes look natural.

Why it mattersThis could allow broadcasters or streamers to fix mistakes or tailor content to specific viewers without stopping the live feed.

In the filing’s own words

Systems and methods are provided for implementing near-real-time artificial intelligence (“AI”) enhancement of video or augmented reality (“AR”) presentations of live events. In examples, a computing system identifies, from a plurality of spoken words in a live audio recording of individuals speaking, spoken words to modify, based on a set of parameters and/or a profile of a user, using an AI agent. The computing system causes a generative AI system to modify, in near-real-time, (a) the identified spoken words with corresponding replacement words in each of the live video recording and the corresponding live audio recording, based on the profile of the user; (b) portions of the live video recording and the corresponding live audio recording to generate video and audio smooth transitions before and after the replacement words are incorporated in the live video recording and the corresponding live audio recording, respectively.

What exactly they claim (claim 1)

1. A system, comprising: a processing system; and memory comprising computer executable instructions that, when executed by the processing system, causes the system to perform operations comprising: scanning, using an artificial intelligence (“AI”) agent and in near-real-time, a live video recording and a corresponding live audio recording of one or more individuals speaking; identifying, from a plurality of spoken words as scanned from the live audio recording, one or more spoken words to modify according to a first set of parameters and based on a profile of a user, using the AI agent; causing near-real-time generation of one or more replacement portions of the live video recording and the corresponding live audio recording, by: causing a generative AI system to modify, in near-real-time, the identified one or more spoken words with corresponding one or more replacement words in each of the live video recording and the corresponding live audio recording, based on the profile of the user; and causing the generative AI system to modify, in near-real-time, one or more portions of each of the live video recording and the corresponding live audio recording to generate video and audio smooth transitions before and after the one or more replacement words are incorporated in the live video recording and the corresponding live audio recording, respectively; and presenting, in near-real-time, the one or more replacement portions of the live video recording and the corresponding live audio recording associated with the one or more replacement words and the video and audio smooth transitions.

Facts

Company
Microsoft (Microsoft Technology Licensing, LLC)
Official title
NEAR REAL-TIME ARTIFICIAL INTELLIGENCE (AI) ENHANCEMENT OF VIDEO OR AUGMENTED REALITY (AR) PRESENTATIONS OF LIVE EVENTS
Publication number
US20260341309A1
Application number
19193093
Filed
Apr 29, 2025
Published
Oct 29, 2026
Inventors
2
Claims
20

Classified under

G11B 27/036 Physics & computingG06F 40/58 Physics & computingG10L 15/005 Physics & computingG10L 15/10 Physics & computingG10L 15/30 Physics & computingG10L 25/57 Physics & computing

More from Microsoft in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.