Apple patents video overlays that track players via voice commands
Apple has been granted a patent for a digital assistant feature that adds graphical overlays to live video events. When a user speaks about a specific participant, the system uses context and where the user is looking to identify that person's location on the screen. It then places a visual marker or overlay at that exact spot to highlight the participant.
In the filing’s own words
An example process includes while displaying, on a display, a video event: receiving, by a digital assistant, a natural language speech input corresponding to a participant of the video event; in accordance with receiving the natural language speech input, identifying, by the digital assistant, based on context information associated with the video event, a first location of the participant; and in accordance with identifying the first location of the participant, augmenting, by the digital assistant, the display of the video event with a graphical overlay displayed at a first display location corresponding to the first location of the participant.
What exactly they claim (claim 1)
1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device having a display, cause the electronic device to: display, via the display, a video event; and while displaying, via the display, the video event: receive a first natural language input that corresponds to a first participant of the video event; while receiving the first natural language input that corresponds to the first participant of the video event, detect that a gaze of a user is directed to a first location; and in response to receiving the first natural language input that corresponds to the first participant of the video event, and in accordance with a determination that the first natural language input refers to the first participant in the present tense: identify a first respective location of the first participant based on the first location; and augment, via the display, the video event with a first graphical overlay that is displayed at a second location that corresponds to the first respective location of the first participant.
Facts
- Company
- Apple (Apple Inc.)
- Official title
- Digital assistant for providing graphical overlays of video events
- Patent number
- US12795192B2
- Application number
- 19279702
- Filed
- Jul 24, 2025
- Granted
- Nov 10, 2026
- Inventors
- 4
- Claims
- 42
Classified under
H04N 21/4312 Electricity & electronicsG06F 3/013 Physics & computingG06F 3/017 Physics & computingG06F 3/1431 Physics & computingG06F 3/167 Physics & computingG10L 15/1822 Physics & computingMore from Apple in AI & software
- Apple patents video call feature that fixes participants' eye contact Nov 17, 2026
- Apple patents AI that acts on voicemails for you Oct 15, 2026
- Apple patents letting users teach custom hand gestures to their devices Oct 22, 2026
- Apple patents AI that acts on voicemails for you Sep 1, 2026
- Apple patents video tool that swaps focus between subjects after filming Oct 20, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.