Apple patents voice commands that change based on what you see
Apple has filed a patent for a system that lets an intelligent assistant understand voice commands based on the specific screen you are looking at. The device identifies words or phrases linked to items on your screen, so saying a specific keyword triggers an action on that exact item.
In the filing’s own words
Systems and processes for operating an intelligent automated assistant are provided. In some embodiments, contextual data is obtained and used to select a set of keywords (e.g., words or phrases) for voice control of an electronic device. When a speech input is received by the electronic device, a determination is made whether the speech input includes any of the selected keywords. If the speech input does include a selected keyword, an action is performed in response.
What exactly they claim (claim 1)
1. An electronic device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: displaying a first user interface including at least a first element; selecting a first set of one or more keywords, wherein a first keyword of the one or more keywords corresponds to the first element of the first user interface; receiving a speech input; determining whether the speech input includes at least one keyword of the first set of one or more keywords; and in accordance with a determination that the speech input includes the first keyword of the first set of one or more keywords, causing a first action to be performed with respect to the first element of the first user interface.
Facts
- Company
- Apple (Apple Inc.)
- Official title
- VOICE CONTROL WITH CONTEXTUAL KEYWORDS
- Publication number
- US20260260653A1
- Application number
- 19656751
- Filed
- Apr 23, 2026
- Published
- Sep 3, 2026
- Inventors
- 5
- Claims
- 14
Classified under
G10L 15/22 Physics & computingG10L 15/02 Physics & computingG10L 15/1815 Physics & computingMore from Apple in AI & software
- Apple patents AI that acts on voicemails for you Oct 15, 2026
- Apple patents letting users teach custom hand gestures to their devices Oct 22, 2026
- Apple patents AI that acts on voicemails for you Sep 1, 2026
- Apple patents video tool that swaps focus between subjects after filming Oct 20, 2026
- Apple patents system that changes room ambience based on what you look at Sep 3, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.