Apple patents assistant that acts on what it sees
Apple has patented a system where a digital assistant uses visual data to identify specific subjects, such as other connected devices. The assistant then proactively performs an action on that subject, like changing its state, without waiting for a direct command.
In the filing’s own words
Systems and processes for operating an intelligent automated assistant are provided. In some embodiments, after providing an initial output with information on a particular topic, an output including updated information on the topic is provided in response to an input interacting with a digital assistant. In some embodiments, based on contextual data including visual context data, an action is identified and proactively performed.
What exactly they claim (claim 1)
1. An electronic device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: identifying, based on current contextual information, a first action available to be performed by a digital assistant, wherein the current contextual information includes visual context information; in response to identifying the first action, initiating performance of the first action; and after initiating the performance of the first action, providing a first output indicating that the digital assistant has initiated the performance of the first action, wherein identifying the first action available to be performed by the digital assistant includes: identifying, based on the visual context information, a first subject, wherein the first action can be performed by the digital assistant with respect to the first subject, wherein: the first subject includes a second electronic device that is in communication with the electronic device; performance of the first action includes causing a state of the second electronic device to change; and the first output indicating that the digital assistant has initiated the performance of the first action includes an indication that the digital assistant caused the state of the second electronic device to change.
Facts
- Company
- Apple (Apple Inc.)
- Official title
- Proactive surfacing and performance of contextual actions
- Patent number
- US12801341B2
- Application number
- 18629312
- Filed
- Apr 8, 2024
- Granted
- Nov 17, 2026
- Inventors
- 3
- Claims
- 39
Classified under
G06F 3/167 Physics & computingG06F 3/013 Physics & computingG06F 11/3438 Physics & computingH04M 1/72454 Electricity & electronicsMore from Apple in AI & software
- Apple patents video call feature that fixes participants' eye contact Nov 17, 2026
- Apple patents AI that acts on voicemails for you Oct 15, 2026
- Apple patents letting users teach custom hand gestures to their devices Oct 22, 2026
- Apple patents AI that acts on voicemails for you Sep 1, 2026
- Apple patents video tool that swaps focus between subjects after filming Oct 20, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.
