AppleGranted patentAI & softwareInterest 7/10

Apple patents assistant that acts on what it sees

Patent drawing
Patent drawing from the official filing.

Apple has patented a system where a digital assistant uses visual data to identify specific subjects, such as other connected devices. The assistant then proactively performs an action on that subject, like changing its state, without waiting for a direct command.

Why it mattersThis could make smart home setups or device interactions feel more intuitive by allowing your phone to anticipate needs based on what is physically present or visible.

In the filing’s own words

Systems and processes for operating an intelligent automated assistant are provided. In some embodiments, after providing an initial output with information on a particular topic, an output including updated information on the topic is provided in response to an input interacting with a digital assistant. In some embodiments, based on contextual data including visual context data, an action is identified and proactively performed.

What exactly they claim (claim 1)

1. An electronic device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: identifying, based on current contextual information, a first action available to be performed by a digital assistant, wherein the current contextual information includes visual context information; in response to identifying the first action, initiating performance of the first action; and after initiating the performance of the first action, providing a first output indicating that the digital assistant has initiated the performance of the first action, wherein identifying the first action available to be performed by the digital assistant includes: identifying, based on the visual context information, a first subject, wherein the first action can be performed by the digital assistant with respect to the first subject, wherein: the first subject includes a second electronic device that is in communication with the electronic device; performance of the first action includes causing a state of the second electronic device to change; and the first output indicating that the digital assistant has initiated the performance of the first action includes an indication that the digital assistant caused the state of the second electronic device to change.

Facts

Company
Apple (Apple Inc.)
Official title
Proactive surfacing and performance of contextual actions
Patent number
US12801341B2
Application number
18629312
Filed
Apr 8, 2024
Granted
Nov 17, 2026
Inventors
3
Claims
39

Classified under

G06F 3/167 Physics & computingG06F 3/013 Physics & computingG06F 11/3438 Physics & computingH04M 1/72454 Electricity & electronics

More from Apple in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.