AppleGranted patentAR, VR & displaysInterest 6/10

Apple patents system that mixes real and virtual objects to run commands

Patent drawing
Patent drawing from the official filing.

Apple describes a method where a device uses sensors to map a physical space and creates a single model that links real items with virtual ones. This combined map allows the system to understand how objects relate to each other, enabling it to carry out specific user commands based on those connections.

Why it mattersThis approach could make augmented reality experiences feel more natural by allowing virtual items to interact with real-world objects in a logical way.

In the filing’s own words

Various implementations disclosed herein include devices, systems, and methods that uses object relationships represented in a scene graph to execute an instruction. For example, an example process may include obtaining sensor data for a physical environment including physical objects, determining a scene understanding of the physical environment based on the sensor data, generating a unified scene graph based on the scene understanding that represents the physical objects, virtual objects, and relationships amongst the physical objects and the virtual objects, receiving an instruction for an action, and implementing the action based on the unified scene graph.

What exactly they claim (claim 1)

1. A method comprising: at an electronic device having a processor: obtaining sensor data for a physical environment comprising physical objects; determining a scene understanding of the physical environment based on the sensor data; generating a unified object relationship model based on the scene understanding, the unified object relationship model representing the physical objects, virtual objects, and relationships amongst the physical objects and the virtual objects; receiving an instruction for an action; and implementing the action based on the unified object relationship model.

Facts

Company
Apple (Apple Inc.)
Official title
Instructional interactions using scene graphs
Patent number
US12802363B1
Application number
17678629
Filed
Feb 23, 2022
Granted
Nov 17, 2026
Inventors
3
Claims
20

Classified under

G06T 19/006 Physics & computingG06F 3/04815 Physics & computingG06F 3/167 Physics & computingG06T 19/20 Physics & computingG06T 2219/2004 Physics & computing

More from Apple in AR, VR & displays

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.