Apple patents AR hands that know where to grab real objects
Apple describes a method for making virtual hands in augmented reality interact realistically with physical objects. The system identifies the type of object, such as a cup or a door handle, and uses pre-defined data to determine the most likely spots a person would touch. It then maps these touch zones to the specific object in front of you so the virtual hand grabs it naturally.
In the filing’s own words
Generating a virtual representation of an interaction includes determining a potential user interaction with a physical object in a physical environment, determining an object type associated with the physical object, and obtaining an object-centric affordance region for the object type, wherein the object-centric affordance region indicates, for each of one or more regions of the object type, a likelihood of user contact. The object-centric affordance region is mapped to a geometry of the physical object to obtain an instance-specific affordance region, is used to render the virtual representation of the interaction with the physical object.
What exactly they claim (claim 1)
1. A method comprising: detecting, based on sensor data of a view of a physical environment, a first user interaction with a physical object in the physical environment; determining an object type of the physical object; obtaining an object-centric affordance region for the object type, wherein the object-centric affordance region indicates, for each of one or more regions of a geometry representative of the object type, a likelihood of user contact; applying the object-centric affordance region to a virtual representation of the physical object to obtain an instance-specific affordance region; and generating a virtual representation of a user performing the first user interaction with the virtual representation of the physical object in accordance with the instance-specific affordance region.
Facts
- Company
- Apple (Apple Inc.)
- Official title
- Smart interactivity for scanned objects using affordance regions
- Patent number
- US12749270B2
- Application number
- 18330652
- Filed
- Jun 7, 2023
- Granted
- Sep 29, 2026
- Inventors
- 3
- Claims
- 18
Classified under
G06T 19/006 Physics & computingG06T 15/10 Physics & computingG06T 2200/24 Physics & computingMore from Apple in AR, VR & displays
- Apple patents AR glasses that zoom in on what you look at Sep 8, 2026
- Apple patents finger ring that controls VR with taps and waves Sep 15, 2026
- Apple patents head-mounted device that types via eye swipes and hand pinches Oct 6, 2026
- Apple patents eye-tracking and hand gestures to select screen items Oct 20, 2026
- Apple patents AR glasses that adjust lenses to fit your eyes Sep 17, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.