Apple patents system to identify real-world objects via voice or gesture
Apple has filed a patent for a system that helps devices understand which specific object a user is pointing at or talking about. The technology uses a database of known locations and sensors to narrow down options, displaying them for the user to select without touching the screen.
In the filing’s own words
Implementations described and claimed herein provide systems and methods for interaction between a user and a machine. In one implementation, a system is provided that receives an input from a user of a mobile machine which indicates or describes an object in the world. In one example, the user may gesture to the object which is detected by a visual sensor. In another example, the user may verbally describe the object which is detected by an audio sensor. The system receiving the input may then determine which object near the location of the user that the user is indicating. Such a determination may include utilizing known objects near the geographic location of the user or the autonomous or mobile machine.
What exactly they claim (claim 1)
1. A method for interaction between a user and a device, the method comprising: receiving a request from the user through an input system, the request including a reference to an object type in an external environment; obtaining a subset of real world object indicators from a database of known real world object indicators, the subset of real world object indicators being identified based on the reference to the object type and an estimated geographic location of the device, the subset of real world object indicators corresponding to at least one potential match to the request; displaying a representation of each real world object indicator in the subset of real world object indicators; receiving a non-tactile selection of one of the representations of the real world object indicators in the subset of real world object indicators through the input system; and determining an object corresponding to the request by disambiguating the non-tactile selection.
Facts
- Company
- Apple (Apple Inc.)
- Official title
- System and Methods for Relative Representation of Spatial Objects and Disambiguation in an Interface
- Publication number
- US20260287888A1
- Application number
- 19693257
- Filed
- May 29, 2026
- Published
- Sep 24, 2026
- Inventors
- 7
- Claims
- 1
Classified under
G02B 27/01 Physics & computingG06F 3/017 Physics & computingG06F 3/04817 Physics & computingG06F 16/58 Physics & computingG06F 16/587 Physics & computingH04W 4/027 Electricity & electronicsMore from Apple in AR, VR & displays
- Apple patents AR glasses that zoom in on what you look at Sep 8, 2026
- Apple patents finger ring that controls VR with taps and waves Sep 15, 2026
- Apple patents head-mounted device that types via eye swipes and hand pinches Oct 6, 2026
- Apple patents eye-tracking and hand gestures to select screen items Oct 20, 2026
- Apple patents AR glasses that adjust lenses to fit your eyes Sep 17, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.