Apple patents eye-tracking and hand gestures to select screen items
Apple has been granted a patent for a system that combines eye tracking with hand movements to control a screen. The method detects when a user is looking at a specific item and makes an air gesture, using whether their eyes are open or closed to confirm or cancel the selection.
In the filing’s own words
The present disclosure generally relates to managing inputs. In some examples, one or more techniques are described, such as managing location-defined inputs, enabling certain inputs, interacting with locked objects, interacting with interface objects, virtual controller-based inputs, and gaze directed inputs.
What exactly they claim (claim 1)
1. A method, comprising: at a computer system that is in communication with a display generation component and one or more input devices: while presenting, via the display generation component, a user interface object, detecting a gaze input directed to the user interface object; after detecting the gaze input directed to the user interface object, detecting, via the one or more input devices, an air gesture, wherein the air gesture is detected before detecting that the gaze input is directed to a location that does not correspond to the user interface object; and in response to detecting the air gesture: in accordance with a determination that one or more eyes of the user were opened in conjunction with the air gesture being detected, selecting the user interface object; and in accordance with a determination that one or more eyes of the user were closed for more than a threshold amount of time in conjunction with the air gesture being detected, forgoing selection of the user interface object; and repeating the method until the method is performed at least one time when the one or more eyes of the user were opened in conjunction with the respective air gesture being detected and at least one time when one or more eyes of the user were closed for more than the threshold amount of time in conjunction with the respective air gesture being detected.
Facts
- Company
- Apple (Apple Inc.)
- Official title
- Devices, methods, and graphical user interfaces for detecting inputs
- Patent number
- US12769431B2
- Application number
- 18616383
- Filed
- Mar 26, 2024
- Granted
- Oct 20, 2026
- Inventors
- 2
- Claims
- 36
Classified under
G06F 3/0481 Physics & computingG06F 3/013 Physics & computingG06F 3/017 Physics & computingMore from Apple in AR, VR & displays
- Apple patents AR glasses that zoom in on what you look at Sep 8, 2026
- Apple patents finger ring that controls VR with taps and waves Sep 15, 2026
- Apple patents head-mounted device that types via eye swipes and hand pinches Oct 6, 2026
- Apple patents AR glasses that adjust lenses to fit your eyes Sep 17, 2026
- Apple patents showing real-world objects in virtual video calls Oct 1, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.