Apple patents eye-tracking to adjust camera resolution in mixed reality
Apple describes a method for mixed-reality devices that tracks where a user is looking to adjust how the device's cameras capture the real world. If the user is looking at virtual content, the system changes the camera's 'binning mode' to capture the background at a specific resolution for the next frame.
In the filing’s own words
A method is provided that includes determining a gaze position of a user relative to mixed-reality content displayed in a first frame, setting a binning mode for a first camera based on the determined gaze position, and capturing, using the first camera, passthrough content for a second frame at a resolution determined by the binning mode.
What exactly they claim (claim 1)
1. A method, comprising: determining a gaze position of a user relative to mixed-reality content displayed in a first frame, wherein the mixed-reality content displayed in the first frame comprises rendered virtual content, and wherein the gaze position of the user is determined relative to the rendered virtual content; and in response to determining that the gaze position lies within a region of the first frame occupied by the rendered virtual content: setting a binning mode for a first camera based on the determined gaze position; and capturing, using the first camera, passthrough content for a second frame at a resolution determined by the binning mode.
Facts
- Company
- Apple (Apple Inc.)
- Official title
- Dynamic binning passthrough content
- Patent number
- US12732589B2
- Application number
- 18110365
- Filed
- Feb 15, 2023
- Granted
- Sep 8, 2026
- Inventors
- 2
- Claims
- 20
Classified under
H04N 5/265 Electricity & electronicsG06F 3/013 Physics & computingG06T 3/40 Physics & computingH04N 23/667 Electricity & electronicsH04N 23/80 Electricity & electronicsH04N 25/46 Electricity & electronicsMore from Apple in AR, VR & displays
- Apple patents AR glasses that zoom in on what you look at Sep 8, 2026
- Apple patents finger ring that controls VR with taps and waves Sep 15, 2026
- Apple patents head-mounted device that types via eye swipes and hand pinches Oct 6, 2026
- Apple patents eye-tracking and hand gestures to select screen items Oct 20, 2026
- Apple patents AR glasses that adjust lenses to fit your eyes Sep 17, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.