Apple patents widgets that stick to real-world objects
Apple describes a method where a device uses its camera to identify physical objects and their meaning. It then displays a digital widget on the screen that stays anchored to that object, even as the device moves.
In the filing’s own words
According to various implementations, a method is performed at an electronic device with one or more processors, a non-transitory memory, a display. The electronic device optionally also includes an image sensor and input device(s). The method includes obtaining a semantic value that is associated with a physical object, based on image data. The image data is associated with a first input modality. In some implementations, the method includes obtaining a widget based on the semantic value, and displaying the widget according to an object-proximity criterion (e.g., display-locked, body-locked, or world-locked) with respect to the physical object. In some implementations, the method includes obtaining user data from the input device(s). The user data is associated with a second input modality that is different from the first input modality. Moreover, the method includes selecting a widget based on the semantic value and the user data, and displaying the widget.
What exactly they claim (claim 1)
1. A method comprising: at a device including a non-transitory memory, one or more processors, and a display: obtaining a semantic value that is associated with a physical object within a physical environment of the device; selecting a widget based on the semantic value; obtaining position data that characterizes a current pose or pose change of the device; in accordance with a determination that the position data satisfies position criteria, displaying the widget on the display; and in accordance with a determination that the position data does not satisfy the position criteria, forgoing display of the widget.
Facts
- Company
- Apple (Apple Inc.)
- Official title
- Displaying a Contextualized Widget
- Publication number
- US20260288301A1
- Application number
- 19695226
- Filed
- Jun 2, 2026
- Published
- Sep 24, 2026
- Inventors
- 4
- Claims
- 20
Classified under
G06F 3/04815 Physics & computingG06F 1/1694 Physics & computingG06F 3/013 Physics & computingG06F 3/0346 Physics & computingG06F 3/167 Physics & computingMore from Apple in AR, VR & displays
- Apple patents AR glasses that zoom in on what you look at Sep 8, 2026
- Apple patents finger ring that controls VR with taps and waves Sep 15, 2026
- Apple patents head-mounted device that types via eye swipes and hand pinches Oct 6, 2026
- Apple patents eye-tracking and hand gestures to select screen items Oct 20, 2026
- Apple patents AR glasses that adjust lenses to fit your eyes Sep 17, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.