Apple patents letting users teach custom hand gestures to their devices
Apple describes a system where a device's camera captures your hand movements and identifies them as specific commands you have personally defined. It uses a simplified outline of your hand's position to recognize these custom gestures, allowing you to create new controls without needing a large amount of training data.
In the filing’s own words
The subject technology provides vision-based hand gesture customization. An electronic device receives image data captured by an imaging sensor and representing performance of a hand gesture by a user. The electronic device extracts a skeleton-based feature representation from the image data and applies a trained machine learning model to the skeleton-based feature representation to generate a classification of the hand gesture as corresponding to a user-customized gesture class. Based on the classification, the electronic device causes an output or device operation associated with the user-customized gesture class. The trained machine learning model is generated by further training an initial machine learning model using training samples representing the user-customized gesture class. By utilizing skeleton-based feature representations and incremental training of a preexisting model, the subject technology can improve gesture-recognition robustness across imaging conditions, reduce training data requirements for personalized gestures, decrease model-development overhead, and enable deployment of user-defined gesture controls.
What exactly they claim (claim 1)
1. A method performed by an electronic device, comprising: receiving, by the electronic device, image data captured by an imaging sensor of the electronic device and representing performance of a hand gesture by a user; extracting, by the electronic device, a skeleton-based feature representation from the image data; applying, by the electronic device, a trained machine learning model to the skeleton-based feature representation to generate a classification of the hand gesture as corresponding to a user-customized gesture class; and causing, by the electronic device, an output or device operation corresponding to the classification, wherein the trained machine learning model was generated by further training an initial machine learning model using training samples representing the user-customized gesture class.
Facts
- Company
- Apple (Apple Inc.)
- Official title
- VISION-BASED HAND GESTURE CUSTOMIZATION
- Publication number
- US20260330877A1
- Application number
- 19730492
- Filed
- Jul 2, 2026
- Published
- Oct 22, 2026
- Inventors
- 5
- Claims
- 20
Classified under
G06V 40/28 Physics & computingG06V 10/44 Physics & computingG06V 10/764 Physics & computingG06V 10/774 Physics & computingG06V 10/82 Physics & computingMore from Apple in AI & software
- Apple patents AI that acts on voicemails for you Oct 15, 2026
- Apple patents AI that acts on voicemails for you Sep 1, 2026
- Apple patents video tool that swaps focus between subjects after filming Oct 20, 2026
- Apple patents system that changes room ambience based on what you look at Sep 3, 2026
- Apple patents voice commands that use cameras to find specific devices Sep 17, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.