Microsoft patents gestures that change meaning based on device position
This patent describes a system where a specific user gesture triggers different actions depending on how the device is oriented relative to other devices. The system learns these associations by observing how users manually interact with the device in different physical settings.
In the filing’s own words
Aspects of the present disclosure relate to semantic user input for a computing device. In examples, user input is identified and processed to identify and automatically perform an associated semantic action. The semantic action may be determined based at least in part on an environmental context associated with the user input. Thus, an action determined for a given user input may change according to the environmental context in which the input was received. For example, an association between user input, an environmental context, and an action may be used to affect the behavior of a computing device as a result of identifying the user input in a scenario that has the environmental context. Such associations may be dynamically determined as a result of user interactions associated with manually provided input, for example to create, update, and/or remove semantic actions associated with a variety of user inputs.
What exactly they claim (claim 1)
1. A system comprising: at least one processor; and memory storing instructions that, when executed by the at least one processor, causes the system to perform a set of operations, the set of operations comprising: obtaining a first instance of interaction data associated with a user of a computing device; processing the first instance of interaction data to generate a first set of user signals; determining, based on a first environmental context, a first semantic action associated with the first set of user signals, wherein the first environmental context includes a computing device orientation of the computing device relative to a first computing device; performing the determined first semantic action, thereby adapting the computing device responsive to the first instance of interaction data based on the first environmental context; obtaining a second instance of interaction data associated with the user; processing the second instance of interaction data to generate a second set of user signals, wherein the second set of user signals is similar to the first set of user signals; determining, based on a second environmental context, a second semantic action, wherein the second semantic action is of a different semantic type than the first semantic action, the different semantic type corresponding to a different computing-device function or application-level operation, and the second environmental context includes a computing device orientation of the computing device relative to a second computing device different than the first computing device; and performing the determined second semantic action, thereby adapting the computing device differently than the first semantic action based on the similar second set of user signals and the different relative computing device orientation of the second environmental context.
Facts
- Company
- Microsoft (Microsoft Technology Licensing, LLC)
- Official title
- Semantic user input
- Patent number
- US12730522B2
- Application number
- 17677623
- Filed
- Feb 22, 2022
- Granted
- Sep 8, 2026
- Inventors
- 5
- Claims
- 20
Classified under
G06F 3/0346 Physics & computingG06N 5/022 Physics & computingG06F 3/013 Physics & computingMore from Microsoft in AI & software
- Microsoft patents method to check AI answers against verified facts Oct 20, 2026
- Microsoft patents AI that presents PowerPoint slides for you Sep 3, 2026
- Microsoft patents method to check if LLM answers are based on real data Sep 10, 2026
- Microsoft patents system to blend multiple AI agents into one chat Oct 1, 2026
- Microsoft patents system that answers questions using your screen history Oct 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.