Microsoft patents desktop mode that highlights usable text and images
Microsoft describes a system that scans what is currently on your screen to find text or images that can be used by other tools. When activated, the system places small visual markers next to these items so you can click them to start actions like copying or editing without changing the main program.
In the filing’s own words
Disclosed is an operating system (OS) discovery mode that identifies and provides access to OS and/or third-party provided features within an applicable region of a desktop. In some configurations, once the discovery mode is activated, the content displayed by the applicable region is analyzed to identify content usable by OS and/or third-party provided features. Visual cues are rendered in the applicable region near the identified content, highlighting the availability of the OS and/or third-party provided features. Users may interact with the visual cues to manipulate the underlying content or to invoke the OS and/or third-party provided features. OS and/or third-party provided features may modify content displayed by an application, launch an inline micro-experience, crop or export images, etc. While in the discovery mode, visual cues are highlighted as a discovery mouse cursor moves around the desktop. In some configurations the discovery mode is triggered automatically, causing an OS service to automatically identify and display a set of entity visual cues across the applicable region.
What exactly they claim (claim 1)
1. A method comprising: retrieving a contents of a display buffer of an active window of an application; identifying a text entity within the retrieved contents, wherein the text entity is located within a parent entity of the retrieved contents; determining that the text entity is usable by an OS or third-party provided feature; rendering an overlay layer over the active window of the application, wherein the overlay layer includes an entity visual cue that appears next to, underneath, around, over, or partially over the text entity, wherein the overlay layer is rendered without participation of the application, and wherein the entity visual cue is rendered in response to a system cursor being over a parent entity visual cue; receiving, at the overlay layer, an indication of a selection of the entity visual cue; and invoking the OS or third-party provided feature.
Facts
- Company
- Microsoft (MICROSOFT TECHNOLOGY LICENSING, LLC)
- Official title
- Feature discovery layer
- Patent number
- US12754649B2
- Application number
- 18216512
- Filed
- Jun 29, 2023
- Granted
- Oct 6, 2026
- Inventors
- 17
- Claims
- 17
Classified under
G06F 3/04812 Physics & computingG06F 3/0482 Physics & computingG06F 9/451 Physics & computingG06V 30/00 Physics & computingG06F 40/166 Physics & computingMore from Microsoft in AI & software
- Microsoft patents method to check AI answers against verified facts Oct 20, 2026
- Microsoft patents AI that presents PowerPoint slides for you Sep 3, 2026
- Microsoft patents method to check if LLM answers are based on real data Sep 10, 2026
- Microsoft patents system to blend multiple AI agents into one chat Oct 1, 2026
- Microsoft patents system that answers questions using your screen history Oct 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.