Google patents changing AI images by waving hand over screen
This filing describes a way to edit images created by AI models using physical hand movements instead of typing commands or clicking buttons. Users can wave their hand over a specific part of the generated image to request changes, such as making it simpler or adding details, without needing a separate control panel.
In the filing’s own words
Implementations set forth herein relate to modifying a generative output of an application according to an input gesture that is performed without necessarily interacting with a GUI element that is rendered separate from a generative output (e.g., a GUI element separate from a natural language output generated using an LLM or an image generated using an image diffusion model). Various different input gestures can be performed by a user to refine a generative output to be simpler, more complex, to include an image, to modify a generated image, and/or otherwise modify the generative output. In some implementations, an input gesture can be processed as one or more predetermined gestures, and/or an input gesture can be interpreted per case using an available model for interpreting such gestures. In this way, models for interpreting gestures and/or refining generative output can be enhanced through further training of such models.
What exactly they claim (claim 1)
1. A method implemented by one or more processors, the method comprising: processing a natural language input generated based on user interface input of a user at a client application, wherein the client application employs one or more generative models to provide a generative output based on the natural language input; providing, based on the natural language input, the generative output to the client application in furtherance of the client application rendering an image file at a display interface of a client computing device; determining, subsequent to the natural language input being received at the client application, that an input interface of the client computing device, or another computing device, has received an input gesture, wherein the user performs the input gesture by motioning over a particular portion of the display interface that is providing a rendering of the image file; generating additional generative output based on the input gesture, wherein the additional generative output indicates a modification to be made to the rendering of the image file; and providing the additional generative output to the client application in furtherance of the client application rendering a modified image at the display interface, wherein the client application receives the additional generative output and causes at least a portion of the rendering of the image file to be replaced or modified by the modified image.
Facts
- Company
- Google (GOOGLE LLC)
- Official title
- FACILITATING MODEL OUTPUT MODIFICATIONS VIA PHYSICAL GESTURE DIRECTED TO PORTION OF GENERATIVE OUTPUT
- Publication number
- US20260259662A1
- Application number
- 19652652
- Filed
- Apr 20, 2026
- Published
- Sep 3, 2026
- Inventors
- 4
- Claims
- 18
Classified under
G06F 3/04883 Physics & computingG06F 3/14 Physics & computingG06T 11/23 Physics & computingG06T 11/60 Physics & computingG06V 40/174 Physics & computingG06V 40/28 Physics & computingMore from Google in AI & software
- Google patents video search that answers questions using AI descriptions Sep 15, 2026
- Google patents photo tool that removes people and their gear Sep 15, 2026
- Google patents AI that writes live meeting notes and summaries Sep 15, 2026
- Google patents AI tool that turns messy emergency notes into clear messages Sep 3, 2026
- Google patents AI that clicks through apps to finish tasks for you Sep 24, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.