Google patents search that mixes photos and text to refine results
Google has filed a patent for a search system that lets users start with a photo and then add words to narrow down the results. The interface shows a preview of the original image alongside a text box, combining both inputs to create a more specific search query.
In the filing’s own words
Systems and methods of the present disclosure are directed to a computer-implemented method for multimodal search refinement. The method includes obtaining a visual search query from a user comprising one or more query images. The method includes providing a search interface for display to the user, the search interface comprising one or more result images responsive to the one or more query images and an interface element indicative of a request to the user to refine the visual search query. The method includes obtaining, from the user, textual data comprising a refinement to the visual search query. The method includes appending, by the computing system, the textual data to the visual search query to obtain a multimodal search query.
What exactly they claim (claim 1)
1. A computer-implemented method for multimodal search refinement, the method comprising: obtaining, by a computing system comprising one or more computing devices, a visual search query comprising one or more query images; retrieving, by the computing system, based on using a visual search operation on the one or more query images, one or more result images; providing, by the computing system, a search interface for display, the search interface comprising the one or more result images and an interface element indicative of a request to refine the visual search query, wherein the interface element comprises a textual input field and a preview element depicting a preview of the one or more query images; obtaining, by the computing system, via the textual input field, textual data comprising a refinement to the visual search query; appending, by the computing system, the textual data to the visual search query to obtain a multimodal search query; retrieving, by the computing system, based on the multimodal search query, one or more refined search results; and providing, by the computing system, a refined search interface for display, wherein the refined search interface comprises a refined search result interface element that comprises the one or more refined search results.
Facts
- Company
- Google (Google LLC)
- Official title
- Systems and Methods for Efficient Multimodal Search Refinement
- Publication number
- US20260300380A1
- Application number
- 19710180
- Filed
- Jun 16, 2026
- Published
- Oct 1, 2026
- Inventors
- 3
- Claims
- 20
Classified under
G06F 16/532 Physics & computingG06F 9/453 Physics & computingG06F 16/538 Physics & computingMore from Google in AI & software
- Google patents video search that answers questions using AI descriptions Sep 15, 2026
- Google patents photo tool that removes people and their gear Sep 15, 2026
- Google patents AI that writes live meeting notes and summaries Sep 15, 2026
- Google patents AI tool that turns messy emergency notes into clear messages Sep 3, 2026
- Google patents AI that clicks through apps to finish tasks for you Sep 24, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.