Google patents visual search that suggests apps based on what you see

1. A computer-implemented method, the computer-implemented method comprising: obtaining, by a computing system comprising one or more processors, a user input on a user computing device; obtaining, with an overlay application invoked based on the user input, display data descriptive of content currently presented for display on the user computing device within a first application; obtaining, by the computing system and with the overlay application, a gesture input from the user computing device; determining, by the computing system and with the overlay application, a region-of-interest within the display data based on the gesture input and image features of the content currently presented for display; processing, by the computing system and with the overlay application, the display data with an on-device segmentation model to generate a segmentation mask based on a detected object within the region-of-interest within the display data and to segment a portion of the display data based on the segmentation mask; transmitting, by the computing system, the portion of the display data to a server computing system and in response to transmitting the portion of the display data to the server computing system, obtaining, from the server computing system, visual search data comprising one or more visual search results determined based on detected features within the portion of the display data; determining, by the computing system and with a machine-learned suggestion model and based on the visual search data, a suggested action comprising a create-a-text suggestion and determining, with the machine-learned suggestion model, a second application is associated with the suggested action; providing, by the computing system and based on the visual search data and within a suggestion panel, an application suggestion comprising an indication of the second application, wherein the second application comprises a messaging application; in response to receiving a selection of the application suggestion, processing, by the computing system, the visual search data with a generative model to generate a model-generated output; and transmitting, by the computing system and via the overlay application, the model-generated output to the messaging application.