Google patents photo tool that removes people and their gear
Google has patented a method for automatically removing bystanders from photos, including any objects they are holding or wearing. The system identifies the person and attached items, creates a digital mask, and fills the space with background pixels.
In the filing’s own words
A media application detects a bystander in an initial image. The media application generates a bystander box that includes the bystander, wherein all pixels for the bystander are within the bystander box. The media application generates localizer boxes that encompass the bystander and one or more objects that are attached to the bystander. The media application aggregates the bystander box and one or more of the localizer boxes to form an aggregated box. The media application applies a segmenter to the initial image, based on the aggregated box, to segment the bystander and the one or more objects from the initial image to generate a bystander mask, wherein the bystander mask includes a subset of pixels within the aggregated box. The media application generates an inpainted image that replaces all pixels within the bystander mask with pixels that match a background in the initial image.
What exactly they claim (claim 1)
1. A computer-implemented method comprising: detecting a bystander in an initial image; generating a bystander box that includes the bystander, wherein all pixels for the bystander are within the bystander box; generating one or more localizer boxes that encompass the bystander and one or more objects that are attached to the bystander; aggregating the bystander box and the one or more localizer boxes to form an aggregated box by: defining the aggregated box as being the bystander box and, for each particular localizer box of the one or more localizer boxes, computing an intersection over union (IoU) ratio of the aggregated box to the particular localizer box, and responsive to the IoU ratio exceeding a threshold value or responsive to an area of overlap between the aggregated box and the particular localizer box exceeding an occupancy threshold, increasing a size of the aggregated box; applying a segmenter to the initial image, based on the aggregated box, to segment the bystander and the one or more objects from the initial image to generate a bystander mask, wherein the bystander mask includes a subset of pixels within the aggregated box; and generating an inpainted image, wherein all pixels within the bystander mask are replaced in the inpainted image with pixels that match a background in the initial image.
Facts
- Company
- Google (Google LLC)
- Official title
- Bystander and attached object removal
- Patent number
- US12737858B2
- Application number
- 18293678
- Filed
- Jun 30, 2022
- Granted
- Sep 15, 2026
- Inventors
- 6
- Claims
- 20
Classified under
G06T 5/77 Physics & computingG06T 5/60 Physics & computingG06T 7/11 Physics & computingG06T 7/136 Physics & computingG06T 7/194 Physics & computingG06T 2207/20021 Physics & computingMore from Google in AI & software
- Google patents video search that answers questions using AI descriptions Sep 15, 2026
- Google patents AI that writes live meeting notes and summaries Sep 15, 2026
- Google patents AI tool that turns messy emergency notes into clear messages Sep 3, 2026
- Google patents AI that clicks through apps to finish tasks for you Sep 24, 2026
- Google patents editing photos by typing what you want to change Sep 24, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.