Amazon patents tool to animate specific objects in images into video
Amazon has been granted a patent for a method that turns a still image into a short video by animating a specific object selected by the user. The system uses a pre-trained AI model to generate video frames that move the object to a target location defined by a bounding box, while keeping the rest of the image consistent.
In the filing’s own words
Techniques are generally described for controllable image-to-video generation. In various examples, a first image representing at least a first object may be received. First input data including a selection of the first object in the first image for animation may be received. Second input data including at least a first bounding box indicating a target location of the first object may be received. A latent diffusion text-to-image model and the first image may be used to generate a first plurality of visual tokens. One or more first grounding tokens may be generated representing a location of the first bounding box. The latent diffusion text-to-image model may be used to generate a video animating the first object based on the first plurality of visual tokens and the one or more first grounding tokens.
What exactly they claim (claim 1)
1. A computer-implemented method, comprising: receiving a first image representing at least a first object; receiving first input data comprising a selection of the first object in the first image, the selection of the first object indicating that the first object is to be animated along a motion path; receiving second input data comprising at least a first bounding box indicating a target location of the first object; inputting the first image into a pre-trained latent diffusion text-to-image model; generating, using the pre-trained latent diffusion text-to-image model and the first image, a first plurality of visual tokens; generating one or more first grounding tokens representing a location of the first bounding box; generating, using the first plurality of visual tokens and the one or more first grounding tokens, a spatio-temporal mask comprising a region corresponding to a first location of the first object and a region corresponding to the target location; generating first intermediate visual tokens by combining the one or more first grounding tokens and the first plurality of visual tokens using a gated self-attention layer; generating second intermediate visual tokens by combining the first intermediate visual tokens and text prompt tokens using a cross-attention layer; generating output visual tokens by inputting the second intermediate visual tokens into at least one temporal attention layer; and generating a frame of video by denoising the output visual tokens using a denoising process of the pre-trained latent diffusion text-to-image model, wherein the frame of the video comprises a representation of the first object at the target location, wherein the pre-trained latent diffusion text-to-image model comprises a zero-shot image-to-video model configured to animate the first image using user-provided motion layouts and the spatio-temporal mask.
Facts
- Company
- Amazon (Amazon Technologies, Inc.)
- Official title
- Controllable image-to-video generation
- Patent number
- US12784315B2
- Application number
- 18604917
- Filed
- Mar 14, 2024
- Granted
- Nov 3, 2026
- Inventors
- 7
- Claims
- 17
Classified under
G06T 13/00 Physics & computingG06F 40/40 Physics & computingG06T 5/60 Physics & computingG06T 5/70 Physics & computingG06T 5/77 Physics & computingG06T 7/73 Physics & computingMore from Amazon in AI & software
- Amazon patents system that cites sources for AI answers Sep 29, 2026
- Amazon patents voice assistants that adjust personality based on your mood Nov 3, 2026
- Amazon patents tech to swap words in movies using actor's voice Sep 8, 2026
- Amazon patents ML tool to add graphics to live sports from moving cameras Sep 15, 2026
- Amazon patents shopping assistant that remembers your search filters Oct 6, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.
