/Patent Tracker
AmazonGranted patentAI & softwareInterest 6/10

Amazon patents video AI that merges image chunks to classify long videos

This patent describes a method for analyzing long videos by breaking them into small image chunks called tokens. A transformer model selects key chunks and merges them with less important ones to reduce data size before classifying the video content.

Why it mattersThis technique could help streaming services process and categorize hours of video content more efficiently by focusing computational power on the most relevant parts of each frame.

In the filing’s own words

Tokens or patches are divided from individual images of long-form video files and merged by transformer models, for the purposes of classifying such video files. Tokens or patches may have any dimensions, and may be partitioned from images of a video file into target tokens and source tokens in any manner, e.g., uniformly, based on their respective locations within the individual images, based on any motion of the respective tokens, or by a learned model that partitions the tokens based on scores representative of their saliency using one or more blocks of a transformer model. Once the tokens are partitioned, source tokens are matched with target tokens, and merged tokens identified from the matched tokens are processed to classify the video file.

What exactly they claim (claim 1)

1. A computer system comprising at least one processor and at least one data store, wherein the computer system is programmed with one or more sets of instructions that, when executed by the at least one processor, cause the computer system to execute a method comprising: receiving a video file over one or more networks; extracting a plurality of image frames from the video file; dividing the plurality of image frames into a plurality of tokens, wherein each one of the plurality of tokens has common dimensions, and wherein the plurality of tokens do not overlap; processing the plurality of image frames by a transformer model, wherein processing the plurality of image frames comprises: selecting a set of target tokens from the plurality of tokens by at least a first transformer block of the transformer model, wherein the set of target tokens is selected according to at least one partition factor; matching each one of a set of source tokens with one of the set of target tokens, wherein the set of source tokens includes each one of the plurality of tokens not included in the set of target tokens; defining a plurality of groups of tokens, wherein each one of the plurality of groups comprises one of the set of target tokens and one of the set of source tokens matched to the one of the set of target tokens; defining a plurality of merged tokens, wherein each one of the plurality of merged tokens is defined for one of the plurality of groups of tokens; and classifying the video file based at least in part on the plurality of merged tokens.

Facts

Company
Amazon (Amazon Technologies, Inc.)
Official title
Merging tokens for long-form video understanding
Patent number
US12749310B1
Application number
18818223
Filed
Aug 28, 2024
Granted
Sep 29, 2026
Inventors
5
Claims
20

Classified under

G06V 20/41 Physics & computingG06V 10/26 Physics & computingG06V 10/761 Physics & computingG06V 10/82 Physics & computingG06V 20/46 Physics & computing
Read the full filing (official PDF) ↗ Open in the Patent Tracker

More from Amazon in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.