Google patents AI tool to cut live streams into short clips

1. A computer-implemented method comprising: obtaining a live stream of video content that comprises a creator content stream and one or more viewer content streams, wherein the creator content stream comprises visual data and audio data provided by a content creator, and wherein each viewer content stream comprises action data that characterizes actions performed by a content viewer while observing the visual data and audio data provided by the content creator; for each time interval in a plurality of successive time intervals during the live stream of the video content: generating a candidate short video using a multi-modal neural network system to process a multi-modal input that comprises (i) a portion of the creator content stream that corresponds to the time interval and (ii) for each viewer content stream, a portion of the action data that characterizes actions performed by the content viewer within the time interval, wherein the candidate short video comprises a subset of video frames selected from video frames included in the creator content stream within the time interval and a subset of audio frames selected from audio frames included in the creator content stream within the time interval; selecting, from the candidate short videos generated for the plurality of successive time intervals during the live stream of the video content, one or more short videos; and providing the one or more selected short videos for presentation to the content creator.