Microsoft patents video tracking that skips frames to keep motion smooth

1. A data processing system comprising: a processor; and a memory storing executable instructions that, when executed, cause the processor alone or in combination with other processors to perform operations of: obtaining a frame of video content at an object detection pipeline, the video content comprising a plurality of frames; analyzing the frame of video content using an object detection model to detect a plurality of objects in the frame of video content, the object detection model associating each object of the plurality of objects with a confidence score; determining whether to perform object detection on the frame or detection propagation based on whether performing the object detection on the frame would cause a frame rate of the object detection pipeline to fall below a threshold; responsive to determining that the frame rate would fall below the threshold if the object detection pipeline were to perform object detection on the frame: performing detection propagation to extrapolate object tracks for a plurality of objects from previously determined object tracks; and outputting the object tracks for the plurality of objects for the frame; and responsive to determining that the frame rate would not fall below the threshold, performing the object detection on the frame by: performing a primary matching operation on high confidence detection objects to determine first object tracks of the high confidence detection objects by associating the high confidence detection objects with an object track of a plurality of object tracks, the high confidence detection objects being objects from the plurality of objects associated with a confidence score that satisfies a confidence threshold, the first object tracks tracking the high confidence detection objects across the plurality of frames; performing a secondary matching operation on low confidence detection objects to determine second object tracks of the low confidence detection objects by associating the low confidence detection objects with an object track of the plurality of object tracks, the low confidence detection objects being objects from the plurality of objects associated with a confidence score that does not satisfy the confidence threshold, the second object tracks tracking the low confidence detection objects across the plurality of frames; and outputting, from the object detection pipeline, the first object tracks and the second object tracks.