Microsoft patents video trick that hides foreground masks in color data
Microsoft has been granted a patent for a method that embeds information about which parts of a video frame are the 'foreground' directly into the color values of the pixels. The system uses a specific mathematical mapping to shift or compress these color values, allowing a decoder to extract a mask that identifies the main subject while restoring the original colors.
In the filing’s own words
Techniques are described for decoding frame masks in a video stream. In some solutions, a computer implemented method receives a frame of video data comprising an array of pixels with encoded color values defined in a YUV color space. The encoded values, converted from original color values using a mapping function, are bounded within a second data range, shifted or compressed from a first data range of the original color values. The method decodes the encoded color values to obtain decoded color values for the array of pixels and extracts a foreground mask based on the decoded color values. The foreground mask identifies a subset of the array of pixels to be in a foreground of the frame. The method then converts the decoded color values of the subset of the array of pixels to new color values by applying an inverse mapping function.
What exactly they claim (claim 1)
1. A computer-implemented method comprising: receiving a frame of video data comprising an array of pixels having respective encoded color values defined in a YUV color space, the encoded color values having been converted from original color values by applying a mapping function, wherein the original color values are bounded within a first data range, and wherein the encoded color values are bounded within a second data range, the second data range having been shifted or compressed from the first data range using the mapping function; decoding the encoded color values to obtain decoded color values corresponding to the array of pixels; extracting a foreground mask based on the decoded color values, wherein the foreground mask identifies a subset of the array of pixels to be in a foreground of the frame; and converting the decoded color values of the subset of the array of pixels identified by the foreground mask to new color values by applying an inverse mapping function to the decoded color values of the subset of the array of pixels identified by the foreground mask, wherein the inverse mapping function reverses the mapping function so that the new color values are bounded within the first color range.
Facts
- Company
- Microsoft (Microsoft Technology Licensing, LLC)
- Official title
- Embedding frame masks in a video stream
- Patent number
- US12732629B2
- Application number
- 18828220
- Filed
- Sep 9, 2024
- Granted
- Sep 8, 2026
- Inventors
- 3
- Claims
- 20
Classified under
H04N 19/46 Electricity & electronicsH04N 19/172 Electricity & electronicsH04N 19/186 Electricity & electronicsH04N 9/75 Electricity & electronicsMore from Microsoft in AI & software
- Microsoft patents method to check AI answers against verified facts Oct 20, 2026
- Microsoft patents AI that presents PowerPoint slides for you Sep 3, 2026
- Microsoft patents method to check if LLM answers are based on real data Sep 10, 2026
- Microsoft patents system to blend multiple AI agents into one chat Oct 1, 2026
- Microsoft patents system that answers questions using your screen history Oct 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.