MicrosoftGranted patentCameras & imagingInterest 4/10

Microsoft patents method to fill gaps in 3D depth maps from sparse data

This patent describes a technique for improving 3D depth maps that have missing data points. The system takes a sparse depth map, converts it into a 3D point cloud, and uses mathematical interpolation to estimate the depth of specific illuminator dots. This allows the system to output a more complete depth value for each dot in the pattern.

Why it mattersThis improves the accuracy of 3D sensing systems by filling in missing information, which is useful for applications like object recognition or spatial mapping.

In the filing’s own words

A method for operating a sparse depth imaging system is presented. The method comprises receiving a depth map of an environment. The depth map comprises a plurality of pixels having locations in an optical sensor coordinate system. A pattern of illuminator dots in an optical source coordinate system is received. Each illuminator dot has a fixed location in a defined plane in the optical source coordinate system. The depth map is projected into a 3D point cloud in the optical sensor coordinate system. Each point in the 3D point cloud is assigned a 2D location in the defined plane. A depth value for each illuminator dot is interpolated based on transformed depth of points in the 3D point cloud. Each illuminator dot is assigned a 3D location in the optical sensor coordinate system. A depth for each illuminator dot is output in the optical sensor coordinate system.

What exactly they claim (claim 1)

1. A method for operating a sparse depth imaging system, comprising: receiving a depth map of an environment, the depth map comprising a plurality of pixels having locations in an optical sensor coordinate system, and the depth map also comprising illumination dots corresponding to illuminator dots of a structured light optical source; receiving a pattern of the illuminator dots in an optical source coordinate system, each illuminator dot having a fixed location in a defined plane in the optical source coordinate system; projecting the depth map of the environment into a 3D point cloud in the optical sensor coordinate system; assigning each point in the 3D point cloud a 2D location in the defined plane; interpolating a depth value for each illuminator dot in the pattern of illuminator dots based on transformed depth of points in the 3D point cloud; assigning each illuminator dot in the pattern of illuminator dots a 3D location in the optical sensor coordinate system; and outputting a depth for each illuminator dot in the pattern of illuminator dots in the optical sensor coordinate system.

Facts

Company
Microsoft (Microsoft Technology Licensing, LLC)
Official title
Sparse depth imaging with interpolated depth values
Patent number
US12793245B2
Application number
18905039
Filed
Oct 2, 2024
Granted
Nov 10, 2026
Inventors
3
Claims
18

Classified under

G06T 7/521 Physics & computingG01B 11/22 Physics & computingG01S 7/4814 Physics & computingG01S 7/4816 Physics & computingG01S 17/89 Physics & computingG06T 3/04 Physics & computing

More from Microsoft in Cameras & imaging

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.