Microsoft patents method to find location by matching 2D images to 3D maps
This patent describes a way to figure out where a device is by comparing a 2D photo or 3D scan to a pre-made map. It works by pulling out key features from both the image and the map, then checking how well they match to calculate the device's exact position and angle.
In the filing’s own words
Determining a location of an entity comprises: receiving a query comprising a 2D image depicting an environment of the entity; searching for a match between the query and a 3D map of the environment. The 3D map comprising a 3D point cloud, the match indicating the location of the entity in the environment. Searching for the match comprises: extracting descriptors from the 2D image referred to as image descriptors; extracting descriptors from the 3D point cloud referred to as point cloud descriptors; correlating the image descriptors with the point cloud descriptors to produce correspondences, wherein a correspondence is an image descriptor corresponding to a point cloud descriptor; estimating, using the correspondences, the location of the entity.
What exactly they claim (claim 1)
1. A method of determining a pose of an entity, the pose comprising a 3D position and orientation of the entity, comprising: receiving a query comprising a three-dimensional (3D) point cloud depicting an environment of the entity, the 3D point cloud omitting visual imagery of the environment of the entity; searching for a match between the query and a 3D map of the environment, the 3D map comprising a posed two-dimensional (2D) image, the posed 2D image comprising a computed 3D position and a computed 3D orientation of a capture device used to capture the posed 2D image, searching for the match comprising: extracting a point cloud descriptor from the 3D point cloud; extracting an image descriptor from the posed 2D image; and using the posed 2D image, directly localizing the entity with respect to the 3D map, directly localizing comprising: correlating the image descriptor with the point cloud descriptor, and based on the correlation, generating a numerical similarity value above a threshold representing a likelihood that the image descriptor and the point cloud descriptor depict a same element of the environment; and estimating, using the numerical similarity value, the pose of the entity with respect to the 3D map.
Facts
- Company
- Microsoft (Microsoft Technology Licensing, LLC)
- Official title
- MATCHING BETWEEN 2D AND 3D FOR DIRECT LOCALIZATION
- Publication number
- US20260278835A1
- Application number
- 19678136
- Filed
- May 14, 2026
- Published
- Sep 17, 2026
- Inventors
- 4
- Claims
- 20
Classified under
G06T 7/74 Physics & computingG06F 16/535 Physics & computingG06V 20/647 Physics & computingG06T 2207/10024 Physics & computingG06T 2207/10028 Physics & computingG06T 2207/20081 Physics & computingMore from Microsoft in AI & software
- Microsoft patents method to check AI answers against verified facts Oct 20, 2026
- Microsoft patents AI that presents PowerPoint slides for you Sep 3, 2026
- Microsoft patents method to check if LLM answers are based on real data Sep 10, 2026
- Microsoft patents system to blend multiple AI agents into one chat Oct 1, 2026
- Microsoft patents system that answers questions using your screen history Oct 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.