MicrosoftPatent applicationRobots & automationInterest 8/10

Microsoft patents robot hands that learn to grip new objects from images

Illustration of the invention
Illustration based on the patent drawing. Not a photo of a real product.
Patent drawing
Patent drawing from the official filing.

Microsoft describes a system that teaches robotic grippers how to handle objects by analyzing how they touch surfaces. The technology uses a library of movement patterns to adapt a known grip to a new object based on a photo, adjusting for the object's specific shape and size.

Why it mattersThis could help robots pick up unfamiliar items in warehouses or homes without needing a separate program for every single object.

In the filing’s own words

Techniques for generating a latent codebook and for generating instructions indicating how an action is performed against an object using the latent codebook are disclosed. The latent codebook is created by generating interaction trajectories that occur during an action performed by a gripping structure against an object. The vectors in the latent codebook describe actions by mapping trajectories of contact points between gripping structures and objects during performance of actions. A service then accesses an image of a second object that is to be manipulated. The service obtains an instruction indicating that the specific action is to be performed against the second object. In response to accessing a vector in the latent codebook, the service uses the image to localize the vector to the second object. The service then generates a sequence of instructions that represent the action as applied to the second object.

What exactly they claim (claim 1)

1. A method for generating instructions indicating how an action is performed against an object that was not previously mapped, said method comprising: accessing a latent codebook comprising a vector, wherein the vector describes a specific action that is performed by a first gripping structure against a first object, wherein the vector describes the specific action by mapping a trajectory of contact points between the first gripping structure and the first object during performance of the specific action; accessing an image of a second object that is to be manipulated, wherein the second object is different from the first object; obtaining an instruction indicating that the specific action is to be performed against the second object; in response to accessing the vector in the latent codebook, using the image to localize the vector to the second object, wherein localizing the vector to the second object includes modifying a first dimensionality associated with the first object, as described in the vector, to account for a second dimensionality of the second object, as identified by the image, such that a new trajectory of new contact points between a second gripping structure and the second object are generated; and generating a sequence of instructions that represent the specific action as applied to the second object, wherein the sequence of instructions identifies the new trajectory of the new contact points between the second gripping structure and the second object, such that the sequence of instructions facilitates performance of the specific action against the second object.

Facts

Company
Microsoft (Microsoft Technology Licensing, LLC)
Official title
GENERATIVE 3D MODEL OF HAND OBJECT INTERACTIONS
Publication number
US20260355959A1
Application number
19200145
Filed
May 6, 2025
Published
Nov 12, 2026
Inventors
4
Claims
20

Classified under

B25J 9/1671 Operations & transportB25J 9/163 Operations & transportG05B 13/0265 Physics & computing

More from Microsoft in Robots & automation

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.