Microsoft patents 3D video calls that highlight body parts doctors mention
Microsoft describes a system for 3D teleconferences that creates a digital model of a person's body in real-time. When a speaker, such as a doctor, mentions a specific body part like a shoulder, the system automatically highlights that area on the digital model to show exactly what is being discussed.
In the filing’s own words
Disclosed are techniques for highlighting regions of a subject's body in real-time during a 3D teleconference. In some configurations, images of the subject are captured in real-time and used to generate an input mesh representation of the subject. The input mesh may be displayed to the subject during the teleconference. As a medical practitioner speaks, references to region's of the subject's body are identified. The identified regions may then be highlighted in the display of the input mesh, making it clear what the medical practitioner is referring to. For example, if the practitioner says “I need to further examine the left foot and the right shoulder”, the display of the input mesh may highlight the left foot and the right shoulder of the subject.
What exactly they claim (claim 1)
1. A method comprising: receiving an input mesh of a subject; identifying a pose of the subject from the input mesh; generating a parameterized model of the subject from the pose; receiving a model-generated mesh of the subject from the parameterized model; generating an input-model vertex mapping from vertices of the model-generated mesh to corresponding vertices of the input mesh; receiving a region name of a region of the subject; identifying a plurality of model-generated vertices of the model-generated mesh corresponding to the region name; identifying, from the input-model vertex mapping, a plurality of input mesh vertices that correspond to the plurality of model-generated vertices; and highlighting the region of the subject by modifying how the plurality of input mesh vertices appear in a rendering of the input mesh of the subject.
Facts
- Company
- Microsoft (MICROSOFT TECHNOLOGY LICENSING, LLC)
- Official title
- AUGMENTED UNDERSTANDING THROUGH ANNOTATION OF 3D TELECONFERENCE INTERACTIONS
- Publication number
- US20260301936A1
- Application number
- 19093225
- Filed
- Mar 27, 2025
- Published
- Oct 1, 2026
- Inventors
- 4
- Claims
- 20
Classified under
G16H 40/67 Physics & computingG06F 40/20 Physics & computingG06T 17/20 Physics & computingG06V 10/82 Physics & computingG06V 40/174 Physics & computingG16H 80/00 Physics & computingMore from Microsoft in AR, VR & displays
- Microsoft patents AR glasses that slide to fit different eye spacing Oct 13, 2026
- Microsoft patents AR glasses that hide your eyes from others Sep 3, 2026
- Microsoft patents thinner lens coating for mixed reality glasses Sep 1, 2026
- Microsoft patents matching 3D spaces to share holograms between remote users Sep 1, 2026
- Microsoft patents AR display that adjusts laser brightness to stay consistent Sep 8, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.