Nvidia patents system that guides sign language users to fix hand positions
Nvidia describes a system that uses machine learning to analyze video of a person signing. It compares the user's hand and body movements to a standard dictionary of signs and provides real-time feedback on how to adjust their pose to match the correct symbol.
In the filing’s own words
In various examples, machine learning model-based sign language corrective feedback is provided. Machine learning model-based recognition of sign language symbols (e.g., body poses and/or movements) may be used to generate real-time kinematic feedback to a sign language speaker that guides them to adjust their hand pose to correctly align with established sign language standard symbols. A sign language feedback framework may process video data representing conversational sign language symbol sequences to extract kinematic keypoints to search a sign language dictionary representing an established vocabulary of sign language symbols. Based on selecting an intended sign language symbol from the dictionary and computing deviations between the kinematic keypoint pattern of the intended sign language symbol from the dictionary and the extracted kinematic keypoint symbol, the sign language feedback framework may provide real-time kinematic feedback to the signer describing how to adjust their signing.
What exactly they claim (claim 1)
1. One or more processors comprising processing circuitry to: receive video data comprising a representation of sign language communication; extract one or more sign language body pose kinematic keypoint symbols from the video data; select a standardized kinematic keypoint pattern corresponding to a sign language symbol based at least on a similarity to the extracted one or more sign language body pose kinematic keypoint symbols; compute one or more kinematic keypoint location deviations between a first set of kinematic keypoints of the standardized kinematic keypoint pattern and a second set of kinematic keypoints of the extracted one or more sign language body pose kinematic keypoint symbols; and based at least on the one or more kinematic keypoint location deviations, cause a user interface to present kinematic feedback data that indicates one or more adjustments to align the second set of kinematic keypoints with the first set of kinematic keypoints within an established tolerance.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- Machine learning model-based sign language corrective feedback
- Patent number
- US12762973B2
- Application number
- 19044244
- Filed
- Feb 3, 2025
- Granted
- Oct 13, 2026
- Inventors
- 4
- Claims
- 20
Classified under
G09B 21/009 Physics & computingG06F 16/53 Physics & computingG06V 10/772 Physics & computingG06V 20/41 Physics & computingG06V 20/46 Physics & computingG06V 40/28 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.