Sony patents game controls that learn your unique hand movements
Sony has been granted a patent for a system that helps video game controllers adapt to individual players. Instead of requiring perfect repetition, the software uses machine learning to analyze a few attempts at a gesture and learns the specific way that player moves their hands. The game then recognizes that personal pattern to trigger actions, making controls easier to use.
In the filing’s own words
A system that teaches players gestures, for instance during the introduction of the game, and asks the player to invoke the gesture. Rather than asking the player to repeat over and over until the player succeeds, the game looks for commonality in the player's attempts, and after a small number of attempts, the game can learn how that player interprets the gesture given the player's own ability. The game can then adapt itself to look for that pattern to trigger the action.
What exactly they claim (claim 1)
1. A computer-implemented method, comprising: for a particular computer simulation command, obtaining sensor data associated with a user performing a gesture multiple times; for each performance of the gesture by the user, generating a training data set for training at least one machine learning (ML) model that correlates input sensor data to output computer simulation commands, the training data set comprising (i) a portion of the sensor data that corresponds to the performance of the gesture by the user, and (ii) a label identifying the particular computer simulation command as being correlated to the sensor data; training the at least one ML model using the training data sets; during execution of the computer simulation, obtaining additional sensor data associated with the user performing the gesture; providing the additional sensor data to the ML model; receiving from the ML model an indication that the particular computer simulation command is correlated to the additional sensor data; and in response to receiving the indication from the ML model, executing the computer simulation according to the particular computer simulation command.
Facts
- Company
- Sony (Sony Interactive Entertainment Inc.)
- Official title
- Gesture training for skill adaptation and accessibility
- Patent number
- US12761737B2
- Application number
- 18648265
- Filed
- Apr 26, 2024
- Granted
- Oct 13, 2026
- Inventors
- 5
- Claims
- 20
Classified under
G06F 3/017 Physics & computingA63F 13/211 Human necessitiesA63F 13/212 Human necessitiesA63F 13/533 Human necessitiesG06N 20/00 Physics & computingMore from Sony in Games & media
- Sony patents game engine that reshuffles scenes based on your heart rate Sep 8, 2026
- Sony patents eSport events where fans' cheers give players in-game power-ups Oct 20, 2026
- Sony patents game gear design using AI mood boards Oct 20, 2026
- Sony patents game feature that warns players before they hit a hard part Sep 3, 2026
- Sony patents AI system to match players based on how they actually play Sep 10, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.