Sony patents game assistant that suggests controller button changes
Sony describes a system where a game player can speak to their console during gameplay to ask questions about their controller. The system uses a language model to understand the request and the current game state, then responds with visual, sound, or vibration cues rather than spoken words. One example shown is suggesting how to re-map buttons on the controller to a new layout.
In the filing’s own words
An electronic system is configured to use a tailored large language model to process natural language spoken by a computer gamer during gameplay. Non-verbal output can then be provided in response, where the non-verbal output may be visual, tactile, and/or audible. As examples, the non-verbal output may respond to a request from the user, may acknowledge that the natural language is being processed, and/or may indicate that a requested action has been registered for execution in the future.
What exactly they claim (claim 1)
1. A method, comprising: executing a computer game; accessing game state data indicating a current state of the computer game; receiving verbal input of a user during gameplay, the verbal input comprising a user question regarding a button on a controller; executing a model to process the verbal input and the game state data; based on the processing of the verbal input and the game state data, determining a non-verbal response comprising a suggestion to re-map one or more buttons on the controller from a first configuration to a second configuration; and based on the determining, outputting the non-verbal response, the outputting comprising causing display of the suggestion to re-map the one or more buttons on the controller from the first configuration to the second configuration.
Facts
- Company
- Sony (Sony Interactive Entertainment Inc.)
- Official title
- MULTI-MODAL COMPUTER GAME FEEDBACK USING CONVERSATIONAL DIGITAL ASSISTANT
- Publication number
- US20260315919A1
- Application number
- 19722296
- Filed
- Jun 26, 2026
- Published
- Oct 15, 2026
- Inventors
- 2
- Claims
- 20
Classified under
A63F 13/424 Human necessitiesA63F 13/215 Human necessitiesA63F 13/285 Human necessitiesA63F 13/533 Human necessitiesA63F 13/54 Human necessitiesG06N 3/0475 Physics & computingMore from Sony in Games & media
- Sony patents game engine that reshuffles scenes based on your heart rate Sep 8, 2026
- Sony patents game controls that learn your unique hand movements Oct 13, 2026
- Sony patents eSport events where fans' cheers give players in-game power-ups Oct 20, 2026
- Sony patents game gear design using AI mood boards Oct 20, 2026
- Sony patents game feature that warns players before they hit a hard part Sep 3, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.