Sony patents server that rebuilds 3D game scenes for free-angle replays
Sony describes a server system that uses machine learning to reconstruct a 3D model of a game scene from the video feeds of multiple players. This allows the server to generate replay videos that can be viewed from any angle, not just the player's original perspective.
In the filing’s own words
In a main image output phase 320 defined as a phase for execution of a content application, a content server collects, as training images, main images transmitted to a plurality of client terminals, and performs machine learning to generate 3D scene information 324 associated with scenes (S 30, S 32). At this time, the content server also creates a heatmap indicating a density distribution of display viewpoints (S 34). In a replay image distribution phase 322, the content server generates replay images viewable from free viewpoints on the basis of the 3D scene information 324, and outputs the replay images together with the heatmap (S 36, S 38).
What exactly they claim (claim 1)
1. A content server comprising: an input information acquisition unit for acquiring, from each one of a plurality of client terminals, details of a user operation performed by a user for content being executed and information associated with a display viewpoint for specifying a display image; a main image generation unit for generating, at a predetermined rate, a frame of a main image indicating a state of a three-dimensional (3D) display world where a situation is changeable in accordance with the user operation, as viewed from the display viewpoint for each one of the plurality of client terminals; a 3D scene information generation unit for updating, at a predetermined rate, three-dimensional scene information indicating three-dimensional information associated with the 3D display world, by machine learning that uses the frames of the main image acquired from the plurality of client terminals aggregated as training data; a replay image generation unit for generating a replay image of the content by drawing an image at the predetermined rate based on the 3D scene information for each time step; and an image data transmission unit for transmitting data of the main image and the replay image to the plurality of client terminals and displaying the data of the main image and the replay image at the plurality of client terminals.
Facts
- Company
- Sony (Sony Interactive Entertainment Inc.)
- Official title
- CONTENT SERVER AND CONTENT PROCESSING METHOD
- Publication number
- US20260268609A1
- Application number
- 19660869
- Filed
- Apr 28, 2026
- Published
- Sep 10, 2026
- Inventors
- 6
- Claims
- 18
Classified under
G06T 19/00 Physics & computingG06T 15/20 Physics & computingMore from Sony in AI & software
- Sony patents AI that plays video games in your style Oct 6, 2026
- Sony patents AI that swaps game visuals for accessibility formats Sep 17, 2026
- Sony patents AI that changes game characters based on your play Oct 1, 2026
- Sony patents method to turn 2D videos into high-detail 3D animations Oct 15, 2026
- Sony patents AI bots that mimic your friends' gaming styles Sep 8, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.