Sony has filed a patent for an AI-driven technology designed to predict game frames before the GPU completes rendering. The system aims to reduce stuttering and minimize latency by generating predicted frames based on the most recently rendered image and player controller inputs, bypassing the need for traditional interpolation between two completed frames.
The architecture utilizes a jointly trained encoder-decoder model. The encoder converts rendered frames into numerical embeddings, while the decoder generates the predicted output. This process runs in parallel with the graphics pipeline, allowing the AI to feed predictions into the display buffer while the hardware works on the next actual frame.
To improve prediction accuracy, the model incorporates motion vectors and real-time controller data. Training involves continuous video frames, where the neural network compares its predictions against actual subsequent frames to minimize loss. The system is designed to function without manual data labeling during the training phase.
The patent also outlines a non-AI alternative that generates synthetic frames by shifting pixel blocks based on motion information. While the company claims these methods could maintain steady frame rates without compromising visual quality, the actual impact on latency and image fidelity remains unverified as no performance test results were provided.
