Game engines are an integral part of today’s gaming industry. They are responsible for running the core game loop, including updating the game state and rendering graphics. Traditionally, game engines are coded in a way that emulates carefully crafted software rules. However, recent advances in AI provide an opportunity to generate game worlds using neural networks instead of manual coding. One such project is GameNGen.
GameNGen Project
Researchers at Google recently published a paper titled “Diffusion Models Are Real-Time Game Engines“, which demonstrated a new approach to this using a technique called GameNGen. GameNGen uses diffusion models trained on gameplay data to act as the game engine itself, simulating a game in real-time.
Examples
GameNGen Two Phase Training Process
GameNGen uses a two-phase process:
1. Agent Training
An RL agent is first trained to play the game by collecting gameplay data over many sessions. Its actions and observations are recorded.
2. Diffusion Model Training
This data is then used to train a diffusion model to generate the next frame conditioned on past frames and actions. The model learns to simulate the game through self-supervised learning from the recorded agent data.
Real-Time Interactive Simulation
Once trained, GameNGen is able to simulate games like DOOM in real-time at over 20 frames per second on a single TPU, responsive to live player input. Moreover, it successfully tracks the game state over long-term simulations, performing actions and damaging enemies and objects as expected.
Evaluation Performance of GameNGen
To objectively evaluate the quality, human raters were shown short clips of games simulated by GameNGen side-by-side with the original game. Raters could only distinguish the real game 58% of the time, showing the high fidelity of GameNGen’s simulations.
Mitigating Divergence
One challenge is that errors accumulate when generating frames autoregressively, diverging from the ground truth over time. To solve this, the researchers introduced noise augmentation, where they added random noise to condition context frames during training. This allows the model to correct for errors between frames and stabilize long simulations.
Future Potential of Gaming With GameNGen
GameNGen represents a promising first step towards “neural game engines”, where games can be automatically generated in a way similar to how images are generated with neural diffusion models. So, this new technique could make game development more accessible and reduce costs.
There is also potential to develop new games or modifications directly from examples rather than code. Overall, this project reveals the capability of diffusion models to simulate realistic interactive experiences in the real-time.
| Latest From Us
- Google Knows Where You Are By Tracking Your Location Even With GPS Disabled
- Nvidia’s New Open Model NVLM 1.0 Takes On GPT-4o in Multimodal AI
- Do AI Coding Assistants Really Improve Developer Productivity?
- Nintendo Is Going Against Popular YouTube Channels That Show Its Games Being Emulated
- By 2027, 79 Percent of CEOs Expect Remote Jobs to Be Extinct
