Lukas Schäfer - Decision Making in Modern Video Games From Human Play to World Models

This talk presents recent research on decision-making in modern video games conducted at Microsoft Research Cambridge. After motivating why video games are compelling testbeds for studying decision-making, I will present work examining how the choice of visual encoders affects the performance and training efficiency of behaviour cloning (BC) agents. Our experiments show that carefully selected pre-trained visual encoders can significantly reduce computational cost and boost performance. Given the substantial data requirements of BC for complex tasks, we then investigate predictive inverse dynamics models (PIDM)—which condition a policy on predicted future states—as an alternative to BC. These models have demonstrated improved performance over BC but remain poorly understood. I will present theoretical insights that show that PIDM's performance gains can be explained with a bias-variance tradeoff: conditioning the policy on future context can reduce uncertainty about action predictions but also introduce bias whenever future predictions are inaccurate. We further show that these insights translate into significant sample efficiency gains in 2D navigation tasks and complex 3D environments in modern video games. Finally, we move from decision-making models that model the future to world and human action models (WHAM), which combine an environment model (world model) with an imitation-learning policy representing human gameplay. Inspired by the recipe behind LLMs, we demonstrate the promise of scale for such models and explore how they can support workflows for video game creatives. Lukas Schäfer is a postdoctoral researcher at Microsoft Research in Cambridge, UK, where he is part of Katja Hofmann's team working on machine learning for video games. His work focuses on developing autonomous agents that enable novel experiences and tools in video games, with an emphasis on imitation learning approaches. Lukas holds a PhD and MSc in Informatics from the University of Edin