AI Powers PAC-MAN - The Game Engine-Free Revolution

Key Takeaways

and video managed to recreate the whole pacman game with an AI trained on the game itself without any game engine and only using games the best thing from this paper is that this pac-man game copy is even playable this is a first in the field and here's how it's made this is what's AI and I share artificial intelligence news every week if you are new to the channel and want to stay up-to-date please consider subscribing to not miss any further news NVIDIA recently published a paper called learning to simulate dynamic environments with game gam where they visually imitated the pac-man game only by ingesting screen play and keyboard actions during training they made that happen without any underlying game engine they called this model game game it leverages adversarial training to learn to simulate games it is trained by observing screen play along with user's actions and does not require access to the game's logic or engine itself in fact it does not even require a game engine at all in this case they trained the neural networks on the pac-man episodes a few million frames in total paired with data on the keystrokes of an AI agent playing the game the main difference with this new model is that game gained features a new memory module to ensure long-term consistency and is trying to separate static and dynamic elements and yes before you ask what you are currently seeing was a hundred percent made with neural networks with no game engine at all to simplify the problem they framed this as a 2d image generation problem given sequences of observed image frames and the corresponding actions the agent took their goal was to emulate image creation as if it was rendered from a real dynamic environment that is reacting to the agents actions so game can ingest screen play and keyboard actions during training and aims to predict the next frame like conditioning and the action in this pac-man example an action will be a key pressed by the agent game gained is composed of three main modules first there's the dynamics engine which enables game Gann to learn how various aspects of an environment change with respect to the given user action for instance it needs to learn that certain actions are not possible like walking through a wall and how other objects behave as a consequence of the action this permit component is able to learn such transitions by implementing it as an action conditioned lsdm the engine maintains the standard state variables for LS TM HT and city which contain information about every aspect of the current environment at time T then it computes the state variables given a t ZT m t minus 1 and XT to communicate with the other modules and itself as you can see in this illustration the next module is optionally applied for environments that require a long-term consistency for example it's useful if you have an agent that needs to navigate through an environment this environment shall not change when the agent comes back to the same location a few moments later it's an external memory module which uses the neural Turing machine that allows their model to remember every scene it generates in the hidden State and design Ellis that enforces such long-term consistency which is a challenging task for typical models such as Aaron ins this module has a memory block and the attendant location at time T as you can see in this picture at all time the model knows the current location that the agent is located at and their previous t minus 1 location as well as the action taken during this previous step to get to where it is currently in short this new memory module encourages the model to build an internal map of the environment allowing the agent to return to previously visited locations with high visual consistency the last module is a rendering engine Theory Clee it can be simply implemented with standard transpose convolution layers however they decided to introduce a specialized rendering engine architecture for answering long-term consistency by learning to produce these entangled scenes I will not dive deeper into the architecture of this module in this video but I invite you to read their paper if you are interested in this part basically it is responsible for rendering the next stimulated image T plus 1 given a state at a certain time frame T using a purposely designed decoder that learns to disentangle static and dynamic components within the image this makes the behavior of the model more interpretable and if further allows us to modify existing games by swapping out different components to sum up everything the model learns key rules of the game both simple and complex just like in the original game pac-man can't walk through the maze walls he eats up dots as he moves around and when he consumes a power palette the ghosts turned blue and flee when pac-man exists the maze from the one side he is teleported to the opposite end if he runs into a ghost the screen flashes and the game ends the game can addition relies on neural networks instead of a traditional game engine to generate pac-man's environment the AI keeps track of the virtual world remembering what's already been generated to maintain visual consistency from frame to frame no matter the game the gang can learn its rules simply by ingesting screen recordings and agent keystrokes from past gameplay since the model can disentangle the background from the moving character it's possible to recast the game to take place in an outdoor edge maze or swap out pac-man for your favorite character game developers could use this capability to experiment with new character IDs or game themes similarities are used to develop autonomous machines of all kinds such as warehouse robots learning how to grab and move an object around or even delivery robots that must navigate side walls to transport food or medicine game game introduces the possibility that the work of writing a simulator for tests like these could one day be replaced by simply training a neural network suppose you install a camera on a car it can record what the environment looks like or what the driver is doing like turning the steering wheel are hitting the accelerator this data could then be used to train a deep learning model that can predict what will happen in the real world if a human driver Namaskar took an action like slamming the brakes of course this was just a simple overview of the game gain network I strongly recommend to read the paper and the interesting post and Nvidia's blog both linked in the description for more information leave a like if you went this far in the video and since they are over 90% of you guys watching that are not subscribed yet please consider subscribing to the channel to not miss any further news clearly explained [Music]