How to Make an Amazing Video Game Bot Easily

Key Takeaways

yes beat you again sucker one day I'm going to make a bot that beats you in any game keep telling yourself that SJ hello world it's SJ and let's make an amazing video game bot in just 10 lines of code that can play a huge variety of games video games have been around since the 50s when Joseph Kates publicly demoed tic-tac-toe at the Canadian national exhibition that bought you simple scripted actions that ran the same way every time regardless of whatever move the player made his demo got people hyped though because no one had ever seen a computer play a game before and they were lining up off the block to check it out the game Bots that were invented afterwards for games like Nim and space War were similar but Along Came pully I mean pong the pong Bots paddle had to make decisions based on the human player's actions and that made it feel more realistic pong marked the beginning of using cisic to create game Bots heuristics are educated guesses and pretty much every single video game bot since pongs has used them a bot will map out a possible set of decisions as a tree of possibilities then use one of many techniques to pick the best one but as cool as that sounds it still always boiled down to a bunch of if then statements if Pac-Man moves this way then the blue go should move this way if Master Chief sees a grunt then it should run in circles like my Facebook Newsfeed if Captain Falcon is being annoying AF then your team bot should help you pone him Squad goals but yeah video game Bots have pretty much always sucked because there are only so many edge cases that a programmer can predict like if the human in Fallout 3 has a pistol and isn't moving and there are no enemies nearby run into each other we need to think about this problem differently when you or I start playing a game we don't know anything about its environment beforehand the Hallmark of intelligence is our ability to generalize but can we make artificial intelligence that can generalize to solve any task a team of researchers at Deep mind recently got close by creating one bot that could beat almost any Atari game knowing literally nothing about the game beforehand no game specific hardcoded rules at all it was just fed the raw pixels of the game and its controls using those two things it learned how to be almost any Atari game it was given it did this using a technology called Deep learning if you take a deep neural network and feed it lots of data and compute it can learn to do a whole lot of incredible things the field of deep learning right now is where physics was in the early 1900s the state-of-the-art in a huge number of subfields like vision and speech is being broken almost every other day it's a very exciting time right now the Marie curries and Albert Einstein of computer science are all alive right now and newcomers are coming in every day Deep Mind is awesome and they keep a good chunk of their code private since Google uses it to outperform its competitors but then Elon Musk came along and it was all like I think it's important if we have this incredible power of AI that it not be concentrated in the hands of a few and so he co-founded a nonprofit called open AI whose goal is to democratize AI so anyone can use it and just today they released something called universe universe is a platform that lets you build a bot and test it out in thousands of different environments from games as simple as Space Invaders to Grand Theft Auto to protein folding simulations that could cure cancer you can create a bot and the better you make it the more games it'll learn to become amazing at you can compete with other bot developers to see whose bot beats the most games and universe has other environments too for web interface tasks like managing emails and booking flights if you create a bot that's able to defeat any environment you're not only the dopest coder of all time you just solved intelligence we could then use your Bot to solve literally everything from global warming to Poverty to all known diseases so with that let's create our first simple bot in just 10 lines of python code in our first two lines of code code will import gy and universe gy is open ai's original code base that Universe Builds on and extends to include way more environments and features those are the only two dependencies we'll need now we can select our environment we'll Define an environment variable called EnV and use jy's make method to Define our environment parameter there's so many to choose from it's hard to pick but let's go ahead and pick the popular Flash game coaster racer Universe lets us run as many environments at the same time as we want but for now let's just use one our next step is to initialize our environment with the reset method it'll return a list of what we call observations for every environment we've initialized an observation is an environment specific object that represents what the agent observes like pixel data of what it sees and the state of the game initially we'll just have an empty set of observations since the game hasn't started yet now that we've initialized our environment let's go ahead and create a while statement so our agent will just keep running inde Ely we're just going to have our bot do one simple thing it's going to hit the up Arrow this is formatted by first specifying the type of event the key then true which means press it and we'll do this for each environment's observation we'll call this an action and store it in our action variable now we'll call our environment step method to move forward one time step and use the action as a parameter this is our implementation of reinforcement learning our bot will take an action in our case pushing the up Arrow then it'll observe the result and may or may not receive a reward if that action was beneficial to its goal which in our case is increasing the game score open AI uses a custom image recognition module here to read the game score in order to return a reward this module is included in the environment so we don't need to worry about it if it does receive a reward we could update our bot to do similar actions in the future so it gets better over time through trial and error so the step method returns four variables an observation of the environment a reward a yes or no value if the game is done and some info like performance timings and latencies for debugging and it'll do this for all the environments you train your Bot in simultaneously lastly we'll render the environment so it's visible to us let's demo this baby I'll run the code in terminal and it'll connect to our VNC server in our local Docker container running a flash enabled Chrome browser the prescripted mouse will click through the necessary screens to get the game started then our bot will start programmatically controlling the game remotely yeah our bot really sucks but how dope is this we can do this for as many games as we'd like and to make it better we can try different strategies like random search or hill climbing or just replicate what Deep Mind did they fed the observations that their bot received into a neural network that updated its connections to get better if it received a reward open aai already has a starter bot that uses deep reinforcement learning via tensorflow that I'll put a link to in the description and so to break it down open AI universe is a platform that lets you train and test Bots for thousands of games and other environments reinforcement learning is the process of using trial and error similar to how we learn to improve a bot and if you create one bot that can succeed in any environment it's given you just solved intelligence the coding challenge for this video is to create a bot for just coaster racer that is better than this video's demo code post your GitHub Link in the comments and I'll give a shout out to the winner in my video one week from today and I'll do a one-on-one Google Hangout with them just to say hi and talk about whatever for for now I've got to make a laundry folding robot so thanks for watching