Teaching Robots to Play | AI and Games #12

Key Takeaways

[Music] this is AI games and welcome to a new lecture entitled teaching robots to play why science loves gaming quick heads up that the original article as well as the video as part of this is a reproduction of a talk originally given during the res sessions of the egx 2015 event at the NEC in Birmingham UK in September mber 2015 one of the most interesting aspects of human behavior and by extension applications of intelligence as a species is the relationship that we humans have with games in principle games are tasks or challenges designed to stimulate the brain however while our brains are often forced to deal with demanding and often stressful mental tasks as part of our daily lives we still have time for games and the Notions of play games are designed to keep us challenged But ultimately do not prove so taxing that they prove too demanding or stressful for our regular consumption it's certainly interesting to observe the relationship between humans and the notion of play as a means of relaxation there are a number of reasons why games prove to be an interesting domain to explore scientific problems and more importantly AI challenges perhaps unsurprisingly these are largely reasons that humans already embrace them one major reason that good games prove so effective is that they act as Frameworks for reward through structured activ ity games Define Loops of behavior that have completed and repeated successfully will reward the player through a number of means some of these rewards are mild in nature and maybe purely cosmetic whereas others allow for a sense of progression to be conceptualized by the player these interactions continue to increase in scale but help maintain a player's interest and momentum until the long-term and explicit reward is achieved the Super Mario Brothers series is one of the finest examples of how reward Frameworks can be used to drive and maintain player interest reward interactions in the loops of behavior required to release them are often referred to as compulsion Loops whereby we maintain a player's interest by insuring a reward within an abstract and relative time frame short-term Loops are often the result of simple interactions that may be largely cosmetic but help maintain a player's engagement the interaction and response from the collection of coins and super mar Brothers may seem simple but the use of counters and sound effects provide positive reinforcement to users that their actions not only make sense but also work to BS their long-term goals this subsequently scales to medium-term compulsion Loops conceptualized through levels given every level of Super Mario games celebrate the fact the player has completed that activity returning back to coin collection continued adoption of the shortterm loop rewards you with extra lives this medium-term Loop is now reinforcing a player's continued adoption of the short-term Loop and gives not only context but a real quantifiable reason to continue doing it this scales farther into long-term Loops of activity as levels are grouped into Worlds with the requirement to defeat a boss enemy with closure achieved through defeat of Bowser in the eighth and final world ultimately the point here is that we have this confined system within which Intelligent Decisions can be made we can quantify their value as well as identify their position within the road map of future actions you can take in order to win that game conversely good games are able to quickly point out bad interactions you're making and thus reinforce to you that doing certain things is bad while Mario does a good job of this it's arguably his competitor son at the Hedgehog who signifies this even better with significantly exaggerated behavior in the event rings are lost thus losing all progress on that medium-term Loop but also when losing lives the next major factor that helps build a game as a valid scientific problem is that it has to be fun while fun is a largely subjective notion there's evidence to suggest that the level of challenge involved must meet a certain threshold in order for it to be interesting in the eyes of when we consider this from a computer science perspective we would actually classify that games must be at least non-deterministic polinomial time hard or NP hyen hard for short in short this means it's something of a non-trivial problem in computer science NP problems are typically ones that require some intelligent algorithmic process in order for them to be solved in a reasonable amount of time if we were to look at the range of games out there from online FPS games such as Call of Duty to racing games like Forza and even smaller and functionally simpler games such as Flappy Bird we can begin to recognize a large range of problems that are not only sufficiently difficult video games but carry a range of equally interesting decision problems in their own right despite this assertion of a certain level of computationally defined difficulty we would not paint all games as the same games can carry a variety of problem traits that make them interesting for autonomous systems to try and solve these traits can change between genres of games and even releases within the same game series these traits not only result in games exhibiting particular artificial intelligence problems but also begin to necessitate the use of certain AI techniques and methodologies given they are useful for that particular type of problem contrary to the popular opinion AI is not some black box design that will work in any and all circumstances AI systems and the techniques used to build them are typically specialist in nature focusing on very particular types of problems only now after over 50 years of research in this area are we seriously looking at the challenge es of building general intelligence systems which we'll discuss later there are several properties of a game that we will typically consider when trying to figure out how best to approach the problem there are three that prove to be rather important number one accessible knowledge just how much do we know about the game we're playing at any point in time this can be both a blessing and a curse depending on how much we actually know in some games we may not actually know everything about the current state of the game at this point in time this typically the case case in card games ranging from Texas holdom to Hearthstone we don't know what cards the other player is holding but can make some educated guesses that ultimately guide our decision- making conversely we exploit this imperfect information given that the opponent does not know what hands we might play but this doesn't mean that knowing everything about the world will help us that much either one of the best examples of this can be found in fighting games such as Street Fighter Mortal Combat and Killer Instinct in each case we can see the whole state of the world where the player is how much health or energy bar they have and the time remaining in that round despite this the number of possible actions that can be executed in that state leads to a large number of possible future States also known as successors this large number of actions in future States implies the branching factor of a given State meaning that even if we start thinking three or four moves ahead we need to start filtering out decisions that we don't think the opposing player will make given the number of possibilities is massive number two predicting the unpredictable one vital aspect of gaming has been able to see things before they actually happen allowing us to make quick decisions and react to changes in the world when playing platforming games we quickly learn the Mantia of the movement mechanics meaning we can predict whether we can make certain jumps in particular circumstances and quickly adapt to survive predictability can also come in really handy for dealing with enemies learning the behavior patterns of bosses in games such as as Dark Souls is key to knowing when to attack and when to fall back and defend but sometimes our model of that predictability is broken and that makes things so much harder for us one of the best examples of this can be found in Pac-Man where the original ghosts are deterministic in nature meaning we can learn in time what an enemy will do at any point however in the sequel Miss Pac-Man the ghosts are able to make random moves at Junctions if they so wish this results in a non-deterministic system meaning that we can no longer predict safely making the game significantly more difficult number three the players the enemies and the actors just how many characters are in this game and making semi-intelligent decisions this ties back to not only the complexity issue but also the branching Factor discussed earlier the branching factors influenced not just on how many actions you can make at any given state or frame of that game but also the actions that any other character can make in that world the number of unique configurations of the game world can explode at an exponential rate once you have multiple characters that can all do different things at once we need to figure out a which information is useful to us B what we can ignore and C how do we ensure that the space of all potential game configurations is tractable meaning that an AI can actually search it to find answers despite all this forboding and Gloom there's still an awful lot to celebrate AI research and games kicked off in full swing in the mid 2000s with a number of big projects bringing the community together as well as solving some interesting challenges the second major problem area to gather attention was AI that complained Mario which actually turned out to be a lot easier than we originally envisaged in fact the more interesting problems were not whether AI could play Mario but whether it could build Mario levels leading to a sudden Surge and research in procedural content generation be sure to check out the rather large overview of the gam plan level generation tracks we have here at AI games something of an interesting challenge to create AI that can play Unreal Tournament 2000 4 however unlike most challenges this was not about trying to beat the game or be the most effective at it but whether you could fool other humans into believing that the bot was not an AI this is an example of the churing test in which you build AI that can tackle tasks we would expect of a human but design it such that it cannot be distinguished from humans when observed this competition ran for several years until a winner was found that was able to fool judges into believing it was actually a human player despite this there's still a lot of work to do and many challenges yet to be solved we break down some of the bigger talking points here as well as point out to some interesting reading material for you to check out if you're interested PCG has become a big talking point in the academic Community for a number of reasons it's perhaps not considered AI as such in The Wider discussion but generative systems are making Intelligent Decisions to craft artifacts what makes this an even bigger task is that how to evaluate this content is highly subjective unlike many other AI problems such as robotics shed eding and even playing games we cannot wholly evaluate the quality of the final output in a robotics problem we can evaluate against the expected Behavior or even how well the robot Works in specific circumstances however with generated content while we can evaluate whether aderes to specific functional aspects we might struggle to identify more aesthetic and subjective aspects of that content so while we can quantify whether a gun can actually hurt an enemy or of a level is playable it's much harder to establish whether that gun was interesting to use or if that level was fun to play some links to check out include Angelina which is the AI system built to create entire games by itself the aforementioned Mario AI competition piece here on AI and games.com as well as checking out proc Jam a game Jam all about procedural content generation one of the most exciting Fields happening in AI right now is the notion of general intelligence the reason for this is in actuality AI systems are typically specialist in nature another they're very good at one thing and one thing alone this is contrary to a lot of Science Fiction in that for example Skynet and the Terminator or showdown in the system shock are systems that are largely omnipotent and can solve any problem placed in front of them this can be seen When developing AI that can play a particular game while we can write an AI that can play Pac-Man it cannot play Super Mario Brothers and vice versa this is an issue that spreads far beyond games and into larger real world problems general intelligence is the challenge of building AI that can solve any problem you give it which is far more in line with the original aspirations of AI from the early 20th century this is now a big problem with research departments at universities as well as big tech companies attempting to solve it some links to check out include the three-part series The Challenge of general intelligence in games hosted on AI and games as well as looking at the general video game AI competition hosted at the University of Essex in the UK as well as Google deep minds work in exactly the same problem area to conclude as games become more increasingly complex so do the artificially intelligent systems that seek to learn from them we are fortunate in that gaming is such a vibrant and creative field given it provides a continuous body of complex and interesting problem spaces to be working within in our own way science loves gaming for our own selfish reasons with complex problem spaces that require reactive and long-term decision-making systems to handle some of the most dynamic and multifaceted domains outside of the real world itself though to be honest science is into games were pretty much the same reasons as everyone else we're here to have fun this has been teaching robots to play why science loves gaming on AI and games thanks for listening and be sure to check out more over on aiam.com