Empowering game developers with Stadia R&D (Google Games Dev Summit)

Key Takeaways

[Music] thank you for joining us today my name is Aaron Hoffman John and I am head of creative for stadia research and development I'd like to start by saying how personal the cancellation of GDC has been for our team in the last 17 years I think I've missed just one GDC for many of us is the week that we set our calendars by an event we depend on for our careers and for refilling the well of our connection with people who mean so much to us so thank you again for joining us today and we hope that you're finding ways to safely connect with each other in this very challenging time I also would like to thank our event organizers here at stadia and with GDC who have poured their hearts into this event behind the scenes year after year it is a privilege to share our work with the game development community and none of this is possible without them I came to Google two and a half years ago after spending the last 17 years as a commercial game designer I started in text-based games and followed the arc of game technology innovation through consoles mmo's kids virtual worlds social games mobile games and more throughout my career I worked on each of these emerging markets ahead of their time and I came to stadia because the prospect of distributed resources for games in the cloud seems like the obvious next step in our progress as an industry at Google I co-founded stadia research and development a small team composed of veteran game designers and developers working closely with machine learning engineers and Google infrastructure engineers you might be familiar with some of our work from the style transfer demo we showed at last year's GDC our goal is to bring the best of Google to game development we do this by building relationships with some of the amazing technology teams inside of Google and creating experiential prototypes based on those technologies last year we showed you some of the early promise of this line of work with our style transfer art demo and also our stream connect demo we also presented our conceptual design for a new technology called game bus which would allow developers to connect the resources between simple stadia instances with subframe latency today I'll be showing you the first light on these technologies working in real game settings first our project codenamed chimera is a strategy card game powered by generative machine learning for art and m/l agents for gameplay balancing and virtual opponents second you'll see how semantic machine learning can power believable King game characters without a single line of scripted AI behavior next we'll show you how game bus takes peer-to-peer player multiplayer networking to a whole new level finally we'll show the first proof of cross instance resource distribution with a game bus demo we call ridiculous soccer computing complex AI behavior in a separate stadia instance and weaving those instances together into a seamless experience when we think about the potential of stadia one thing that virtually every developer we talked to you is excited about is its potential to reach millions or even billions of new gamers today there are more than 40 million Chromebooks in use in education alone and Chromebooks make up over 20% of the commercial notebook computer market these devices and the tens of millions of pixel phones in circulation added to the tens of millions of chromecast's in the while our stadia perfect surfaces and those numbers will only grow Google products touch over 2 billion users in total a platform built for accessibility across more than a billion existing potential users requires a new kind of thinking how do we convert those users into the next billion gamers and when those gamers arrive how can we be ready for them this order of magnitude increase in the potential high fidelity gaming audience requires new approaches to game development one of our most critical goals at stadia research and development is to empower as many developers as possible to reach that potential massive audience enter chimera our experiment into applying machine learning to every aspect of the game development process our goal with chimera was to create what some have called human AI centaurs a seamless partnership between human and machine the combination of which is greater than the sum of its parts advances in machine learning have shown us clearly that the next leap forward in technology comes from empowering people with machine learning tools so what does that mean you apply it to game development one of the biggest problems in game development is rapid often repetitive content generation a game and development wants to be fluid and iterative responsive and agile but when the work of hundreds of people depends on a fixed target iteration slows and so does innovation in 2019 we showed a vision for a rapid visual ideation process made possible by artistic style transfer applied to real-time game graphics but we wanted to go further we wanted to go beyond the post-processing layer and into generating game art itself so that's what we did first we thought about what an explosive quantity of instantly generated game art might make possible virtually every developer who has shipped a game has experienced having to cut or cut back a feature because of its art requirements so we turned that limitation upside down and said what would we do if there were no limit on the kind of 2d assets we could generate creature breeding came to mind but of a kind not possible in the real world what if we could combine animals arbitrarily and keep combining them the chimera concept was born we knew that generative adversarial networks had been used to generate images and deep tore one of our machine learning engineers demonstrated a version of the technology and argued for applying it to the game assets his early experiments proved promising so we went all in on creature combination asset generation and the generation of backgrounds for behind those creatures a machine learning model is nothing without the data it's trained on the most impressive image generation relies on tremendous amounts of common data such as human face photographs or landscape photography by definition what we were attempting to do had no basis and existing data if it did the problem wouldn't be nearly as interesting or as applicable to games which are usually creating artistic works of fantasy for the landscapes we began with license sources of landscape photography for our final background another of our ml engineers strongly produced the style filter that softened the edges of the backgrounds while giving them a brush-like effect we wanted to evoke a sense of the biome the creature would live on but not overwhelm the card with landscape detail and that brings us to the creatures we began by training on a large animal data set even though we knew it wasn't likely to produce the results we wanted we wanted to prove that it couldn't so we trained a network on a large animal photograph data set and produce these animals some of them are pretty cute but they definitely didn't have the drama we wanted for our epic card game so we had to generate our own data our technical artist Lee Dodson collaborated with our machine learning engineers to create turnarounds of 3d animal models essentially procedurally generating a massive image data set by looking at the gap between our initial photographic animal tests and the kind of dramatic portraits players expect in card games we were able to design the parameters needed for our data set characteristics we needed to create a dramatic presentation the animal had to be posed dramatically not for instance like the standard image of a cow in a field or the kind of animal reference you can commonly find on the Internet the model had to be dramatically lit usually from above and the camera had to be zoomed in an angle just right once trained on this data set we could generate a novel animal that did not exist animals very like it existed on our data set but there was no exact match all of the animals you see here are generated by our conditional generative adversarial network system then pass through a procedural style filter the backgrounds are generated as well but creating convincing animal cards was just the beginning now we needed to combine them this also proved uniquely challenging when we just let the machine do the work it created a kind of average of all creatures that became indistinguishable and also had the tendency to spit out what the team referred to as nightmare fuel we knew we needed to give the artists more control ultimately we took a two-pronged approach in one approach we collaborated with a team in Google brain to produce the chimera painter an artist tool that could be used segmentation maps that a conditional generative model could fill in then applying our procedural approach we used Houdini to combine parts of the 3d models and render them in vector color to procedurally generate the segmentation maps then combine them this didn't produce perfect results but it got us 80% of the way there and also produced general shapes that the model was better at recognizing one important part about the way this generative model works is that we're not telling it what animal to put where it's deciding what animal to use based on what silhouette it sees in the segmentation map the result of both approaches eventually produce creature combinations that were convincing dramatic and machine generated here are a few of our favorites that came out of this system there are literally thousands more but we'll have to keep going here you can see the final card composition we were taking our generated landscape laying it behind the chimera and unifying both using procedural filtering this allows us to reach the card combinatorics we were looking for in allowing players to combine any creature in the game to create hybrid game pieces once we had this amazing ability to generate thousands of creature combinations on demand we had a different problem how do you design for such an explosively complex system just as every developer knows what it's like to cut a feature based on art budget every strategy game player knows what it's like to play a game that has gone out the door with a game ending exploit in its design from a designer standpoint it can be a lose-lose situation either you design the game to be simple enough that you can predict all outcomes and it's boring or you design the game to be so complex it isn't realistically feasible to predict all of those edge cases and combinations generally the balance of a complex game with many interrelated dynamic systems is a problem broached only by large well resourced teams with dozens of designers we wanted to tackle it with one there is some precedent for using neural networks to assist in game balance in 2018 theresa Deranger gave a terrific GDC talk called race for the galaxy a neural network in production which walked through the process of using temporal difference learning to evaluate board States this approach is fascinating and clever and promising as well and how it hybrid machine learning with traditional AI scripting for our purposes it was important that we didn't use any hand scripting in the machine process because we wanted flexibility in trying different game rules so we trained a single network to play the entire game we also wanted an explosive combinatorial system involving the merging not of not just visual assets but of their underlying game behaviors this meant we had to take an unsupervised learning approach an agent that learns through self play it put us more in deep mind territory however as with real-time artistic style transfer the unsupervised learning community was skeptical we could pull it off within reasonable compute limitations Magic the Gathering for instance a game whose combinatorial complexity was closer to our ambition is famously considered impossible to solve with AI we ultimately arrived on a hybrid solution that used a data-driven game simulation architecture designed by our UX engineer Maxwell Hanuman to generate all playable moves in the system then paired this with a reinforcement learning agent trained on millions of playthroughs of the game here are some analytics that came back from those millions of playthroughs you'll notice that's one of the exploits I mentioned on their own the agents found a degenerate strategy by applying the dodge power up to the Taranis or I knew that the t-rex was strong and I knew that I wanted it to be the strongest creature in the game but something this okie is just not fun so we made some adjustments a series of adjustments as you do with these things that preserved the intent of the torrontés or which was to keep it big and frightening but brought that curve back under control ordinarily this kind of thing might not be caught until the game could be played by hundreds of real people but because I now had access to agents who could play the game tens of millions of times after every change I made to the game balance I could progress through that system iteration much more freely and quickly without needing access to hundreds of play testers it was also fascinating to me how as a designer I could look at these analytics and see creative choices that I'd made in the balance of the game often fluidly and intuitively during the initial prototyping processes I built the cards reflected back to me in the analytics I'm used to this when working on a live game with thousands or millions of players but it's quite a trip to see this data before anyone outside the team has played the game so let's look at it all together during gameplay I have a hand of cards and can begin playing the game by placing buying on titles onto a common board space these biomes can be expanded upon or evolved into new kinds of biomes and creatures in the game represented by cards have different biome requirements my chimera begins as an egg but in this case I've evolved it once already by absorbing a bat I'm gonna merge it now with one of the creatures on the board the Taranto sort when i absorb a creature with the chimera the absorbed creatures abilities are translated into new abilities for the chimera as are the creature stats so we can see here the resulting double chimera half bat and half Tyrannosaur I could absorb any of my own creatures here but I think I'll pick up the hyena I can also attack any of my opponent's creatures with my own creatures or play spell cards that have direct effects on any entity on the board so the strategy space of the board game here is exponential especially as the board increases in complexity during play and as I make decisions about what kinds of abilities I want my chimera to have and what kinds of creatures I can support on the available biomes meanwhile my opponent is an ml agent and it's deciding on a turn by turn basis what the next best move is I have to say after playing this as much as we have it becomes evident that the game is just heart there's a lot of depth due to all the combinations and we definitely get that unsettled gut feeling from watching how good the ml is at playing this it knows without us having explained to it that it definitely wants to place all of its creatures on the board it also wants to enhance as many of its allies as it can with special abilities it makes it extremely interesting to watch what it chooses to do now the thing with an unsupervised learning agent is you don't necessarily know why it's making the choices it's making this is called legibility in ml terms and we wanted to get some insight into the decisions that the agent was making so one of our ml engineers she found you worked with our ml team to create an interface that would let us see what the machine was thinking at any given moment in gameplay terms this is an automate coach but again we didn't script any of it I can click a hint button and it will give me a recommendation of three moves along with the estimated game when probability associated with each I can also check the graph to see how the machine thinks I'm doing in the game overall in overtime now let's say I decide not to do what the machine says and make a few moves that I know are bad in this case I'm gonna buff one of my opponent's creatures and then buff one of my own if I check the graph again we can see the ml was not impressed with the first choice I made and then things got a little bit better and then they got worse again what's next for us with this work is really to let the agents loose and bring together these threads of game balance and game agent play our goal was a game that would allow the machine itself to build decks and to personalize those decks to the skill level of the player based on what it's estimate of the player's ability was because that graph is also measuring the player's ability as they play what's clear to us thus far is that these methods show significant promise in increasing the reach and potential of game development teams if you'd like to find out more about each of these projects we'll be releasing a series of blog posts on stadia dev where you can dive a lot deeper now we'll show a technology that attacks the same problem saving developers time by bringing machine assistance into the game development process in this case the technology is semantic machine learning and specifically a model trained by our colleagues at Google AI semantic machine learning models are trained on massive datasets of language and create multi-dimensional graphs of the relationships between words used in those datasets Anna Kipnis describes this application of semantic ml coming from her 17 years of experience as a narrative and gameplay programmer a double fine she recounts infinite seeming hours spent scripting for every possibility of a player's interaction with an NPC what you're about to see in her verb world demo has zero AI behavioral scripting a journalist recently described this demo as granting our fondest secret wish to talk to an adorable Fox I'm pretty sure was an is not so secret wish in this the player can type whatever they want to the Fox for instance saying hi to everyone and it'll wave back or asking the Fox for some coffee when asked the Fox brings a small mug to the player again it has not been told to do this it used semantic ml to connect the request of coffee with the nearest related objects in this case the mug and then brought it we don't leave this purely up to the free association of the machine however anna has built a developer tool that allows a designer to alter the categories of responses the Fox might have two things this is a bit more like behavioral scripting but radically simpler since it's just interpolating between very broad commands so in this case she's used it to create two different foxes with different personalities one happy and one sad note how the foxes respond quite differently now to the same commands or interactions and how this shows personality anna describes this as giving the Fox a rich in her life that mysterious sense that the best AI driven games have and give us that the game characters are alive for more on this technology keep an eye on our blog at stadia dev for a deeper dive into verb world with Anna Kipnis to check out the semantic technology our friends at Google AI have created a tool called the semantic reactor it's a Google sheets add-on you can use to experiment with language models it's available as an experiment within the Google cloud a I workshop check the description of this video for a link and we'll also link it from our upcoming blog post now we'll take a step back from the magical but often head-spinning world of ml and into the distributed computer tsa's for games with sub-millisecond latency if you need a snack the great thing about a virtual conference is you can pause the video the first demo we'd like to show you uses the proximity of stadia instances in the cloud to take peer-to-peer multiplayer techniques to an entirely new level our development partners and friends tell us that one of their biggest challenges is multiplayer networking and we know well that it's generally just not a fun challenge when you get it right it's invisible to the player and when you get it wrong rage ensues so one of our senior engineers Michael Todorovic set out to figure out what game bus message-passing technology could mean for multiplayer game networking what we have is another early proof of a pretty promising idea what you see is guilt the beautiful narrative game from tequila works those familiar with tequila works noticed that this studio excels in artistic sensibility storytelling and puzzle design they have never launched a multiplayer game so what if we could take their game and seamlessly transition it to multiplayer Michael had a theory that the proximity of stadia instances in the cloud would mean that peer to peer approaches to networking could have new potential by distributing the player connections differently we could produce a scalable solution that was a fundamentally different kind of multiplayer networking architecture that would mean a technology that could go from two players to a thousand players seamlessly and elegantly we wanted to prove that this could work even with games that had never been designed for multiplayer so we asked tequila works if we could borrow a couple of sally's we built a peer-to-peer networking system that could connect to game bus and then placed some simple hooks for the relevant data into guilt what you see is the end result working here naturally the question is how many players can this support for that answer we have to turn to simulation into the architecture ordinarily a major challenge with multiplayer is round-trip latency between the client machine of one player the server the client machine of the second player and back again the further apart the links in the chain are the higher the latency and subsequently the probability of poor experiences D sinks or errors increases reconciliation of these errors situations where either of the clients or the server has a different picture of reality than the others is one reason why multiplayer networking is so difficult with stadia the proximity of the client instances in the cloud allows us to take away the intermediary and use the server less architecture with unprecedented scalability when we calculate bandwidth Crispus per second needs for this architecture we're confident in reaching 1,000 simultaneous players without D syncs are perceptible loss of quality in the future with improvements and hardware and firmware this number should only grow instance proximity of course is just the beginning even when it comes to leveraging it four new multiplayer technologies in 2019 if you caught our vision for game design in a streaming future you will have seen our early conceptual diagrams for a technology called game bus game bus is named for the message passing bus inside a computer but in our case we're passing messages of arbitrary size and not just within a machine but between them sometimes those machines are side by side in a rack and sometimes they're not game bus is the transport that allows us to pass chunks of data large enough to meet game development needs between resources in the cloud because of Google's unique cloud infrastructure we can pass this information with subframe latency this is the first demonstration of game bus working in a game context you see here two teams playing soccer unlike chimera this is a traditionally scripted game AI system we wanted to show something practical and beefy these scripted agents have a lot to keep track of where the ball is where their teammates are where the goal is they work as a team and the members of the team have roles as they would in a real soccer match if we drop a few more balls on the screen the work the agents need to do increases you can see here that the compute load is pretty low our interface on the right shows you what parts of the game consume various quantities of resources these are bowls running through it right now our AI load is low because we have so few agents on the screen but what if we add some complexity to the environment the AI task starts to increase in complexity and if we add more balls and more teams it increases even further and you can see that load starting to go up more Bulls more bulls is always the answer now we can see that we're beginning to hit the limits on even what a quite powerful stadia instance can provide if we continue to pile on load by adding even more teams we can see the framerate drop as the compute resources are totally consumed by the complexity of the aggregate AI being taken on by the processor but because we're in the cloud we can take advantage of adjacent resources and simply rebalance them inside the rack if you watch the columns on the far right of the screen you're seeing them take that resource load off the column on the left in real-time this is game bus now I'm seeing the framerate pick back up again as the other resources naturally balance the load being distributed to them by the parent instance the game is running smoothly and I can continue to add more units to the game without taxing framerate the combined power of the instances you see here is simply not possible on any other platform and yet these are resources were able to access readily on Stadium thinking about the potential of game bus and distributed real-time compute for games is to think about the architecture and capability of games in a whole new way we wanted to demonstrate this to you today to spark the beginning of a new kind of thinking about game resources and what we would do with them if the boundaries were so accustomed to could be lifted game bus is the reason why we continually ask developers where they're hitting the limits on their current platforms and what barriers they just aren't able to break today this is just the beginning for game bus we're excited to show you the first look at the technology working but we're even more excited to hear your ideas of what to do with it our next steps with this approach include expanding our AI service to navmesh navigation and then real-time physics thank you very much for tuning in for our research and development update if you have feedback for us on what you've seen today we hope you'll consider filling out our developer survey you can find it linked from this video and also linked from our blog post at sadya dev over the next few weeks we'll be releasing articles that delve deeper into each of these technologies along with links to how you can begin experimenting with them on your own thanks again for watching and we hope to hear from you soon [Music]