Sonnet 2.0 (TF Dev Summit ‘19)

Key Takeaways

[Music] my name is Malcolm and I'm a research engineer at diamond high - Tamar and I'm a software engineer at MIT so we have state to talk about summit and hopefully answer some questions for you so what is Sonic why deep - found it useful and why it might be useful to you and then talk a bit about our plans for the future of the library so sonic is deep mines library for constructing neural networks intensive flow we've been working on it since 2016 and it's been open source for the last couple of years and you can try the turns flow 1.0 version with with this command line here but you might be thinking sounds interesting why does this exist and that is a very legitimate question so to answer that we have to take a little trip down memory lane so 2014 tense flow is in development of brain but there's no consortium released yet and unrelated lis I guess deep mind is acquired by Google so a little bit of code archaeology here so does anybody recognize what system this is written in hands up if it looks vaguely familiar okay not too many hands and so I censored some of this just to make it slightly harder so now it may be clearer that this is actually torch 7 code or Lua torch and specifically we're defining a module here that's what the the red arrow is pointing to so deep mind was an entirely torch 7 operation at least in the research side when I joined and it's a really great piece of software if you're you know interested in the sort of history of deep learning abstractions you know the abstractions in torch really go back to stuff from the 90s like lush so fast forward a few years we have like dozens of research projects going on hundreds of people involved and being able to kind of quickly share code between the projects was like really critical so somebody comes up with a new RNN everyone else can try that out with like minimal fuss and like no changing of the code and you know we thought we had a pretty good workflow and we're pretty happy with it and then we decided to just get rid of it all and start from scratch seems questionable but the reason is we were transitioning to tense flow and you know there's many great reasons to choose test flow we've been hearing about them for the last two days like top of the list for us was better alignment in collaboration with our colleagues at Google it was designed for distributed training from day one which you know torch really wasn't and that was becoming a problem in some of our research and we wanted to really take the best use the best advantage of TP use and obviously tensile is the best way to use them and you know we knew from initial assessments this was going to be flexible enough to build whatever we need on top of but we weren't actually sewing from scratch you know we had some stuff we really wanted to preserve and it's not that we want the same api's as lure torch that would not have been the right decision well you wanted the same engineering philosophy so what we mean by this is like hopefully we're gonna at least eight through this talk so first of all kind of composable modules using object-oriented programming now it's definitely totally valid so you have a kind of purely functional library but we found that when you start having lots of variable reuse are n ends those kind of features objects really made a lot more sense so we definitely wanted to keep that we wanted to decouple the model definition from how it's actually trained and this is really key and I will comment to this again shortly hack ability is really crucial we don't think we can anticipate everything I research scientists might want like you know years or even months ahead of time but what we can do is have code that they are able to like dig down hopefully not too far like make the changes they need to maybe by forking the module and we're kind of comfortable with that and then continue with their research and it's important to emphasize we're really optimizing for research like you've heard a lot about a lot of great solutions for kind of moving things into production and that's really important but the deep mind research that really happens very rarely so we prefer to optimize the libraries based on you know research progress next paper that kind of thing and overall we're looking for like a standard interface with really minimal assumptions that doesn't prevent you from doing anything so composability like what do I mean by this like hopefully you know anyone who's used in l SCM would be like yes you can implement this l of some linear modules and they contain the variables and they're less cm as a thing around all of that but you know you can go many levels up from there so the differentiable new your computer which was the first project I worked on a deep mind you know that's a module that contains an LS TM plus some other stuff and you know we worked on that and got the paper out and then you know the code is sitting there and then another team can come along and be like hey you know we have some system which requires an R or an N is currently an LS TM but like maybe we will just try this instead and you know it's been used in many places but I think the most significant may be was the capture the flag work which had some really cool results specifically with the dnc controller and you know they could reuse the code like they didn't need to ask us about how it works they didn't even need to read the code they're just like this is an RN that conforms to some API we'll just drop it in and it works well LC to the training cell donation so I did a quick survey we have roughly two setups for unsupervised learning four for reinforcement learning and many more cells for single projects and to be clear what I mean here this is kind of everything that goes on above the model like how do you feed it the data how do you launch experiments monitor them like what configurability is there for the researchers like all this kind of stuff is what I'm referring to is training set up and this might seem like a ton of you know repeated effort and duplicated code but we don't actually view this as a problem these different setups exist for like different research goals and like trying to push everything into like a single training cell even just for RL it's been tried we don't think it works so we're kind of happy with like lots of different selves which kind of coexist and the reason it's not a problem apart from that serving different research goals is that we can reuse soil modules between all of them and that means you know we're not like redoing the DNC from scratch so I've talked a bunch about what we've already done with sahn it intense flow one and tomorrow is going to talk a bit about the future so yeah a smoker so what will sonic look like in TFT deepmind are really excited for tf2 and are planning once again to invest and change your workflow and adopting tf2 egor execution has already been widely tried by researchers and they really enjoy the improved flexibility debugging and simplicity from the come from this and I'm looking forward to the benefits of this being part of tf2 but one thing we've noticed this is Sonic 1 was based on features like variable scopes quite heavily the and these are going away this isn't something that we're like worried about we're really excited that we follow TF into this much more natural and pythonic world so how do we plan on making Sonic in this new world so we've built TF modern as the base of Sonic T this is a stateful container which provides both variable and model tracking it's been designed by engineers at deep mind in collaboration between many individuals at deep mind and brain learning lessons for both Sonic 1 and the newer talk it's now been up streamed intense play and you can try it out in their alpha that's been released over the course of dev summit and it will soon form the basis of many higher-level components within TF including things like TF carrots so what about modules modules can have multiple food methods addressing a limitation that we found within sonnet 1 they can be nested arbitrarily allowing for the creation of complex models like the newer machine there was talked about earlier and they have automatic name scoping which allows for easy debugging so while we're trying to do a solid tea we're aiming to create a library which makes very few assumptions about what users want to do both with their networks and their training links it's been designed in close collaboration with researchers and we think it's a lightweight simple abstraction that's as close to the maths as it can be so what the features of site sauna has the ability to have multiple ford methods something enabled by TF module but what does that really mean there'll be an easy way to create a set of methods on an object they can hold have access to the same state variables an example of whether this is useful we're in a few slides time one thing that we've found that's really crucial with Senate one is that it really easily to compose it with other systems so no one works at the box with replicator which is our internal solutions to dish to eater training even though these libraries were never designed with each other in mind they don't even have a dependency on each other they both just use tents flow under the hood it's up to users to define how they want to compose these systems together and decide where they want to put in things like that oh geez to ensure that the gradients are the same across all the devices we plan for the same philosophy to be implemented in SONET to replicator has been mail merge with distribution flushing so you'll all be able to try this out and you can find more details on replicator on the deep one blog post so what do we want to do with insight we also want to continue to provide the flexibility and composability that we've already heard about that they haven't v1 such as things that modal's can be composed in arbitrary ways and that no training levels provided we still don't want to aim to predict where the research will go so what does the code look like this is at the linear layer it's pretty much the implementation just without some of the comments and one thing we've been really excited by is how simple element plantation we've been able to create on top of TF module and TF terrace researchers by Nature about pushing boundaries and we think these simple airs with minimal boilerplate will encourage four key inheritance as a way of trying out variations another thing I just wanted to highlight is that modules don't necessarily have to be less or even half state they just inherent from TF module and get the name scoping from that in SONET we also implement optimizers as modules that simply nodes on a graph and benefit with the names paper in from TF module which greatly adds to the visualization of graphs and debugging of operations later on the forward method optimizes isn't even called cool it's their supply updates which doesn't return anything it just takes your gradients and your variables and applies them together and as promised here's the example of using multiple four methods making something like a variational auto encoder where encapsulate both an encoder and a decoder so they can be trained together whilst exposing both the forward methods on them both as a way for testing and using them later on so what we found when implementing SONET in tf2 we've been doing it for a while now and we've found that we've had a much simpler more understandable code than before the debugger has been vastly improved and less painful with eager mode and we have initially seen some significant speed ups with tear function I think this will be a hugely powerful tool going forwards so when can you try this out the roadmap for sonic 2 is on the way it's on the way TF modules already available in the alpha release and TF cool we're iterating currently on basic modules now as researchers with the alpha release and github soon and hopefully we'll have a beat released sometime in the summer and now over to Malcolm he's going to talk about some of the projects some of which you may have heard of which have used and been enabled by features of sonnet one that we're taking forward into sonic 2 thanks to our and so just a few like samples of stuff that's been done over the past few years in SONET and tons of flow one so this is the generative query Network so it's kind of receiving like 2d rendering observations at the top and then you know it's able to like infer what the 3d environment looks like and you know put the camera in any new position and you can see here like the the different probabilistic views of like what the whole environment could look like and this is like a classic example of several papers worth of research which were all like modules which then get kind of built on top of so the original paper was draw back in the newer torch days and then we expanded on that with convolutional draw and then the generative query network alpha star something quite recent and so on it was really key here because there's you know there's no assumptions on the shape of you know what's coming in and out of each module both the observations and actions for the starcraft environment are like extremely complex hierarchical structures the action species have a kind of also regressive model so like you know where you're gonna move a unit and then where you're gonna move the unit to and like the fact that Sanne had been built with like minimal assumptions about what goes in and out meant that this was actually quite easy and another interesting aspect here is how this module sorry the model is actually trained so I mean the details are on a blog post on the deepmind website but it's the Alpha Starleague is like quite a complicated setup where there's lots of different agents and they train against you know frozen versions of themselves from the past and try and learn to exploit them and then you make a new Asian it tries to learn to exploit all the previous ones and you know this kind of custom setup is a good reason why we don't just try and stick with one training cell for everything because you know even a year ago I don't think we could have anticipated how what kind of elaborate training scheme would have been necessary to make this work and finally began I suspect a lot of people are quite familiar with this but just in case you're not this is not a real dog not a real photo this is randomly generated and I think the key aspects of site here was that the underlying gang architecture was like maybe relatively similar to things that had already been in the literature though there was a few key components which made it like really a lot better one of them was being able to train it on an entire TPE pod and a key aspect here was cross replica passional so you know that the researchers involved could just say you know here's a normal gun made out of normal modules we need to like hack the batch norm like completely replace it to do kind of interesting TPU stuff and then everything else is just kind of just works as normal and obviously we can't talk about began without briefly having a look at dog ball so here's a little bit of dog ball and then that's probably enough so in conclusion Sanne is is a library that's been designed in collaboration with deep mind research scientists we've been using it very happily for the last few years we're looking forward to using it in the tense flow to version it works for us and it might work for you thank you very much [Applause] [Music]