Remote Collaboration as a Data Scientist

so let's kind of take a step back then and say well how can how can properly implemented ml ops help me in in the current global situation that we have where we're now all suddenly working from home and in brackets if I'm a data scientist or or a manager of data scientists well the answer to that question comes down to the collaboration piece that I mentioned earlier so that's kind of do a deep dive on collaborations so there are two wait there are two fundamental modes of collaborating between different people doing doing work and there's synchronous and there's asynchronous and synchronous collaboration is when people are sitting in a room together and they're interrupting each other when they have a question and when they need to get something done and in particular with machine learning it might even be that they're sharing an environment so they they might even be pair programming sharing a text editor so they take it in turns to use the keyboard they would be working on exactly the same data set in exactly the same environment and then you're kind of time slicing it well hopefully pair programming is is effective and okay so people can't make changes to this thing at the same time but we just take it in terms or there's an asynchronous approach which is to say that different people should be able to do work on different copies of things and problem with an asynchronous approach of course is that you then need to be able to cope with conflict it's like merge conflicts in in git which takes me to the second point here which is how how does software DevOps teams do it well with github basically and tools that are like github like git lab and bitbucket and and all of the tooling around that and the way that that works as I'm sure most of the people in the audience know here is that you can fork someone else's project effectively or if your or you can make a branch from master you can make some changes in your branch and while you're making the changes in your branch you're not trading on anyone else's toes when you then propose those changes back to the master branch that's when you can pull in new changes from the master branch and integrate them into your branch and that's when you have to deal with merge conflicts and then you can propose a version of the change which is up-to-date with respects the master branch that's basically commonly known as git flow and it's been very successfully used in pretty much every team on the planet people have tweaks to that approach with multiple master branches and so on but it's all fundamentally the same idea of asynchronous collaboration and then with DevOps team's adopting things like git ops where your source of truth for what's running in production is also in a git repo well you can use tools like terraform or you can version control your cuban Nettie's yeah yeah moles and then happy days you can use the same collaboration approach pull requests when when you're deciding to scale up the cluster or deploy any database and doing the other things that devops teams do so so how can you so what are the challenges applying asynchronous collaboration to machine learning well they are numerous the first problem is that it's the Jupiter notebooks don't go and control very well and a lot of data scientists use Jupiter notebooks another challenge is the data versioning and data sharing is is difficult in in a machine learning context sorry in in a collaboration context because you can't very easily put your data and get there is a project called get LFS but it has some significant restrictions and and so what we find is that teams normally just don't bother the data versioning and they just rename files or rename folders and have folder names with like underscore final and underscore final final v2 and all these funny little strings that refers to the things that have been done and then they have to share those folders around and it becomes quite messy the other thing is metric and parameter tracking you didn't have to do that when you're doing software but you do have to do it with machine learning you have to keep track of which parameters you used in which accuracy score you got because models aren't either sort of green or red they're not either working or broken they're kind of somewhere in the middle and the metrics like the accuracy score will tell you how how good a model is against a certain test set and so you need to keep track of that and you could put that in the in the get commit message but then you have to have human remembering to do it people about it remembering to do things or you can use tools that help you with experiment tracking and there are challenges with with using a combination of local development environments you might have a GPU or an IP you in the machine in your desk or you might be using machines in the cloud and with gifts it's quite easy to switch from a local machine to machine in the cloud or a machine in your datacenter but doing it effectively and with machine learning where you've also got data that you need to move around and you've got to keep track of metrics and parameters and maybe you can't even really use gifts with your groups and notebooks and still do effective collaboration it makes it a lot more challenging with respect kind of moving around where you're where you're doing the work and so there are some tools that that help to solve these problems for machine learning and it's actually a very exciting space and there's lots of new innovation happening around this and so obviously I'm from dot science I'd love it if you started using dot science but I also wanted to give a knowledge meant to the fact that there are lots of other tools out there and so ml flow is quite strong in the experiment track experiment tracking space weights and biases is very strong in like comparing relationships between metrics and hyper parameters DVC is a promising project in terms of doing data version control as is a project called pachyderm and then in terms of differing and merging groups and notebooks there's an open source project called MB dime in fact many of these many of these projects are open source and and so what we've tried to do with dots science is bring the capabilities from these kinds of capabilities into a single [Music]