Code Quality in Data Science // Laszlo Sragner // MLOps Meetup #105

Key Takeaways

hi everyone thank you for coming um so today we've got two amazing speakers uh first we'll start with last law who will talk to us about good practices around coding in data science uh a bit random class for last law for the first ever speaker at envelopes community bristol thank you [Applause] thank you very much for the kind words and uh thank you for uh the attendees to come to my presentation um i would like to start my presentation with a quote from 18th century uh prussian general and military strategist font clause machine learning is the continuation of software engineering by other means you're going to see in this presentation that we can borrow a lot of ideas from software engineers and applied to machine learning to make our life easier the title of my talk is clean architecture how to structure your ml projects to reduce technical debt first we are going to see how technical that ruins your productivity in machine learning uh why machine learning products are particularly prone to gather technical debt and we are going to go through a couple of software engineering principles that can help you to mitigate this like decoupling refactoring or design patterns about me i have a background in finance i was an administrative mobile gaming i was head of data science at the london fintech where we specialize in industrial scale natural language processing for investment banks most of the content of this presentation is coming from the experience we gathered during these projects and please check out my blog laszlo.substance.com first of all it's important to understand of what do we mean by ml products ml products are not machine learning envelopes infrastructure envelopes infrastructure is a capability or a feature and a tool what you are using to create ml products and it is also not data analysis even as data analyzes in advanced analytics uses machine learning technique to solve the problems usually data analyzes have a beginning and an end essentially when you are providing the insights to the stakeholders if the time period is too short to talk about the product um ml products or complex business processes which are described by end-to-end input output relationships rather than rules they work autonomously so they don't require substantial amount of manual activity and usually has a long lifespan and that's exactly this long lifespan which makes them removing technical debt a high added value activity because you are going to have a long period of time uh when you are yielding from these activities why does technical debt matter in ml products um you're solving fuzzy real world uh business problems there is a lot of like edge cases and ad hoc rules rules of thumbs what you try to solve that makes machine learning products inherently unspecifiable so it means that no matter how much specification you are doing in advance you are going to find features and aspects of the problem that you haven't thought about even when you are in product production and there is another problem which is machine learning products and the business problems what you are solving are changing while you are solving them it means that this is always a catch-up problem it means that you by definition need to prepare that you are going to need to change ml products let's take a look at ml lifecycle and how ml lifecycle impacts the creation of technical debt in ml products as we understand now there is five major steps in machine learning first you have eda which deals with data problems then you have a poc which deals with the modeling problem the mvp deals with productionization and the a b testing deals with fine tuning while we think about business as usual as a part of monitoring what's the problem here if you want to change often this is not even done by the same team the first two steps eda and poc is done by data scientists who then in turn hand their results to machine learning engineers who deal with this in the other three steps if you try to get a feedback which is essentially coming to the data scientist that they need to change the product they first need to analyze with how to change the the data definition and the poc and hand this knowledge back to the machine learning engineers who in turn again implement that and this is what's called a friction because the communication between these people need to be synchronized if this communication is uh slower than instant it means that there is a chance that you are going to accumulate changes in one go and start handing it over in a package soon it's going to turn it into a versioning which is so soon going to turn into a business critical problem because you're trying to create a large change and suddenly you become a waterfall uh place often what we see in our practice that instead of actually updating a machine learning product the team just opts to scratch it and write a new one instead of that which is not a particularly a ci cd solution what's the difference between technical debt and technical mass it's important to specify that because often a lot of things are claimed to be technical that which are not actually are lack of specification documentation and infrastructure is not technical that you are just not finished with your work yet missing features are just missing features and broken models which are particularly prone in machine learning are just broken model you need to retrain them or create a new one instead of them and most importantly technical debt is not bad practices uh you are performing bad practices uh that just reckless uh behavior so what is technical that technical that is an attempt to gain knowledge with a plan to correct it later it's a very important concept that it is actually you are taking a step in an unknown direction with the knowledge of how do you get back to the place where you want to right and i think it is important to stick to the original definition of technical debt which is incorporate that so you're essentially investing in something that is going to have a higher yield than the debt itself if you are uh thinking of credit card debt that's an incorrect uh analogy or metaphor uh because you know buying a new tv is not going to give you too much yield right uh ward cunningham so the person who coined the term technical said said that paying back technical that depends on writing code clean enough to be refactored and you can ask what refactoring means refactoring is means changing the code without changing its behavior you can think of why would you change the code without changing its behavior we know that there are multiple way to solve every problem in programming there is no reason that because you implemented it once that's the best way to solve a problem right so refactoring allows you to once you solve the problem turn it into something that enables you to solve future problems and this is exactly the purpose of refactoring it requires two things first it requires fast testing and the second it requires what's called experimental operational symmetry or eos for short let's take a look at this what is eos it's essentially google's fancy name for running the same code everywhere where do you run the same code you're running it on your laptop and you're writing it you're writing it using it on testing um evaluation of your models training of your models and then eventually in production and eos requires you that these four five places are running the same code we are going to see how to um achieve that um eos enables refactoring but it also requires the coupling let's take a look at what decoupling it means decoupling is reducing the dependency between components of your system why would you do that it allows you to decompose a problem into smaller pieces and it allows you to hide implementation details of external peripheria which is not relevant for your ml products as you remember ml products are usually high-level business processes and automations they should be less concerned about implementation details around databases or external peripheria and it has the additional value to breaking what's called cake changing anything changes everything this is particularly prone in machine learning that helps you to detach from that from a software engineering perspective it doesn't help you from a from a machine learning or a mathematical perspective but that's a higher level problem that you can solve in a different place let's see how def decoupling can be solved in a machine learning situation let's imagine that your code is the blue box defined as a class and we need to be familiar with the two related concept which is called inversion of control and dependency injection in a traditional setting your program is calling the infrastructure that's called direct control you know you are going through and then uh let's say you do load the database then you are delegating your code for training on a gpu and then you get the results and then do something with it through an api in a new version of control the infrastructure calls the main entry point of your class it's instantiate your product as a class and then cause the main entry point to do its business and dependency injection allows your code which is an abstract python class to connect to external peripheria and solve get the information from it or put the information out there it is not aware of where is it going it just understands that it is need to have an interface and connect to this interface this allows you to create uh decoupling from these interfaces but still maintain eos because in a test setup for example you can imagine that your test harness calls the main entry point of your code and the injection of the external peripheria through files the file system or mocks devices where you can you can see how your uh how your code behaves and it doesn't behave in the correct way so let's see how to structure the blue box you know your code as a product uh it's very important that in a clean architecture your main focus is on the center in the business process that you are trying to implement which is consist of a domain model which is essentially classes that describe what you are solving business logic essentially extending uh the business process into your code that is uh you are implementing and the models that are automating these problems right that is at the center this had the additional benefits of most of your attention is at the center of this code you are going to write most of the code here you're going to do most of the refactoring here while you are testing your code and it allows you to make this the cleanest part and this is surrounded by the classic machine learning features like data sets feature engineering and other parts the outer layer which is treated as an external resource for this product or the databases uh the computing infrastructure and external services if you are creating this plug-in architecture with dependency injection that has the benefits of outsourcing these components to other teams and therefore it helps you to um build like a distribution of responsibilities uh they can you can incorporate other people's help into your product in a seamless way this is what's happening in testing you can see that the middle of the of the of your structure so the blue box what you can see on the on the previous uh slides are are not changing you are just replacing the external environment when you are putting it in testing obviously you can create another environment and put it into evaluation and so on but the middle the business logic is the same no matter where you're running it we are going to look at three useful design patterns that are going to help you in implementing this which is called the adapter pattern which is allows you to decouple from external services the factory pattern that allows you to decouple from data sources and the strategy pattern that allows you to decouple from algorithmic decisions usually you are experimenting with you would like to try new ideas but you don't want to destroy your code and strategy pattern is the best way to solve this problem so the first one is an adapter pattern as you can see it is already reflected refactored our code into a class the infrastructure on line 17 is playing played by a shell script which is instantiating the main class and then calls the main entry point which is the run function the run function in turn because creates a an engine connection a database engine and then runs a simple query to get a data frame uh what's the problem with this if you take this class and instantiate in your only machine and you don't have access to this database this code can never run all right so it means that this is you cannot achieve eos through this because you can't run a test so you can't do refactoring so what do you do essentially the simple idea in an adapter pattern is to wrap the code that is dependent uh on an external resource and move it into its own class which is here called sql source and the main code you can re see that i removed the database specific part and replace it with an interface call from now on your main code just knows one thing that it has a source uh that needs to call um through an interface and it's going to get a data frame from it and you can see the the infrastructure now instantiates the main code and this database connection if you want to run it on your own machine you only need to rewrite that line place another class which gets the data from a data frame a file and move on right so the next one is a factory pattern usually when you are writing your own product you would like to have all of the code to be domain specific terminology for example if you are dealing with customers you know you would like to have a customer class you don't want to have a data frame which is connecting various bits of data together having a domain-specific term is a good idea and you can see that i refactored here that the source class now returns a list of domain-specific classes that you can use as the native language of the business process and the business problem what you are solving uh later so essentially a factory pattern a factory is just a class that creates other classes all right and the last one is a strategy pattern which is particularly prone in machine learning essentially you can see here that the customer class is turned into some set of features and that set of features added to a model and the model creates a prediction the problem here is if you would like to try a different model you need to rewrite this class which is not great you can imagine that this is a simple example but if it's hundreds of lines you would have a problem of finding where is exactly your model is so you can wrap all of your problems and then and then the same way as you did this with the adapter pattern you can uh someone said uh you know one of our lectures uh you treat uh the algorithm as an external resource to your your code and you can see that here the model is instantiated in line 19 and the same way it's called the predict function passed a domain specific class and get the prediction back but the main function doesn't know what kind of model it was running it has an other good aspect which is called coherence which is essentially let's imagine that your model is a deep model which requires uh hum to decide how many layers it have this these layers need to be passed on to the the main class uh in if you don't have this refactored um and this need to be stored and the this data needs to be passed on to uh the primary function when the model is instantiated obviously if you would like to replace it let's say from a deep model into a random forest which doesn't have layers it has trees you will need to change your code multiple ways and the main the more places you change the larger the chance that you are going to make a mistake so what you can do obviously you can pass the layers to the model if you have refactored it into its own class and now the main function has no knowledge of the the feature and your code is much cleaner uh if you want to write a different type of model like a tree then you just need to write a different class and plug it in in line 19 and you solve the problem so what's the workflow from scratch when you are approaching a problem through these paradigms first you define your data classes you know this is usually part of the analyzing the business process that you want to solve with uh machine learning and the business logic then you wrap any kind of data source with factories that are creating these data classes for you and then in the future you operate on this so set up a test environment usually what we do is just run the same pipeline on one percent of the data this is good enough to pick up if you create a typos or any kind of like stylistic errors and and the majority of the problem appear to be here you define your interfaces any kind of verbs that you want to do with this data what you want to create these and then group these into service classes if you want to move towards like a service oriented architecture or microservices then these service class is going to be the services eventually then you structure these through dependency injections as you see the coherence before you you create yourself opportunities uh to uh to enable yourself future experiments then you wrap these external services in production that you want to achieve when you move your into production with the adapter pattern and then you can set up a production environment and deploy the same code that you are running in tests to production as well uh once it is running and you have new ideas of what you would like to do you can set these up through the strategy pattern without actually trying changing the production code and you can run experiments of course a lot of people works in notebooks and how do you incorporate this workflow when you are doing your analyze in notebooks we load the data classes with factories the factories are classes you can instantiate them in the notebooks get the data turn the data into data frames through dictionaries and do any kind of analysis and plotting in in notebooks and because your main program is a class you can instantiate it in your notebook and pretend that you are in production with the right plugins and you can create experiments through implementing strategy patterns ad hoc in that location right so what's the benefits we recognize that the biggest benefit here is it has good layers of abstraction uh you have a place to do everything so you have a place to do analyzes in notebooks you write your code in your ide your product code and then you can externalize infrastructure and outsource it to other teams you have a high level of flexibility you can do experiments without touching the production code and you have a large amount of optionality and you have what we call managed complexity often you have complex problems uh that cannot be solved with simpler solutions you don't want to have simple solutions what you want is manage the complexity of your existing solution and this allows you to to to create uh difficult solutions or or complex solutions and still maintain uh coherence and consistency and your sanity and the lastly it allows you to have high speed iteration because your code is running on your laptop and in test environment and in in evaluation you can change it and then you can if you can run it a test and it passes the test you can immediately push it into production and this is coming from the os take away so we see that amali is inherently unspecifiable tech debt is inevitable um only refactoring can remove technical debt uh no refactoring without testing and and the eos and eos needs decoupling decoupled through clean architecture that's one way to solve that and you can use ioc so inversion of control and dependency injection to solve that and the three simple patterns adapter and factory and strategy are the ones uh that you only need to learn obviously there are other ones as well but this is a 30 minute presentation so i try to go through as many content as possible and please check out my blog on laszlo.substance.com and thank you for uh for your attention and we can go to questions [Applause] [Music]