What Is a Unit Test? | Unit Test for Data Science Code | Community Webinar

Key Takeaways

go ahead and get started then so thank you everyone for joining today my name is Niall Wilson and I'm a data and applied scientist at Microsoft industry Solutions engineering and as Nathan mentioned I'll be speaking today about writing unit tests for data science code and I'll be going into what unit tests are why they matter and what makes it so different in the data Science World versus software engineering right so a little bit about myself I come from a background of biomedical engineering in Academia I completed my PhD with a focus on brain computer interfaces so bcis back in 2019 so for the past three years or so I've been working at Microsoft as a data and applied scientists but my background in biomedical engineering isn't necessarily a direct one-to-one translation to what I do today so a little bit about this presentation is is including my journey as well coming from an academic background to Industry and hopefully they'll be relevant for some of you as well in terms of learning some new skills and understanding what best practices that you need to implement to really improve your quality of work within the within the um non-academic space so if you'd like to learn more about me feel free of course to reach through Linkedin and there's also the deafblogs.microsoft.com link for more information about me and also you can see some blog posts that written as well and I do see the Q a notification bumping up do we want to address something right now um I think we could save it for the end right yeah sounds good all right so go ahead and move on then so for today again uh if there's any questions that come up that are really blocking your understanding of the presentation please feel free to ask right away otherwise feel free to leave them at the end I'm really happy to go over any of the topics here and also just go through and discuss what might work best for everyone and so in terms of today's presentation I'll be also taking breaks here and there just a few pauses to allow for questions to come in as well but otherwise feel free to to interrupt as needed okay so before we get started I want to set the stage and make sure we all have the same basic understanding uh since some of these terms can mean very different things depending on who you're speaking to and what field you're working in so first off what is unit testing so unit testing is a way of ensuring that code that you write is working as expected so a unit refers to a small piece a small piece of code a single function with not too much going on inside of it typically you consider that a unit but when you're thinking about a full solution it's a lot of different pieces moving together and while you can also test that larger system which we may call integration testing for example testing each of those individual components is called unit testing so when it comes to software engineering and data science this is important for both situations where you want to write code that would be going out into production so if you have any sort of code that will be customer facing or up we'll be using customer facing scenarios and has things such as envelops or continuous integration or continuous deployment CI CD you'll want to make sure that you have unit tests written so when it comes to comes to the tools for this there's a few different tools that you can use when it comes to python code I tend to use Pi test as I find it easier to work with but unit test is also a python library that could be used for this all right so where does unit testing come into play what are some real world examples that we can see that really help solidify the importance of this so unit testing is applicable for pretty much any sort of data science scenario where you're working with production level code and so just some examples from my previous work there's a lot of different projects from different Industries and different machine learning domains as well but just to show the range I picked two here that are fairly different that had unit testing as a core component of the solution so first off we have a project where we're working in the media and entertainment space specifically working with order media so this is a project I got to work on where we were able to use some of the film that they're they were providing to build a solution to automate scene and take detection using digital dailies which is like the footage that you get when creating a movie that is running for a long time and then eventually you cut that down and edit it into the actual final film and this is a fairly complex solution and so unit testing was pretty important for ensuring that we're able to have every component work as expected so if you're interested in learning more about this specific problem and this specific solution feel free to visit the the blog link here aka.mslash and I believe someone is going to paste that into the chat right here on the opposite end we have something in the retail space so a recent project where I got to work on forecasting sales opportunities at scale so for a lot of different time series in parallel and so with this we are training and selecting the best from thousands of univariate Time series uh forecasting models for this one so so what specifically was the impact of unit testing for each of these so for that first example in the media and entertainment space with the solution we were able to automatically identify discardable footage and reduce the storage cost by 44 which is quite large and we did so with the Precision and recall of 96 percent and so the impact of this was fairly large so what was the role of unit testing here what was why was it so important so again when it comes to writing production mobile code it's really important that we have these unit tests to ensure that as we build complex Solutions or even simple ones every little piece of code that we're writing functions as expected we're not accidentally leaving in any sort of hard-coded values um or not considering certain edge cases this allows us to really be confident in the code that we're deploying and ensuring that we have the high quality in the final solution likewise for the retail project we're able to have high confidence in the performance of our of our solution not only from the machine learning performance standpoint but also from a code perspective this was a pretty complex solution that required using Auto ml to do the training for many models and selecting the best model as well and having these unit tests really helps solidify our confidence in being able to share this with the customer we co-developed with them but being able to ensure that everyone was really happy with the performance and also the transparency in in the entire solution itself okay so before I jump into a deep dive of how to actually write these unit tests for data science code any sort of questions regarding the applications or any of these use cases I'd say let's give it about 10 seconds because sometimes it can take a minute for people to type but but um uh there there is a question maybe we can just start with the question that was asked at the start and this person's just asking if they can learn data science in one year huh yeah I would say it depends on your background it depends on your background for sure and what you consider learning data science is uh there's a lot of different levels of proficiency and because it's such a large field that's evolving fast and is always changing the if you mean learning data science in a year is mastering absolutely everything then no one year definitely will not be enough but if you have background already within statistics or computer science or working with data and you're interested in the basics and getting started then yes you can definitely get started within a year start with building a foundation yes um all right I'm not seeing any other questions um about the what we just went over so why don't we go ahead and continue sounds good so when it comes to data science code there's a few different common scenarios or common areas that we'll see unit tests be applicable for so for data science code there's a lot of data pre-processing that usually has to happen so our data wrangling cleaning things up and formatting them into the way that we want them to be to themes for models feature engineering Etc all these functions that we're writing for data pre-processing if it's in a production setting we'll want to have unit tests for them similarly if you have any sort of helper functions to assist with formatting things for model training or formatting the results as well that's something you want to write tests for and of course anything that does that works with data in terms of evaluating it and deciding the next steps or doing whatever needs to come next should also be tested so this is pretty much the full gamut of the the data science life cycle in terms of once you have a model cited on so we'll be talking about unit tests in the context of data science code after exploration okay so before we get into the testing itself what does the code look like so as many of you are likely familiar with the code for data science can be in a few different languages today I'll be focusing on python but uh even just within that broader space there's a few different stages when it comes to writing data science code usually I'll start out in a notebook so that I'm able to quickly iterate on working with the data fastly developing things and just being able to iterate iterate pretty quickly and it's nice to have a visual immediately for what you're working with however um when it comes to the exploration phase you don't necessarily really need to write unit tests uh it can be helpful but because things are changing so much and some functions let you write or some code that you use might not necessarily end up in the final solution it's not really necessary to write unit tests at that point however once you get to what you believe will be closer to the final solution and once you're moving things into scripts then yes you'll want to start writing unit tests so first thing first is breaking the code down and Nathan I do see your video actually so feel free to introduction so so we do have a question about uh do you also do functional testing in each Dev environment oh yes so when it comes to functional testing definitely so in terms of this talk today I'll be focusing on unit tests specifically however integration testing functional testing is all very important as well so I do highly recommend uh doing that within your Productions environments and and also in depth before you deploy it to QA or to prod perfect sounds good all right uh so when it comes to the code uh first thing first is even if you're converting your code let's say from a notebook into scripts if you copy and paste things exactly is how they wore the notebook it might be a bit messy one of the key things is to break down the code into nice chunks or nice little units such that it's easier to test so when it comes to unit testing you want to make sure that the functions that you're testing are pretty modular and are not massive and doing a lot of different things another thing about the code is just personal preference and also coding best practices just to include doc strings and type hinting so ways to ensure that's easier to read your code will you come back to it and also for anyone else reading your code as well and lastly it's important to organize your code into different functions but also organizing where those functions lie is important as well so if you write any sort of utility scripts and organize your functions into those I find it easier to work with that way okay so what does a test look like so unit testing can take on a few different forms but this is the format I like to use when it comes to defining the test themselves I start out with a preface or prefix of test underscore and then the original function name and so this is within a file called test whatever your script name is dot pi and so this is a file that contains all the tests for that particular script that you're testing and with this you'll see that on the function we have a happy path appended at the end I'll typically append happy path for exception depending on if it's a test to test if the function is working as expected or if it's doing the error headline correctly when it comes to the test format I like to think think of the Three A's so we have a range or a sign act and assert where a range is the space for you to could have built up the the arguments that are going to be fed into your function so this is things like loading any necessary data mock data formatting things that's necessary going into act which is calling the actual function you're testing and then assertion which is uh or sorry which is making sure that the expected values are are what you're actually getting and yes Nathan IDC so I'm going to interrupt you so so on average what percentage of time of your coding do you spend on unit testing uh does it become a lot of the same things that you plug and play eventually that's a good question so it depends it really depends so I'll say if I'm just considering the amount of hours I'm putting into writing code itself let's say I'm writing some some basic uh small solution that has like a few files for regular Python scripts for the functionality and then a few as well for the tests I would say majority of the time for sure is developing the actual code because there's a lot of trial and error in terms that that creation and then I would say uh depending on the complexity like 20 to 30 of the time could be allocated to unit testing okay and then I had another one for you um so does that mean uh this is I'm not going to try and pronounce this person's name so apologies to this person but uh does that mean unit testing is not required if the model will not be deployed into production good question so it's depending on who you're working with things may or may not be required I would still recommend it um if you have time unit testing can be really helpful for catching just any sort of issues before they become like especially any sort of Silent issues and for example in grad school there's an issue where I I didn't really know about unit tests I definitely could have benefited from them because there was some sort of experiment that I ran and the analysis looked great I was ready to get ready to present this to the group and then I realized oh no I hard-coded something by accident and I had to you know fix things rerun them and like things were drastically different so if I wrote in a test for that code back in grad school I would have caught that before I got all excited so same thing applies within data science as well uh unit tests can be helpful for catching any sort of Silent issues but it's not absolutely required if you're not going into production setting okay perfect and then we have two more uh just two people asking for elaborations on a few things sure um the first is can you elaborate on um no it's in the chat there is can you can you please explain what you mean by different utility Scripts yep so I wish I had a screenshot here to show but essentially let's say that you have a python script or some sort of notebook you're working with and you want to import some code that you wrote let's say you have a function called you know filter like data or something like that and you define that within some sort of dot Pi file somewhere else that's not where your notebook is uh you can consider that utility script if that dot Pi file is something like generic utils.pi you import that module then that is considered your util script okay in the next one before we move we start moving on because there are more questions but I want to make sure we have enough time to get through the presentation um can you elaborate on Happy path versus exception yes uh so I will say for the sake of time today I won't go through every single slide I have available I actually do have some hidden slides that elaborate more on exceptions and so once the slide deck is sent out afterwards definitely feel free to review that but short answer is that happy path is what I like to call the test for making sure things are working as expected with input that is valid exception tests are for when the input is not valid does it does the function handle the errors as expected okay perfect and I know there have been a couple of questions about will the PowerPoint be available you just heard it from Niall herself yes okay all right now I'll let you continue and again I'll pop up when there's more questions sounds good oops all right all right so moving on from the test format then so for sample data so data science we're working with data it's the core of everything that we do and naturally when it comes to testing our code we also have to work with data however a key thing is that we don't want to use our full actual data set when it comes to running unit tests one of the key reasons is that the focus is on testing the functionality of the code that we've written and it's not necessarily about the results or the data itself another key thing is that unit tests you want to be able to run those locally so if you have a large data set let's say like a terabyte or petabyte even of data you don't want to be feeding that through every single time you're running a data unit test you'll want a really small sample and so another thing is when it comes to running tests locally a common practice is to save some of that sample data directly inside the repository that you're working in and to minimize the size of the repository you don't want to have any sort of larger than necessary sample data saved for you testing there so as you can see in the image we just want to take a small piece of the puzzle a small part of the data to to run our tests and an important thing is if the data you're using is sensitive it's private sensitive data that you shouldn't be uploading to repository in the first place then don't use a subset of that data instead use some sort of synthetic data that's similar or some sort of Open Source for your own generated data that is similar enough and meets the specific criteria but is not wooden compromise any sort of security or privacy uh rules okay so uh before I get into this I just want to State real quick that this is the core concept that I want to get through the presentation today and it's mocking this is something that I struggled with quite a bit when I started my journey for learning data science uh you know testing so I I'm trying to break it down in these presentations today to make it fairly uh straightforward to understand but if you have any questions definitely please feel free to ask them uh and I just want to actually take a moment to share a little bit about my history when it comes to when it came to learning view to testing for data science code so as I mentioned earlier my background to biomedical engineering from Academia and so at that point joining Microsoft in 2019 I had never ridden a unit test before would I joined I was tasked with learning a lot of new things getting up to speed with the industry and making sure that I learned some software engineering best practices to be able to uh to work effectively alongside software engineers and other more experienced data scientists within production level settings with uh while working with customers so one of the first things I did was I had to learn what unit testing was and at the time there really were not a lot of resources at all in fact most of the resources that were available uh were very strongly written for software Engineers or for people with a strong background in software engineering well I had a little bit of experience here and there the assumptions and the lingo just really was not conducive to what I needed to to learn and what I needed to do and so I really struggled at first and I had to reach out to a lot of different uh data scientists within the company colleagues who are more senior had more experience about unit testing and luckily through experience and with their guidance over time I was able to learn how to do this and improve my skills as well but this concept in particular is something that can be tricky and so this took me a bit of trial and error to get used to so all right enough with that and I'll go ahead and move into what this is okay so mocking is a way to simulate functions that are called within what you're testing it's also a way to simulate the output of what you're calling so let's say for example we have a function that we're working with not a test but just a function let's say we're working with format historical data frame you'll see within format historical data frame that we have another function that we're calling side of it called filter columns now on the right hand side you'll see a unit test written for this the simple paired down version and you'll see that there's also this reference to format historical data frame here now I noticed that this is a mocker.patch call and what this is doing is this is blog is to say that when we call format historical data frame instead of actually using filter columns as specified there let's actually just automatically return the return value of an empty data frame so once we actually get to the ACT section of our unit test and we call our original function format historical data frame this is what it looks like so at the top you'll see what the function looks like normally when you call it then in our test case where we do have the mocking you'll see that the line that originally called filter columns is now replaced with the return value so instead of filter columns actually being called we are mocking we're patching that we're saying okay don't actually call that let's just assume that the output is an empty data frame and so that allows our test to run for format historical data frame to instead focus on the functionality of format historical data frame rather than focusing on the functionality of the functions that are called within it okay this is a I think fairly simple example compared to how complex it can get but hopefully this makes sense to everyone if not are there any questions related to this at this point I'm not seeing any questions yet and that either means all right I guess we do have one question why is that an advantage yes good question so here we go why so uh should I Mark and understanding when and why why not to mock so um the reason that we want to do this is that whenever we write a unit test we are really making sure that it's the functionality of that particular method or that particular function that is what we're testing so uh when it comes to unit testing you want to make sure you write something for each of those functions that you write in the case of our previous example I'll go back to that real quick in this case we were writing the test before my historical data frame but we would also have a separate test for filter columns So within the test for filter columns that's when we can actually test that as expected we can make sure the output is correct things like that but when it comes to testing for my historical data frame we've already tested filter columns so there's no reason to to have that go again we can just go ahead and filter reduce the the amount of time that it takes to execute the test especially if you have more complex functions if you know that you can work with a particular output to mock and so in terms of when should you Monk you do want to mock imported Library calls if they're not small essential operations so things that can be a bit larger especially any store custom functions as well that you don't already have tests for definitely mock those and calls to external services are things you usually want to mock as well so as mentioned before unit tests should be able to run locally so if you have any sort of calls to external services that would require a connection and there's a lot of other things that could go wrong there there's different sorts of tests that you can do to ensure that your full solution Works end to end which includes connecting to different services but for testing small independent functions it's typically not necessary uh also you do not want to mock custom functions you have that don't have any tests already so write those tests and also any sort of basic necessary operations so let's say within your function you need to read in a data frame or you need to convert something into an array these basic things there's no need to mock them because they are very basic operations that need to happen for the rest of the function to to actually work as expected right um before I go to further any other questions on unlocking I was gonna say I was gonna give it a second before I said no because because of last time but I'm I'm still not seeing any pop-up so I think you're good to go okay sounds good and again if anyone has any questions at the end definitely feel free to ask okay so uh this next thing parameterization is something that I felt really helped me step up my unit testing game it's something that allows us to cleanly test multiple sets of input for any of the tests that you write so instead of writing let's say five different tests for the same function with different uh different edge cases and different types of inputs or different categories of input this allows us to use that same test code and just stay at the top these are the different sets of input that I want to run through it to test and so in the in the case here you can see we have that pytest.mark.prometerize and then that first line afterwards is just stating what are the arguments that were going to be um represented within those sets below so in this case we have a path to a forecasted data frame as well as a kpi value and in each of those sets are parentheses below our top levels we can see that we have a path and then some sort of kpi value so each of these sets of parentheses are being fed into the test so this test actually runs three times and I feel that this is really important when it comes to handling edge cases as well so this just has three examples but sometimes I'll have tests where the parameterize section is quite extensive so I'm making sure I cover a lot of different ground um to make sure that the test runs as expected so uh that's also really helpful for when let's say I want to improve a function or make some changes to it uh having all those different sets of input make sure that when I run the test again after making changes that it clearly goes through all different sorts of options and made the fiducio yeah and this kind of goes back to mocking um so this person wants to know how would you do Mocking when the actual data matters for the functionality you are testing and there's a there's a second part too but we'll start with that one no problem so when it comes to mocking and when it comes to the data itself there are definitely times where you have those functions where the intermediate functions will process that data and you'll need to rely on that process data to actually continue through the actual main function that you're working with so in that case you just want to make sure that you're providing mock data that is appropriate so in the example that I provided we had a return value of an empty data frame and that was just very simple whatever worked for that specific instance but there are cases where you'll want to say instead that your return value is let's say uh some specific array that you know is going to be representative of data you would have at that step so that's an example of mocking data as well you can save that as like a numpy file or CSV file for example save that within the test data section of your testing folder and then load that in during your unit test to ensure that you can feed that into mock data okay and then um and you may have already answered this so let me know if you did but how can you mock for a non-generic function that depends on the data if that makes sense got it so let me go back actually just a little bit okay so here we go on the right hand side again this is showing that mocker.patch function when it comes to mocking a specific function you don't need to worry about the input into that function you're mocking so in this case filter column should take into arguments we have historical data frame and some sort of config note that when we're making that mocker.patch there's no reference whatsoever to those input arguments so whatever goes into that function doesn't matter since we're actually not going to be calling it itself it's more about what is returned all right sounds good and then we also have a question about uh parameter the next slide yes from the parameterization is there a way to automate the the parameterize function as an auto gen as in Auto generating the test data to be used in the function oh that's a good question um I haven't tried it there might be a way to do it but I honestly don't know but it's worth looking at you because I feel that could be helpful like setting a range of values for example okay that's it we can go forward awesome okay uh so I know there's a few other topics I definitely could have covered but in terms of sake of time just wanted to cover these Core Concepts when it comes to unit testing for data science code so uh first off just reiterating that unit testing is a must-have for production level Solutions so for anything that is going to be customer facing anything running continuously or running through large amounts of data are really important you want to make sure that you have unit test written for your code that helps grow code quality and also helps you catch mistakes early on it also helps you think of better ways to improve the code I was actually writing unit tests this morning for something and realized that there was a functionality that was missing and so while writing tests I actually edited the the function itself improved it and added that into the pull request as well so really key thing here is that data scientists and anyone here should feel empowered to write unit tests it might seem a bit daunting at first but there's definitely more basis of resources available now to help out with that so this presentation is just one small example I do have a resources slide here where I believe the links will be posted instead of chat and also available in the slide deck but there are different articles out there now example repositories and different documentation so I highly recommend giving these things a read and also experimenting yourself with writing these unit tests awesome thank you Nile um and and we got a question asking can we get a link to these test codes and readme file as steps on how to perform such testing um yes they'll be in the resources uh the link was just posted on Zoom if um if you want to click there and also when we post the recording on our website as well as YouTube channel we'll have all these links listed out as resources so you will have access to them um if if you're on Zoom that means we have your email address and we will also email them to you so so don't worry about missing out on them now you will have access to them once the recording is posted as well um okay so now we do have some extra questions of course so um we're going to go through them one by one um so this question was asked more towards the beginning of the presentation uh and this is from I think I pronounced her name Sharif so does that mean tdd won't work with data science projects so I was the acronym again t d d I am not sure what the acronym stands for I am not either if uh if this person could reiterate then it might be indicated test driven development ah okay okay I would have to look into the specific definition of it but if it's something for example saying that you need to develop tests with everything as you go then I think that could be possible with data science code but it it's not standard within the workflow since exploration is such a huge part of data science um unit testing typically comes in after the expiration phase okay perfect and it might have been just you and me not knowing what test driven development was it seems like everybody in the chat knows what it is so thank you all um uh so uh next question is which role does or do which role does the unit test work like is it a data engineer is it a data scientist is it the person who develops the code um who is it yep so it depends it depends on what kind of team you're working with and what environment but it in my experience typically the person who wrote the code would write the unit test which would be the data scientist unless uh you intentionally want to do a knowledge share and help other people gain a better sense of ownership over the code as well so let's say you're co-developing something with a customer and let's say I wrote the initial functions but I wanted the customer to feel more comfortable and have a deep understanding of the code itself in that case I might ask them to write the unit test such that they're really forced to look through the code and understand exactly what's going on okay and we have a little bit of a clarification on test driven development test driven development is when you write the test first then and write the code to pass the test oh okay I don't know if that changes things but that's what it is I feel like you definitely could do it but I think again since when it comes to data science life cycles typically you start with the expiration phase and then you will kind of bring things to production typically the code you're writing that will go into production has its Origins from the exploration phase uh so in that case you're already starting with existing code rather than starting with the test okay um okay next question uh this person says thanks and sorry for interrupting again don't worry about it do you have a software engineering guide or best practices to follow in an industry based data science project yes so we go back to this list of resources real quick so Resource number three the code with engineering Playbook I would say is pretty valuable for understanding how to write production level code especially in a customer facing setting but even just for yourself internally as well there's a lot of good resources there and it's written with having software engineers and data scientists and program managers in mind so the machine learning folder specifically has a lot of machine learning best practices and outside of that above it there's also different resources for software engineering perfect and again just a reminder these will be posted with the recording so don't worry about having to trying to type them out or copy them down there'll be plenty of opportunities to to grab these um next question is from Jack so how does a release pipeline for data science differ from that of a software pipeline does the environment differ significantly to accommodate for mocking Etc good question uh so the environment doesn't have to be significantly different to enable mocking as long as you have some sort of environment set up where you can run your code let's say for example you have a build pipeline for your repository just to make sure that whenever a pull request is created or merged into the main branch that there's a certain series of checks that are made so you can include the unit testing as part of that that can still use the same environment as all the other testing as well the only difference is you'll want to make sure you have the data available somewhere for testing and again not your full data set but just whatever you're using for testing so if you have a folder in a repo called test data that's all you'll need okay perfect and um uh this person uh John maybe you joined late or maybe I just missed something but um the presentation seemed to cover most of the things you would do for writing unit tests in general software engineering are there any so outside of software what we would do for software engineering are there any special considerations for data science code got it uh so specifically for data science code I think the key thing is that sample data portion um when it comes to things like trying to test functionality of a particular model training um again I would recommend mocking things that are from imported libraries so if you're importing something from I don't know for pie torch or something like that that kind of code typically already has its own tests so you don't need to read reinvent the wheel and test those as well so in those cases I recommend mocking those calls and returning test data that's appropriate for your situation okay and then last question for now unless more pop up how frequently should should someone run unit tests cases on production code yep good question so again that depends on the specific solution was seated for the business value but typically you'll want to run unit tests any sort of time that the code has changed in any sort of way um so if you're creating a pull request for example into your repository you'll definitely want all the unit tests to pass before that PR can be merged perfect sounds good we do have a couple of more questions pop up and I'm going to let people continue writing I'm actually going to steal the screen from you for a second I want to tell people about our code or not our code but our our event for next week sure um so let me just grab make sure I'm grabbing the right screen here um there we go all right so uh Nyle I'm hoping that we are seeing a crash course on data wrangling using SQL um perfect so so next week December 7th at 12 p.m Pacific so the same time as today um we'll be doing a crash course on data wrangling using SQL and this is going to be presented by cell Selma dodgic she's a manager of analytics Coe and strategy at Carter's um but in this session if you are new to SQL you're a beginner you're trying to learn this is probably a session that you want to be a part of um we're gonna or Selma's gonna present it from the expectation that you know people have little to no experience in SQL um and by the end of the session you're going to know how to query data a data set you're going to do some joins append data apply filters to your data sets and create new uh data fields so again if you are new to SQL and SQL is something that especially at data science dojo we tell all of our data scientists if you don't know SQL we're going to help we're going to teach you teach you it because you need to know it um if you're new to SQL definitely something I think you should think about joining you can RSVP on our website by just going to um uh data science shelter.com events and finding uh clicking on the event or you can also join us live on LinkedIn Facebook uh Twitter or YouTube all right and um so that's all I had so now we can we can there's a few more questions popped up while I was talking so um so we can go ahead and answer those now uh one person is asking if there was a recording and it's okay to join late yes there's going to be a recording we'll send you an email or you can find it on data science dojo's YouTube channel as well as our our website um and it'll also include all of the resources that Nile has has shared with us today um all right let's get through some more questions um is there a list or um of a walk through like a to-do list um you know step by step on um or uh on unit testing or doc string for data science to production okay check out the what I listed the blog post at the very top I think is helpful at explaining that but just quick answer those three a study mentioned the assign act and assert I think that's your basic skeleton that you want to go with when writing data science uh tests or just unit tests in general okay um and then how do you handle the scenarios where you get new features as input into the pipeline and how is the model training handled in that case especially if the existing model doesn't support continual training hmm okay so in this case I feel like it's typically necessary to have some sort of pre-processing functions or helper functions to bring that data into the model in the first place and that's where you can put in those checks okay and then uh this would actually probably a good question to and on end on by Gordon on projects you have worked on how have you balanced time spent on unit tests versus model iteration or exploratory analysis and do you run into barriers with leadership or management for not prioritizing unit tests development yep good question so it depends on the stage of the project luckily with my work we have distinct phases uh so we have the feasibility study phase or exploratory like data analysis Workshop area so like that period unit testing is not really in the picture and I can focus on the exploration however once we get to the portion where we're working with a lot of Engineers or breaking things to um you know into pipelines and production level that's when unit testing is pretty important so luckily our leadership's pretty understanding they recognize the significance of unit testing uh so I haven't run into any issues regarding making sure I have enough time to develop those okay perfect and now that is all the questions we have today so thank you so much for being here um for everyone that joined thank you again uh