100% CODE COVERAGE - Think You're Done? Think AGAIN.☝

Key Takeaways

today i'm going to be talking about unit testing and coverage in python unit tests help you to keep your code stable and bug free but only if you write them well there are lots of different tests you can do you can do integration tests user interface tests acceptance tests regression test and so on and so on for simplicity today i'm only going to look at unit test and coverage coverage is basically a measure for how much of your code is tested and is generally expressed as a percentage unit test setups can actually become really complicated really fast especially have to deal with complicated test fixtures and mock and stop objects test fixtures basically give you an environment or context in which your code runs for example it could be a fake data set that you use for testing your code or it can be mock or stop objects that help you single out a particular piece of code for example if you have a function that enters a new user into the database and it also sends that same user a welcome email then you might want to write a mock email sending service so that you can test the code without having to send the user an email all the time it's very well possible that the code you will write for a unit test will actually be more extensive than the actual application code that's completely normal but it also shows you that if you want to set up unit tests well it's a really involved process there are different ways in which you can incorporate unit tests in your software development process you could either write your code first and then add the unit test to test it or the other way around if you do that then it's also called test driven development i'm going to show you an example of both of these approaches in this video in order to explain how to add unit tests to your code i'm going to use an example that i've introduced in the first video in this series if you want to watch that video first there's a link to it in the description below finally i'm also going to show you two huge mistakes that a lot of developers make when they write their unit test and interpret their coverage reports and you're going to save yourself a lot of trouble by not doing these things in the first place this is an example that's based on the example i used also in my cohesion and coupling video i slightly modified it to show a bit better what you can do with unit test but the idea is the same so we have a system that stores information about vehicles in this case there is a class vehicle info that has a brand name for each vehicle whether the vehicle is electric and the catalog price of the vehicle and then you can create these vehicles here as an object and then do things with them now vehicle info has two methods here it has a compute tax method and it has a can lease method compute tax basically determines what amount of tax you have to pay if you buy this car and there's a parameter tax exemption amount so there is an amount below which you don't have to pay any tax inside the method there are a few checks like tax exemption amount should obviously never be less than zero if you provide a parameter that's less than zero it raises the value error and then depending on whether car is electric or not it determines what the right tax percentage is and then it computes the actual tax that you should pay then there's a second method called can lease that's not implemented yet there the idea is that you can only lease a car if the catalog price of that car is not more than 70 percent of your year income as an example i just created the vehicle here and printed out the tax you had to pay for that vehicle and if i run that you get this as a result currently this application doesn't have any unit test so let's add a few to increase the stability of this code i'm gonna use the unit test built in library there are other options as well such as pi test which i kind of prefer because it understands the built-in assert syntax of python unit test provides a number of different assert functions that you can use and it's pretty useful as well so i'm just going to use it here in this example the first thing i'm going to do is create a separate file where i'm going to put the test in it's good practice to put your test code not in the same place as your actual application code just looks much cleaner if that's not mixed up you could even create a separate folder structure for your test that is separate from your application folder structure now in order to get started with the unit test platform we first need to import it there we have it and then let's create a class that's going to contain our union tests and for unit tests to know that this is a class that contains tests we need to inherit from the unit test test case class for now let's just keep this class empty and then you need to add a line to actually run the unit test code i'm just going to remove this code because we don't need it anymore the tests are going to do that for us and then let's go back into vehicle info test and import that class here as a next step let's run this unit test even though we don't have any tests yet and i'm gonna use a library for this called coverage.bi and i'll show you in a bit why that's useful in order to run coverage on this unit test file i'm gonna need to enter this command so as you can see it has run zero test which is what we're expecting but you can also see that it has generated a coverage report so we can actually not read it here but what you can do is actually ask coverage to create an html report for you and now you see that there is a separate html coverage folder and if you open the index file inside the folder and this is what you see the test file itself is not that interesting but let's look at vehicle info so what we're seeing is an overview of what parts of the code have been run what has been accessed by the unit test and what hasn't so obviously we didn't add any unit tests so basically only the definitions have been imported but the actual bodies of these methods have not been run at all so they haven't been tested and the coverage percentage gives you information about how much of the actual code that you've written you have actually tested so in this case that's 41 of the code and that's just the definitions that we put in there so as a next step let's add a few unit tests to actually test this code let's start with the compute text method and then it's important to look at the paths that exist in the code so we have an if statement here that sometimes raises the value error depending on whether it's a lecture get a different tax percentage and then we finally have the actual computation itself so as a first step let's see if we can add a test for a non-electric car so in order to do that we add a method to this class and inside the test method we create a non-electric vehicle in this case and then we're going to check that the text that's computed actually matches with what we expect so i'm just giving it some nice and handy catalog price so we can easily compute the tax value a unit test has a number of assert functions to check all kind of aspects about your code so you can see here we have cert almost equal that means it's equal with in a certain error margin you can check whether some kind of result is false or greater than something else whether it's in a list whether it's an instance of some type so for now let's just add an assert equal and we're going to check that the tax that we compute for this vehicle so that's v dot compute tax equals 5 because a non-electric vehicle should have a tax percentage of five percent and five percent of 10 000 is 500. so this is our one single test method and let's run the coverage test again and we see now it says it ran one test and if we generate the html and we look again at the coverage we see hey it's become 82 and that's because it's now tested all these lines of code as well and you see that there are some cases that the test has not handled like this error has not been raised so this part of the code is never reached by the test and also since we only did a test for a non-electric car the body of this if statement also never executed and of course still the can lease method implementation is missing now we need to do is to extend our unit test so that we're also covering these particular cases here so let's do that as first step let's also test an electric car and since it's an electric car we now expect that it is 200 instead of 500 and let's also add a test that verifies that we take this tax exemption amount into account so if we have an exemption amount of five thousand dollars then the tax should be half of this and finally let's add a test that checks whether this value error is raised as expected in order to do that i'm going to use the assert raises function assert raises verifies whether a function call actually raises an error in order to do that we need to give it the type of error that we expect so that's a value error in this case we need to provide the function that we're going to call so that's compute tax and we're going to provide the parameter to that function which is in this case let's check that a negative tax exemption actually raises an error so there we have our four different tests for the computex method and let's run this code again and see what happens so running this we get four tests all test paths that's what the okay means and then let's generate the html and let's look again at the result and we see that now we have at least of the compute text method a hundred percent coverage all code in this method is being run and tested overall it's not yet 100 because we have not yet tested can lease but i'm going to do something different with that in a few minutes i'm going to tease you a little bit actually compute tax contains a pretty major bug you can pause the video if you like try to figure out yourself i'm first gonna cover the implementation of the can lease method but then after that in a few minutes i'm gonna reveal what that bug is and how you can solve it so in the compute text method we actually have the implementation and then we added unit tests to check whether it's a valid implementation the other way around is that you do not write the implementation yet but you start with the test that's also called test driven development writing those tests before actually writing the implementation of the function allows you to define what you expect of the function then test it while you're writing the function so that in the end you can get at a situation where all your unit tests pass and you are sure that the function adheres to the specifications in this case ken lee should check whether the catalog price is not more than 70 of your year income and it should also check that your income should be at least zero so we can add tests for that before we start actually implementing the methods so let's add a few tests that check for this first let's write a test to check that low year incomes indeed results in you not being able to lease this very expensive bmw so creating the vehicle info object again and then i'm just going to check that that the expression which is can lease with let's say a year income of 5000 should result in false and here you see i use the assert false function you could also say assert equals and then compare to false but this is also possible and let's also add a function that checks the case where you can actually lease the card so we have these two variants and then we have the final check that verifies that your year income should always be larger than zero and here i'm using the assert raises function again so we expect a function to raise a value error if we call candles with a negative number let's run on the tests so we see we have failures let's create the html report and you see we're currently at coverage of 100 but obviously tests have failed so now let's add the implementation of the method to verify that the test path as a first step let's simply add the computation of whether you can lease the car so catalog price should be less or equal than 0.7 times the year income so for now we're on the test again we see that we instead of two fails we have only one failure because it computes correctly whether you can lease the car but still it doesn't handle the negative year income so let's add that if statement as well and now you see we run the test and we get them okay again so let's create the html report once more and let's take a look so now we see the implementation of the can lease method is here and our coverage is 100 now i told you there was a bug in the compute tax method so i'm going to reveal that to you now but first if you're enjoying this video please give it a like the bug is that tax exemption amount can actually be higher than the catalog price and then the compute tax method is going to return a negative tax value if your system doesn't handle this properly then it's going to be very costly bug because you're going to pay out a lot of money to people who fill in a very high tax exemption amounts so we need to fix that problem and of course what the method should do is if the tax exemption amount is higher than the catalog price tax should simply be zero so let's first add a test for that and the test is that if we have a vehicle that costs ten thousand dollars then with assert equal we can check that compute tax with a higher exemption amount should result in zero let's run the test here so you see that this test is actually failing because we're getting a negative tax amount and then in order to fix that obviously we have to do something here so what you can do is actually take this difference which determines the tax amount and then we use the max value like so and now let's run this test again and now our test passes again if you figured out what the bug was all by yourself before i revealed it to you then you are now officially approved by the software development elders just kidding there are no software development elders they don't exist they're not real i never mentioned the software development elders the point i'd like you to take home from this and this is really important is that a coverage of 100 doesn't necessarily mean that your code is block free so here you saw an example where we had a coverage of 100 but it was still a pretty major bug in the software so coverage is very useful it helps you determine how complete your tests are but it doesn't tell you necessarily anything about how bug-free your tests are and this is a really important distinction to make a lot of developers make the mistakes where they think oh hey i reached coverage 100 i'm done no you're not writing tests is hard and you need to make sure that you're covering all the cases not just to reach coverage 100 but also that you make sure that the cases that may result in errors or problems are actually covered by a unit test now second thing that i see some developers do is that they use random data in their unit test so if you look again at this code for example instead of simply computing tax for a non-electric vehicle of a value of 10 000 you could randomly generate a number between 0 and 10 000 and then actually compute it here and verify that it works and then do that a couple of thousand times or something like that that's not a good idea you want your unit test to be completely 100 deterministic if your unit tests are non-deterministic so they contain random data that changes depending on when you run the unit test this is really problematic for example if you're using unit tests as a part of your deployment strategy and you're testing your code before you're actually deploying it then you can have a situation where sometimes the code works and passes the test and sometimes the tests fail and you have no idea why because you're generating random data so you can't figure out what the problem is make your unit test 100 deterministic otherwise you end up with really big problems to recap first don't assume that a coverage percentage 100 means your code is bug free it's not and second make sure your unit tests are completely deterministic never use randomized data always use the same data set so that you know exactly what to expect as an output of your unit test now the nice thing about unit tests that next to code stability they also force you to write code that's actually easy to test and overall if you have code with strong cohesion so functions that have only a single responsibility at low coupling so less dependencies between the different parts of your application that's also code that's a lot easier to test if you want to learn more about how to write code with strong cohesion and low coupling check out this video here you can find the code that i worked on in this video in the description below there's a github report story that contains everything you need if you want to watch more of my videos don't forget to subscribe thanks for watching take care and see you in the next video [Music] you