Lecture 12: List Comprehension, Functions as Objects, Testing, and Debugging

Key Takeaways

okay let's get started with today's lecture um it's going to be more of a a chill lecture than what we've done in the past even though we've got quite a few things to cover as you can see from this title slide um I'm not going to go super duper fast so please feel free to ask lots of questions and then the second half of the lecture will be really chill because we're going to be talking about testing and debugging strategies so super highlevel uh topic but first we're going to tie up some loose ends related to lists and related to functions so we're not going to introduce uh a lot of new syntax um these ideas are sort of more optional in your day-to-day coding but they're just really really nice to know so let's first start talking about this idea of a list comprehension so uh you've been writing functions that deal with lists and one really common pattern that I hope you've seen so far is the following so this code right here shows something that we've definitely coded together and you've definitely coded in the finger exercises and the quizzes um and so it is a really common pattern so the idea here is you have a function that creates a new list where the elements of this new list are a function of the input list okay so the pattern here is we create a new empty list inside the function we have a loop over every element in the input and to each one of these elements in the input we apply the same function so in this particular case we're taking that element and squaring it and each one of these elements were appending to this new list originally empty until we've reached uh we've done this function to every element in L and then we return this newly created list okay so since this is a really common thing that programmers do python allows you to do this exact functionality with one line of code and the way we do this is using something called the list comprehension so the way that we do a list comprehension essentially taking these four lines of code from this function we are are going to write it write them in this one line of code that looks something like this right so the idea here is with this one line of code we're going to create a new list we're going to have an iterator that goes through some sort of sequence of values and we're going to apply the same function to every one of those elements and the other optional piece that we can add inside this list comprehension is to only apply that function if some condition holds okay so let let look at this uh uh how let's look at this example and see how we can convert this these four lines of code to one line of list comprehension code so we've got creating a new empty list this is going to uh tell python to create a new empty list for us okay so just open and close square brackets and Within These open and close square brackets we're going to write a on line expression okay and this one liner is going to encapsulate these uh these two lines of code here so the expression sorry the function we're going to apply to every element in L is going to be taking that element and squaring it so on the right hand side here in the list comprehension we've got some e squared well what is e well it's going to be every element e in L okay so if we read this in English we basically say say uh L new is going to contain elements e^ s for e and l right so it sounds weird but it kind of makes sense even if we read it in English and behind the scenes python will take uh one by one each element in L Square it and that's the sequence of elements it will populate uh this L new with okay now what if we add a condition to that so let's say we want to create this new list of elements only only for even elements so we only want to square the even elements within my original list L well if we were to write a function that does that we have to add this extra condition here so everything else is the same except for this if e% 2 equal Z this tells python to only grab elements that are even right divisible by two so how do we write this in list comprehension form so here's a new list and this is the function to apply only if the test is true in list comprehension this is my new list I've got the four Loop is over here and then the test to apply is at the end here if e% to equal equal Z and then what is the function we're applying it's just e^2 like before so the test just gets appended to the end of this list comprehension expression here yeah is it running faster is there a reason to do that does it run faster I'm not sure actually it might run marginally faster but probably not significantly the reason to do this is because as you get more practice with it this will be easier to read in code and often if you see a large chunk like this your eyes will glaze over right you're not going to want to read a chunk like that but if you see it all in one line you're going to think well how bad can it be and so you can come up with really complicated list comprehension Expressions um but usually we reserve them for you know really simple really quick ways to create these lists that you just populate you know with some values right off the bat so it just makes the code a lot easier to read okay so list comprehensions are are pretty useful if you get a little bit of practice with them you'll find yourself kind of using them all over over the place and they basically replace code that looks like this okay so these lines of code is a very generic way of writing this one uh liner list comprehension so here I've got uh a function f that I would like to apply this XPR uh expression is the function I would like to apply to each element this is the list I would like to apply that function to and the test is going to be you know the condition in this particular case this will this test means I apply it to every single element but you can imagine having a function which you know in the previous case we would say Lambda x x% 2 equal Z as our conditional and then the function that we're essentially replacing is uh is this with list comprehensions right we create this new list again this is the the pattern that I that we saw in the previous slide we Loop through every element in the list if that condition holds append that function applied to each element and then at the end return the list okay so this is just a very generic way to write a list comprehension so let's look at some concrete examples so here I'm not applying um the function e s to a particular set of elements from a list I'm applying it to the sequence of values given by range remember when we were talking about for Loops iterating through things they can iterate through integers foll some pattern like range six range one comma 9 comma 2 something like that as long as you have a sequence of values you can iterate over you can plop that into this list comprehension so you could iterate over lists you could iterate over tupal you could iterate over uh these direct ranges you could iterate over range of the length of a list whatever creates an iterable for you you can put that put that in the list comprehension so in this particular case the way I read this is I've got something that I'm squaring and what's the thing that I'm squaring it's going to be each value in range six so I think about it like what is this sequence of values that I'm going to operate on well it's going to be the number 0 1 2 3 4 five and the thing that I'm going to do to them is square each one of those values so the end list that I get out of this one liner here is a list containing 0 squ 1 squ 2 squ 3 squ 4 squ and 5 squ we can add a condition to that so here I've got the each element squared for e in range eight only if e is even so then the way I think about it is let's start off with what every element in the range is well it's 0 1 2 3 4 5 67 the condition I'm applying to that is that it's even so the numbers I I'm going to end up with I'm filtering all those to only contain uh 0 2 four and six because we go up to but not included eight and then I'm going to square every one of those so the end result from this list comprehension is a list containing the elements 0 squ 2 squar 4 squar and 6 squar and lastly you know we've been doing just single uh integers in the resulting list but as I mentioned we can do more complicated things so as long as we can write a little expression here for the thing that we'd like to calculate or add to the list we can put it in the list comprehension so in this particular case the element that I'm adding to my list comprehension uh or my resulting list from the list comprehension is a list itself so each element in my resulting list is another list right and that inner list is going to contain two elements every time the thing I'm actually iterating over and it's square and I've got a condition here here so I've got uh the elements 0 1 2 and three that's the range but I'm only grabbing the odd ones in this particular case so the resulting set of numbers that I'm going to I'm going to apply this to is going to be the number uh is the numbers one and three right because those are the two odd numbers in range four and so the resulting list is going to contain two elements so this outer square bracket is the list that I've created and its elements will be the element that I have actually iterated over and it's Square as a list right so one and one squared for e and e squ when e is one and then three and N 3 squ when e is three questions about that okay so pretty cool it's a really nice way to create lists really quickly like if you wanted to create a list full of Zer full of 100 zeros no need to do a loop you basically do a list comprehension that says square brackets 04 e and range 101 or 100 right and then you've got yourself a nice little List full of 100 zeros all right so think about this and then tell me what the answer is so the idea here is we have this list comprehension and just go through it step by step it looks a little bit intimidating but the first step is to look at the for Loop and ask yourself what are the values I'm iterating over then look at the condition if there is one there is one in this case it's at the end here so now what subsets of those original things you're iterating over are you actually keeping and then from those things that you're keeping what function are you applying it's the one right at the beginning so think about it and then I'll ask you to tell me so step one what are the values I'm iterating over the the full values not including the condition someone yell it out yeah that list in the middle awesome okay so XY um ABC d right and then seven and then what's the last thing is it the number 4. or string yeah exactly 4.0 okay string string step two from this list what are the values that I'm actually keeping right based on the condition if there're string all right which one's a string is XY yes is ABCD yep is seven nope is 4.0 yes excellent okay good okay so then these are the elements that I'm keeping and now what's the function I'm applying and what's the result going to be it's going to be a list containing yeah three uh two four3 yeah two because that's length two four because that's length four and three because that's length three great and we've got ourselves a nice little list based on that condition that sequence of values and that function applied yeah why does it return a list uh the the whole thing or I guess I it return oh yeah so we're not printing things out here when we're writing this as a list comprehension we're essentially telling python to create this resulting list of values that's just what a list comprehension does and so just kind of this expression here with these outer square brackets around our entire expression tells python that the resulting thing is a list yeah this is a good question other questions okay okay so that oh yeah question does it support multiple conditions does it support multiple conditions yes so at the end here um you know you would say if and then you could wrap them in parentheses I think I don't know if you have to but just to be safe I would wrap my conditions in parentheses and you'd use and or or or whatever you want to combine the expressions or the conditions with there a question yeah this one the Lambda um here this is a Lambda function that we talked about I forget when a couple lectures ago um it's basically an anonymous function and all it does is return true all the time so the test will always be true which means that when we do if test parentheses e this will always be true in this particular case but with given a different Lambda function that might not be the case Okay so let's move on to the next topic the next I guess two topics will be dealing with functions and I want to wrap up a couple things here just to give you um a couple more ideas regarding functions so the first one is actually related to this last question is the idea of a default parameter so this is going to be um uh a way for us to add parameters to our functions that get some default value and that's what that Lambda thing actually uh was in that example but hopefully this piece of the lecture makes that a little bit more clear and then the second part regarding functions we're going to go over is the idea of um functions as objects kind of working up on that and we're going to see what happens when we return a function object from another function right we've seen functions as parameters to other functions but we're going to see what happens when you make a function be the return value of another function but that's in a little bit for now let's look at default parameters okay we've seen this code before um triggering flashbacks so this is by section root uh I'll go over it uh just to remind ourselves what it does we've got uh this code inside this function we wrote a long long time ago right and then we decided to wrap it in a function so that it's a really nicely useful piece of code that we can run many many times so the parameter to this function was x a value we'd like to approximate the square root of right and the code we're using to approximate is using the bis section search algorithm which initializes some variables namely Epsilon how how sort of how close we want to be to The Final Answer low and high end points we remember that and then initial guess the halfway between low and high and then we keep making guesses between low and high right being the midpoint of low and high as long as we're not close enough to uh the uh the final uh we're not close enough to the final answer right so we're going to either reinitialize our low end point or our high end point depending on whether that guess was too low or too high and then within the loop we make another another guess using those changed values of either low or high based on if or else and then we keep doing this process of making more guesses at the halfway point as long as we're still farther than Epsilon away okay that was a a recap of what we've done so far the interesting thing that we had done with this function was or when we turned it into a function was to return our approximation so this guess instead of just printing it to the to the user we returned it so that it could be um useful in other parts of the code and so when we called the function we just said name of function and then some value of x now there are situations where a user would want to change the value of Epsilon right right now the way we wrote this code Epsilon is set to 01 and whenever you run the function it always finds um the approximation to the square root of x to that Precision 01 now sometimes you know depending on the application the user might want an even better approximation so 01 or they might not care to be as precise and they want you know maybe approximated to one or to 0 five or something much bigger than 01 so what are the options in this particular case right for this these scenarios one option would be obviously to go inside our function and say well I'm going to change Epsilon to be something super duper precise 00001 and so people who call this function will always get an approximation to that Precision but what about people who don't want it that precise right so all the function calls are going to be affected by making that change and so that's not really desirable we'd like to let the person who makes the function call be in charge of what Precision they'd like another option is to put Epsilon outside the function so to say okay well the only parameter is going to be X and let's not set Epsilon within the function let's let the user maybe set Epsilon outside the function and then they can use and then our code will basically pop up one level to the global scope and use the Epsilon that the user chose not a good idea because as soon as we allow somebody using our code to kind of make their own variables within our code uh we're putting our trust in somebody else's hands and and you know they might forget to re uh reset Epsilon or they might forget to set it to begin with and so that's just using Global variables is not a good idea in the first place we'd like to keep control of the Epsilon that's being used inside our function so unsurprisingly the last option is going to be our best option let's just add Epsilon as another parameter to the function right so there it is we've got uh bisection root again as a function we've got a parameter X and we have Epsilon as a second parameter that the user can can uh can uh call the function with okay so other uh other than that the function body is exactly the same right except that right now when we make a function call we have to pass in epsilon as the second parameter so in terms of code this is the by section root with Epsilon as a parameter and so now the user can find the approximation to 123 to 0.1 it's 11.88 in case you wondering and then the approximation to 123 to 001 which is 11095 okay so much better the user can now be in charge of deciding how close they'd like the approximation to be for every one of their values but notice that this code is kind of verbose and really most of the time maybe the users don't want to be in charge of setting the Epsilon maybe they don't know what a good Epsilon might be right so how do they know that they should choose 0 Z1 by default maybe that's something you could put in the function specification for anyone using your function but it's you know you're going to rely on users reading your specifications and that's a little bit uh scary so instead the functionality that would really like to have is to say Okay I want to write a function that does take in two parameters but by default one of those parameters is something that I set as the person who's writing this function right so what I would really like to have is Epsilon to have some sort of a default value so if users don't know what to call it with that the code will just use that default value and otherwise if the user is more experienced and they know they'd like an Epsilon of you know 1 time 10 the -10 or whatever it might be then they can be in charge of setting it okay so most of the time we want to call the bisection root function without an Epsilon parameter so that it may use a default one but sometimes we'd like to allow the user to actually set the Epsilon and so to that end we're introducing the idea of uh keep keyword parameters also known as default parameters and they are set like this so the bisection function definition still takes in the thing we'd like to approximate the square root of x but the second parameter here Epsilon will be equal to something inside the function definition so we as the people who are writing this function are going to say the default value of Epsilon is 01 okay so that means when we call the function down here if the user makes a function call without explicitly passing in a second parameter python will use the default one that the person who wrote the function set right so python will run by section root of 123 with Epsilon being 01 and otherwise if the user does want to override that Epsilon they can just pass it in themselves and that default value of 01 will be overwritten to be .5 okay and so in our code here this is the bis section root with the default values and so you can see here if I run it with 123 even though there are two parameters here for the bisection square root function it's python doesn't complain because it's using Epsilon as 01 so I run it and it runs just fine but in the second line here if I actually want to use 0. five as my default as my Epsilon value it overrides my default parameter and it calculates of 123 with Epsilon being .5 okay questions so far so now that we've introduced default parameters there's a few sort of rules about making function calls when you create the function definition so over here right when you're the one defining a function and you decide to allow some default parameters in your parameter list everything that's a default parameter needs to go at the end you can't switch these around you can't say epsilon equals 01 comma X python will not allow that so anytime you have default parameters they always have to go to the end that's the only rule for making the function called or making defining the function with default parameters and then once you have default parameters you can actually call the function in many many many different ways okay and I know these some of these will be confusing you might not know whether they're allowed or not you can never go wrong with the last one as we're going to see in a bit so the first one here showcases what happens when you give values for everything that's not a default parameter right in this case just X if you just give a value for non-default parameters python sets default parameters for everything else so not a big deal um alternatively you can pass in just like we have in the past when we write our own functions with multiple parameters you can pass in one at a time in the same order values for every one of those uh parameters default or not and if you pass in values for all of them python will not be confused and it'll just match them one at a time um variations on that you can always pass in a value for a parameter name okay so looking at the function definition we can see the parameter names the formal parameters are named X and Epsilon so when you make your function calls you can actually explicitly tell python something like this xal 123 Epsilon equals 0.1 right and if you have more parameters you say that parameter equals whatever value you want to run it with and so that will not confuse Python and if you do it in that way you can actually do it in any order you'd like because python will just assign each one of these variables to be whatever you told them to okay so you know worst case you just do something like this right where you one at a time you just say what the formal parameter is and its value and then python will not get confused the ones at the bottom though is where we run into trouble so for example if you put the default parameter first and then you put an actual parameter uh sorry you put the default parameter first and then any parameter that's not a default one afterward python gives an error because the default ones have to go after the non-default ones and the last one doesn't actually give an error but python remember matches parameters one by one so it's actually going to find the an approximation to the square root of 0.1 to an Epsilon of 123 okay because it's just mapping the parameters one at a time and so that's not an error but it's not exactly what we wanted to do questions about this okay so now let's move on to another thing uh another uh sort of nuance about functions and we're going to go back to the idea of functions being objects in Python so I drew this picture uh back when we first learned of objects of functions as objects so I'll just do it again just to jog your memory so remember that when we make a function definition inside uh the memory python creates an object right as soon as we see just this function definition python doesn't care what code is inside here this code does not run right it only runs when it's being called and right here I have not made a function call at all all python knows at this point is that there is a function object inside memory and it name its name is is even and this is exactly the same as creating an integer object inside memory and giving it the name r through a line like this or creating a float object in memory and giving it the name PI right it's just some object with some name and so that means that we can have some code that looks like this which is uh going to essentially create an alias for that function object in memory okay so here the name is even refers to that function object and I'm telling python that I would like to refer to that function object using the name my funk as well so both my funk and is even are names that point to this object in memory right it's not a function call right I'm not P I'm not trying to figure out if some number is even I am literally giving another name to this function this code right that does this thing here okay and so that means that if I have two names that point to the same object if I am going to invoke those two names as I do here with some parameters python is going to say well I'm going to run the code pointed to by these names with these parameters so they will both run the code that they're pointing to right this is even and so it's just going to return true or false we we've seen this before so remember just another name for that object in memory so we've seen already how we can pass functions as parameters to other functions and now we're going to see what happens when we uh return a function from another function so we're not returning a function call here right we are returning a function object so in this particular code we have only one function it's named make prod and it happens to have some stuff going on inside it so what's the stuff that this function will do well this function itself will create another function so this G only exists whenever make prod exists the main program at you can think of it as sort of this level of the code right in terms of IND entation the main program does not know about g g is only defined inside make prod right so when we first run this program as is there's no function call being done so the main program does not know anything about the internals of make prod okay so make prod creates its own function here and then all it does is return this function object right notice it's not a function call there's no open close parentheses with a parameter in it it's just the the name g it's this function object that's the key thing here so let's run two codes this one and this one they will do the exact same thing they're going to call make prod um with some parameters and then we're going to see what happens when we return this G and notice already it's looking slightly different than what we've been doing before yes we have a call to make prod here but we've kind of chained another function call right after make prod right we've got make prod parentheses 2 parentheses 3 and so this is kind of like I think of it as chaining a bunch of function calls together and this is possible as we're going to see when we step through the uh the uh the function environments that are being created this is made possible because make prod this function call returns a function itself okay so let's step through the code on the left very carefully and then I'll step through the code on the right which will do the exact same thing and hopefully it will clear up confusions if we do it twice so this is the code from the left let's say we have this exact program here I've got one function definition and then I've got one function call here and then I'm going to print the return value so as soon as I run my code python creates my global environment and in the global environment this is the scope of you know the the main program what do we have well we have one function definition which has some code within it I don't care what it is at this point because I don't have a function call so then the next thing that I need to do is go down here and say Val equals so I'm going to create a variable Val in my global environment and and I'm going to make a function call so function calls are done left to right right just like expressions and the first thing python sees is this function call make prod parentheses 2 it's a function call so we need to create another Orange Box because a new environment gets created every time we create we make a function call so here I have my scope my environment for make prod and I'm currently just stuck here trying to figure out what this is going to return okay just the red box here well every time I have a function call I need to look at the function definition and the function definition says well there's one formal parameter a that I need to map to the actual parameter so the thing I'm calling make prod with is two that should be pretty straightforward right and then I can move on to do the body of make prod okay so the body of make prod says I would like to create a function definition the name of this function is G so there's G and it contains some code again I don't care what this code is because I'm not making a function called to G yet right right now I'm just defining G okay so so far so good so this G I want you to notice only exists inside this call to make prod the global environment does not know about G at this point right because we only Define G inside make prod it's here right I didn't Define it outside of make prod so the globos scope doesn't know about it but make prod does know about it and so the only way that the global environment can know about G is if this make prod function some how returns G okay so if we pass G back as a parameter as a as a value sorry to the main program scope the main program can know about G but otherwise G is kind of stuck in this little you know little subtask little environment of make prod and the main program doesn't know about it and so that's what this code is doing it's essentially saying well I've made my definition and now I return turn G so here this G the co and and it's code and the associated code right so this object pointed to by G is going to be returned back to the main program okay so now the main program knows about this object G that has some code associated with it right this line here where it returns a star B so the thing that I've boxed Bo and read down here is the return value from make prod 2 and make prod 2 return G so this you can essentially say is G is that okay does that make sense right we're passing functions along right not function calls and so this is just a function named G and so now this line of code Val equals if we were replace the red box with G Val equals g parentheses 3 so G parenthesis 3 is another function call right just clearly we we look at it it's a function call it's got a function name parenthesis and a parameter and so since it's a function call we create another scope for this function call as before we look at what G takes in as a parameter it's a a variable named B right a formal parameter B and we map it to three because that's our function called G parentheses 3 and then we have to do the body of G the body of G says return a multiplied by B well I know what B is it's three because you just called me with that value but what is a right the scope of G has no a within it so thinking back to our function our lecture on functions if a function called doesn't know about a variable name within its environment within its scope it moves up sort of the the the function called hierarchy so it says who called me right where was G defined well G was defined inside make prod and so it was called from make prod does make prod have a variable named a it does right and its value was two so we didn't need to go any further up the hierarchy we've already found a variable named a so python will use b as three and a is two okay multiplies that to B6 and then uh the G function call can return six it returns it back to the main program because that's where this function call was being done right remember we had this uh replaced with G parentheses 3 out in this Global scope here and so that's six gets returned back to the main H main program and then Val becomes six and we print six okay so that was showing you how to chain function calls together and this was only made possible because make prod as a function returned another function object if make prod returned I don't know a tupal or an integer or something that was not a function this code would fail because the return from make prod would be let's say it returned the number 10 the return from make prod would be replaced with 10 and then python would see this line as 10 parentheses 3 what the heck is that right and so it would completely fail and so this is only made possible by the fact that this make prod function returns a function object and so we're able to chain these function calls together so so let's look at the exact same code except this time instead of chaining them in a row let's explicitly save the intermediate steps okay so what I'm going to do is say make prod parenthesis 2 I'm going to save as a variable and then make that variable call uh the three right the the second part of my chain from the previous slide and it's going to do the exact same thing so here I've got the global scope just like before I've got a function definition for make prod so this is the name make prod it uh it points to some code and then I've got this variable doubler that's going to equal something so that's a function call the function call says here's my environment for make prod with its scope so in this particular scope I've got my formal parameter a that maps to two and then the function body itself creates this variable G that's just some code exactly the same as before any questions so far based on what happened last in the last sort of example and here or is this okay so far okay so now I've set up my my code and this is where the interesting part comes in right make prod is going to finish its call by saying I'm going to return something and the thing it returns is G so it returns this name g just happens to have it happens to be a function object but you know think of it as anything else we're basically saying doubler equals 10 or doubler equals some list or some Tuple doubler is going to be some some value right this value is just code associated with a function so in my main program scope I've got doubler equals g which based on the memory diagram we did like five or 10 slides ago right it's like when we had my funk equals is even I basically have two names for the same function object right doubler is a name and G is the other name and they both point to this function object does that make sense that that's okay okay okay so now that I've got two names that point to the same function object we can just use this doubler in the next line and this doubler is like saying G parentheses 3 except that I'm using the name doubler which I saved it as on the previous line so G parentheses 3 is another function call create another environment for uh for G or doubler uh whatever name and here I've got one formal parameter B it's values three and then we do the same trick where you ask what is the value of a I'm going to look up the hierarchy of things that got called to see what the first value of a that I grab and the first value of a that we grab is the two right and so we're going to multiply the two with the three and that six gets returned back to whoever called it which was out here in the main program scope and so this Val will be equal to six and that's it questions which one was easier to understand this one or the one where we did the chaining just show of hands who liked this one more who liked the chaining more oh interesting okay was the chaining just easier to to grasp because there were less names okay cool I'm glad I showed it first then any questions though yeah no reason in fact you would want to do the chaining way because then you avoid extra lines of code and again with practice it just becomes really easy to know what's going on yeah okay so um that might have been confusing um why do we bother doing that because that particular example all we were doing is multiplying two or I guess doubling a number okay um we could have easily written that code to you know to double a number and without actually returning a function that seemed way overkill for what that code was trying to do um well it was kind of showing you what you can do with an easy example and you would definitely never ever write uh you know functions returning other functions for such simple examples but it's really a method for um cases where you have larger pieces of code that you'd like to write because if you're trying so so so if you're writing a a larger piece of code right some software project and every single function you'd ever want to use is kind of defined at the top level in the main program it would become really messy right and so there are cases where you would like some functions to only be visible or accessible by other functions right and so you only Define those functions within the scope of other functions that's that's one thing the other thing is using this chaining method allows you to have some uh control over the the flow of control of a program and so you can imagine in the pre in the example here where you basically create this you you have this line here and at some point you return G right and you don't want to do the doubling right away right so you don't want to do Val equals doubler right away you can imagine having a bunch more lines of code here that do other stuff before you actually do the the the doubling right and so in that in that case in this larger more complex program you're essentially interrupting the flow of control here you're not doing the doubling right away but you did grab this function back and then you can maybe do other things with that function before finally doing the the the doubling and so in that case you you can basically execute some code partially do some other operations and then finish executing um you know at at the end after you've done these operations so again for this example it doesn't make much sense but in a larger piece of code it's this um this idea of uh functions returning functions is just another tool to achieve these ideas of decomposition abstraction which lead you to write more organized code more robust code more easy to read code and so on and so on okay so you don't have to to do this but you do have to understand what it means for a function to uh return another function any other questions okay so now we're going to do the last piece of today's lecture ideas of testing and debugging um this uh this lecture is usually pretty dry so I'm going to try to make it more fun uh as as fun as I can um the reason why we introduced this lecture now is because I'm hoping that by this point in the course you've had a chance to do some uh testing and debugging strategies on your own by kind of a trial and error uh thing on quizzes and on pets so you've gotten a chance to maybe use the python tutor you've gotten a chance to use print statements um you know various things like that and see what works best for you what doesn't work at all things like that so you've maybe gotten a little bit burned by some of these strategies but I hope that by you being burned by you know some things that you've tried that work that didn't work you'll maybe appreciate this lecture a little bit more than if I just showed you this lecture you know back on uh you know day one or day two or something like that because it's a lot of common sense stuff but um there's a little bit of actual strategy as well in this particular uh set of slides so your programming experience so far I know this is certainly mine is I hope that when I run my code it immediately runs perfectly but instead what ends up happening for me is I run my code and it immediately crashes I've got my red errors on the side and I I get a little bit flustered okay so this is exactly what happens probably for you too and the idea here is that you want to write the code in such a a way that um it makes it easy to test and debug and I know I always say this and I actually don't always practice it but it's important to write The Code by writing it with uh uh by adding comments as you're writing the code right so writing specifications writing comments for yourself as you're actually writing the code not when you finished it is very important it helps you as you're writing it or coming when you're coming back to it in a couple days um modularizing the programs also helps so if you see a chunk of code that you're copying and pasting all over the place you want to you know plop it out in a little function that you call multiple places so ideas like that um kind of employ the this defensive programming mechanism and it allows you to uh perform really easy testing and validation when it comes time to do that and then possibly debugging when it comes time to do that so the lecture is going to be divided into two pieces the first we're going to talk about testing and validation some nice testing strategies and we're going to talk about some strategies for debugging as well so the testing and validation part is where you come up with a set of input test cases and expected outputs and all you're doing is running the test uh running your code to make sure that the expected output matches the output that you actually get from running the code the debugging Strate uh the debugging part is where one of your tests don't match the expected output okay one of the outputs that you get don't match the expected output and at that point you have to figure out why the code is not working obviously so before you even test your code as I mentioned before it's you have to set yourself up to do the testing and debugging so to ease this part it's important to write documentation very well so when you're writing your own function not functions that we've given you document the doc string what are the inputs you expect what should the function do what should the function return things things like that um if you're writing the code in a sort of a strange way or if you Ed some piece from stack Overflow or something like that document it to make sure that if you're looking at it a week from now you still remember what that piece of code did so really really simple things like that can make a really big difference when it comes time to test and debug um breaking up the code obviously into smaller chunks is very important because if you're copying and pasting the same piece of code over and over again um you remember to make a change in one place you might forget to make that same changes on in all these different places and so that'll be very frustrating uh when it comes time to actually run and debug the code so once you have code that's written you would start the testing process um you remove all the errors static errors uh static semantic errors and syntax errors are really easy to remove python immediately tells you right index error on this line or syntax error on this line those are really easy to figure out um using a paper and pen or typing it out on your uh you know on your uh on your uh in your python file you come up with a bunch of test cases and for each one of those test cases the way we you know the way we write on your micro quiz uh test cases you say what you expect the output to be so when you actually run it you don't need to remember what this act output should be right it's just written down on uh down somewhere on paper or on the uh on the screen so when you're creating a bunch of test cases you can create some different classes of tests right so hopefully we're modularizing our programs which means that we're creating functions the simplest classes of tests are called Unit tests and these tests basically test a function with different inputs okay so what you're going to do is come up with a bunch of different test cases for one particular function and run these test cases on the function if they all work perfect but if they don't or if you find a bug as you're writing test cases in the code you'll want to perform regression testing and regression testing means that as you find a bug you add a new test case for them right or as you fix a bug you make you run the code uh you run the same uh the same code with all of the previous test cases to make sure that the bug you fixed didn't introduce errors in a previous test case okay so you there's a bunch of iterations of unit testing and regression testing to test all of these different modules all the functions in your program and at some point you're ready to do integration testing and in integration testing you've got all these modules for example as you did in hangman you've got all these little functions right that do individual things you put them all together into a larger program right in hangman it was you know a bunch of big while loop where you check all these different things that the user might input and then you call the different functions you wrote and as you find errors in uh in the integration right when you've written code that integrated all these different pieces together you might have to go back and do more unit tests for some of the uh some of the functions that you wrote okay so you've done unit testing regression testing and integration testing what are some actual testing approaches how do you actually create these test cases to run your code so I guess the most natural way uh to write a test case is just intuition about the problem so given a a dog string what are going to be some natural boundaries some natural values of X and Y for which you test this code with you guys tell me what's some values that we could test this code with think about the boundaries to the question right yeah three and four is good so X is uh less than y is a good one vice versa right four and three is another one where Y is greater uh less than x we could test them being equal right what about zero and zero what about a thousand and a thousand so we could do extremes we could do bigger than less than we could do equal things like that right so mathematical functions are kind of easy to apply this idea to because they just have natural boundaries but often there are functions which don't have these natural boundaries and then we might be stuck doing random testing and in random testing obviously the more test cases you have the better chance you have of finding a bug but there are actual techniques for coming up with test cases so the first one is called blackbox testing second is called glass boox testing now in blackbox testing you're going to treat the code of the function as a black box so we don't even look at what the code is doing all we're looking at to guide writing our test cases is the specification the dock string right and so hopefully the person who wrote this function wrote a really nice dock string because that's what we're going to use to write our test cases so the way that we're going to um write a test case for this square root function is by saying what is the value of x and Epsilon according to to this uh to these constraints here right so obviously we're not going to test the code with values that don't match those constraints because the person who wrote this function doesn't guarantee that this function will work for those out of you know those those weird values so the good thing about blackbox testing is if we're the ones testing this function we're only using the specification to write the test cases so if for example this person implemented square root using approximation method I I don't care my test cases will work if the person changes their implementation to use the bisection method right my set of test cases will still work even if the person who wrote this function changed the the the the black box right the implementation so it's blackbox testing is is is really nice in that respect and so for this particular function here's a bunch of test cases that I would run it with right so obviously X being a zero perfect square less than one are are are kind of you know nice ones to test irrational values and then a bunch of extremes is also good to test and then Epsilon the same we've got some reasonable values of Epsilon and then some extremes and we can even you know mix and match we can have zero and extremes epsilons and perfect squares and extremes epsilons and things like that so lots more test cases than this but this is a really good start in glassbox testing we're going to use the code itself to guide the test cases that we write so if we uh write something a test a test Suite that's path complete that means that we're going to hit every single path inside the program okay so that means we have to look at the code to guide the test cases that we're writing which means that we're going to have to write a test case for uh the code hitting the If part of a branch we have to write a test case for the code hitting the else part of a branch or the L If part of the branch if we have a for Loop we need to write a test case where the code doesn't go through the loop at all it goes through once or it goes through many times through the loop right same with while Loops we write a test case so that the code doesn't go through the Y loop at all it matches the condition once or it matches the condition man