Generators in Python - Advanced Python 14 - Programming Tutorial

Key Takeaways

hi everybody welcome to your new Python tutorial today we're gonna talk about generators and Python generators are functions that return an object that can be iterated over and a special thing is that they generate the items inside the object lazily which means they generate the items only one at a time and only when you ask for it and because of this they are much more memory efficient than other sequence objects when you have to deal with large data sets they are a powerful advanced Python technique so let's have a look at some examples to understand how they work a generator is defined like a normal function but with a year keyword instead of the return keyword so let's define a function call it my generator and here I can you return or I can yield some values so here I use the yield statement and yield a value so I want to yield one and then I can have multiple use statements inside a generator function so I can for example also yield two and then yield three and now I can create a generator object so I can say Chi equals my generator and now if I print this and this will only print that this is a generator object and now what I can do for example I can loop over this object so I can say for I in Chi and then I print the value so this will print one two and three and I can also get the values one at a time with the next function so I can say value equals next G and then I can print the value so this will print one and this will execute the function and runs until until it reaches the first youth statement and here it returns the value and pauses at this line so the next time if I want to get the next value again with this next function so again I say value equals next Chi then it will continue here and runs until the next Youth statement so it runs until here and returns to and pauses here so if I run this now it will print 1 and 2 and if I do it again then it will also return and print 3 and now what will happen if I try to run it a fourth time so now if I run it this will raise a stop iteration because a generator object will always raise a stop iteration if it does not reach another youth statement so yeah this is how generators work and you can also for example use them as inputs to other functions that take iterables so for example the built-in sum function takes a iterable so I can give the generator object here and I can print this so this will calculate 1 plus 2 plus 3 equals 6 or I can for example use the built-in sorted and method and put the generator object here so this will return this will create and return a new list with all the objects in a sorted order so for example if I have it the other way around 3 2 1 and then with this I can sort it again and then it prints 1 2 & 3 and now let's have a closer look at the execution of a generator function again so let's say I have another generator and I call it countdown and it takes a starting number and then I say first of all I want to print starting and then I say while num is larger than zero I yield the num and then I also want to update the numbers I say num - equals 1 and then I create my generator object so I say CD equals countdown and for example I want to start at 4 and now if I let's first of all run this and notice that this will not print starting here so nothing will be executed here and now the first time I want to get the first value with let's say value equals next of this countdown generator object now if I run this then now it will start from the beginning of this function and execute it so this will print starting and then it will run until it reaches the first yield statement and here it will return the number and stops at this statement so I can also print the value and then it prints 4 and again the next time I want to continue here with again with this next statement let's say print see print next CD then it will continue here it will remember the current state so the current number is 4 then it will update the number now the number is 3 then it will continue in the while loop and then it stops again at this line and now returns 3 so now if I run this this will also print 3 and then again it remembers the state and the next time I continue it will continue from here and so on and again if I run this a couple of times so again if I print next and it will also print too and now it will also print one and now it will raise the stop iteration so this is the execution in detail and now let's have a look at the big advantage of generator so as I said generate us a very memory efficient so they save a lot of memory when you work with large data so what this means is let's have an look at an example let's say I want a function call it first n and it takes a number as input and this will return a sequence with all the numbers starting from 0 all the way up to n so usually what you would do is you create a list call it numbs equals an empty list then you also say num equal 0 so this is your start number and then you say while num is smaller than n num stop append num so you add the current number to your list then you update the current number so you say num plus equals 1 and at the end you will return this list so you return nums and now I can say for example I can say my list equals first n and give it for example 10 and then I can simply print this so now this will print all the numbers from zero to nine in a list and for example I can also calculate now the sum of this so this will print 45 and now here with this way all the numbers are stored in this list so this takes a lot of memory and now if I use a generator instead I can say I define another function first n underscore Jenna rater and now it also takes n as input and now I don't need the list anymore I simply say num equals zero and also the while loop while num is smaller than M and here I simply yield the current number so I yield num and then I also have to update the number so I say num plus equals one so this is the whole implementation of this as a generator object and now I can for example also print the sum of this first end Qin er radar object and now you see this will and give the same result and this will also print 45 but here I don't have to save all the numbers inside this array so I can save a lot of memory here and for example if I analyze this I can import this and now I can get now the size of this object so I can say sis dot get size of this object this will return the size of this object in bytes and again here I also say print sis dot get size of this object so first I print the size of my list of checked and then I will print the size of the generator object and here we see that already the generator object is smaller and now let's say I don't have ten numbers in here but let's say I have 1 million numbers in here and the same number of elements in here then this you see this takes way more memory so in use cases like this the generator object is very useful so remember this and another advantage of the generator object is that we do not have to wait until all the elements have been generated before we start to use them because we can for example get the very first item with the first next statement and we don't have to calculate all the numbers yeah so this is the big advantage of generators now let's have a look at another example to to practice the generators a typical example is the fibonacci sequence so we say define if you will not cheap and this will give this will get a limit as argument and the fibonacci sequence works like this so the first two numbers are 0 and 1 and then all the following numbers are a sum of the previous two numbers so now we have 0 plus 1 is 1 now 1 plus 1 is 2 1 plus 2 is 3 and so on so then we have 5 8 13 and so on and to implement this as a generator first of all we have to store the first two values so we say a and B equals zero and one and then we say while a is smaller than our limit we yield the current value so the current value is a and then we update the current value so now we say a equals B and also we in the same line we update the B value and now the B value is the sum of a plus B the sum of the previous two numbers so we say a B equals B and so a is B and B is a plus B so this is the whole implementation of the Fibonacci sequence and now we can say for example fit equals Fibonacci and as a limit for example I give it 30 and now I can loop over this object I can say for I in fib and then print I and now we see this will print the sequence until until this limit and now as a last thing let's have a look at generator expressions so generator expressions are written the same way like list comprehensions but with parentheses instead of square brackets and this is a very simple syntax and a shortcut to generate some generate to implement a generator expression so I can say my generator equals and now I use parentheses and here I can use an expression with a for in loop so I can say I for I in range for example ten and I can also use an if statement I can say if I modulo two equals equals zero so this will put all the element all the even elements from zero to nine in a in my generator object and so for example I can print or I can loop over this object so I can say for I in my generator and then print I so this will print zero two four six and eight and this is similar to the list comprehension so the list comprehension works the same way except that they use square brackets here instead of the parentheses so I can say my list equals this expression and then if I print the list this will print the same sequence as a list and by the way I can also say I can convert I generate an object to a list with the list function so I can say print list my generator and this will and do the same thing and again let's analyze the size of this so let's say print sis dot get size of this object and here also I want sister get size of this object and now they here they are almost equal but let's say again I have a large number 100,000 then again my generator object is much much smaller and saves a lot of memory so yeah that's the concept about generators and I hope you enjoyed this tutorial and see you in the next tutorial where we talk about threats and processes in Python