5 Custom Python Decorators For Your Projects

Key Takeaways

what is going on guys welcome back in this video today we're going to take a look at five awesome and useful python decorators that you can Implement for your projects so let us get right into it not a it's all right so we're going to take a look at five useful python decorators in this video today which we can use in our project and these are going to be decorators that we Implement from scratch manually in this video today we're not going to use decorators from core python or from some other package we're going to manually Implement them and customize them for our specific use cases now for those of you who don't know what decorators are there are basically these little annotations that you can add on top of a function to change or extend that function's behavior for example the cache decorator from core python can add caching to a function now let me maybe show you a quick example just so you know what we're talking about here if you've never encountered uh these decorators before basic idea is I have some function this function has some uh obviously functionality so let's say I have process input here with a parameter n and this function does some calculation do some calculations um and in the end we're going to get a result for example N squared which is now a very simple calculation but let's imagine we have something more complicated let's use time sleep 2 here to wait 2 seconds and let's just simulate by doing this that this takes some time to compute now if I print process input called on five for example and I do this a couple of times when I run this what you'll see is I'm going to wait 2 seconds I'm going to get the result 25 I'm going to wait again 25 wait again 25 because even if this was a real function it would have to do all the calculation all the time the idea now is there is a decorator called cache or also lru cache in fun tools so I can say from fun tools import for example cache and by adding cache as a decorator to this function this is how you add a decorator at cache this is going to now be a cached function which means that when I run this again the first time I'm going to have to wait the next time it's going to recognize I already know that uh input value I can return 25 immediately because that's a cache result so this video is of course not about caching but this is just an example of a simple decorator uh it's a little thing a little annotation that I add on top of a function to extend or to change the behavior of that function now of course in the case of caching this only works or should only be used for deterministic functions but again this video is not about caching what we want to do in this video today is we want to implement ourselves five different decorators that can be very useful in any project I would say they're quite Universal in general and these are decorators that can really help you to avoid duplicating code to do basic stuff for example let's start with the first decorator right away let's say I want to have a decorator that allows me to measure the execution time of a function fun call so I want to know how long does it take when I call this function to actually get the result of that function call and for this actually let's go back and use the same function that we had before this process input function which of course is going to roughly take two seconds now what I can do if I don't use a decorator is I can say import time and then basically I can say start is equal to time. perf counter or time. time whatever you want to use and end is equal to that as well and then in between here I can say process input two and I can say in the end after calling all of this I can say uh time taken and then end minus start and of course if I run this this is going to work it's going to execute the function call and it's going to measure the time it's going to give me that result now if I have to do that over and over again and I want to do that all the time everywhere this becomes quite tedious to do it every single time like this so what I can do instead is I can take this logic and turn it into a decorator which I can call time logger for example or I can also log into a file it depends on what you want to do but the basic idea now is that I go ahead and Define a function called time logger for example this is going to be the name of The Decorator and this function here takes a function as input so the time logger will take the function whatever it is it doesn't have to be focused on process input it can work with any function uh it takes that that function as a parameter now in this function I have an additional function called the wrapper function and this function needs to be very flexible in terms of the signature so we're going to allow any arguments and we're going to allow any keyword arguments and this wrapper function will do the following it will take the function it will call it it will take the result and return it but it will also measure the time that it takes to do all of that so it's going to say start is equal to time. perf counter uh and is equal to the same thing and then I'm going to say result is equal to calling the function on the arguments and keyword arguments passed and then we're going to return the result here in the wrapper and um of course before that we're going to print the time it took to do that time [Music] taken and minus start and then we're going to say return wrapper the idea is we take the function and we call the function in another function and we actually return this function now uh instead of this function so we basically wrap this functionality around the function that we're passing which means that process input is going to be turned into this wrapper function that calls process input uh but it also does all the stuff around this so if I again just do the same thing if I just uh call all print process input 5 this is not going to do anything so the function is still the same it will just execute and give me the result but if I now add to the function up here The Decorator time logger like this this is now going to enhance this function with a timing Behavior so if I run this again now um it will still give me 25 nothing will have changed but I also get this logging message here time taken 2.97 and so on maybe we can format that a little bit better so maybe I can use an F string here time taken and then some curly brackets here and then let's say point for f and when I run this now we should have a better formatting but that's the basic idea of this simple uh decorator here I think this is quite useful because often times you want to know how long it takes to execute something if you try to see how efficient it is when you change the input size this is a very simple way to do that you just have to Define this once and then you can add and remove it from functions this is also a little bit related to debugging maybe maybe you don't want to have this necessarily in production or maybe in production you want to actually use the logging module but this is quite simple because I can add and remove this easily I can comment it out and then I just have my basic Behavior as before but with this annotation with this decorator I get the information about how long it took to run this so that's a very simple and useful decorator the second one I want to talk about is the retry decorator so sometimes you have functions that are error prone or prone to error so let's say I have here a error an error prone function and to make it somewhat realistic let's say I have a random chance of this function failing so let's say if random random is less than 0.9 then going to raise some value error I'm going to say error and otherwise we're going to just print success so what could this actually be this could be some API call this could be some operation that often times fails often times does not fail but if this function can produce errors we want to maybe retry because at some point it might actually work so in this case 90% of the time this function will fail we can try to simulate this I can just call this function um and see what happens if I just run the script I'm going to get an error I'm going to get success now I'm going to get an error error there you go rarely I will get success but sometimes I will get success so you can see most of the time I get the exception there I had a success again that's the idea of the function now what I can do with such a function is I can call it and if I fail I can try it again and try again until until I succeed or until I reach a maximum level of retry so for example I could say something like try to call the function error prone function um if this fails because of a value error then just try again so maybe we can do some some while loop around this while [Music] true if I get an exception here I can say pass or I can say uh time sleep 1 second and try again which can make sense for API calls for example um but I can Implement a logic like this what I can also do is I can turn this into a decorator and I can also parameterize this decorator so I can say up here uh Define a decorator retry and retry is going to have the parameters uh retries so the max retries let's set this to three by default and let's say that the exception that we're waiting for is going to be Exception by default so any exception and now in this method in this function here I want to have another function that I'm going to call decorator this function is going to take the function as before as an input and this function will have another function in it the wrapper so like this and this will take again arcs and quarks and the idea now would be to try as many times as we Define it here in this parameter and then if it still fails we can rease the exception or we can erase the exception in general and otherwise we're just going to try to succeed so we can say attempts is equal to Z by default and then while the number of attempts is less than the number of retries or Max retries uh we're going to say try to return the value of calling the function on the arguments and keyword arguments and if I get the exception type so accept exception s e then we're going to just say attempt plus equals 1 and we're going to print maybe uh I don't know let's say uh failed and then we can say maybe in parenthesis here attempts out of retries yeah and in the end if we get out of this while loop we want to erase the exception and here we want to now return the wrapper we want to return the decorator and that should be it because here we return here we Erase here we return return and that should work now so if I go ahead and I call the error prone function again a single time I will get an error if I do this now with uh The annotation up here if I say with a decorator retry and I say retries is equal to five and the exception is equal to to Value error I can run this and you can see I get fail fail fail fail success this was now coincidence that it worked I can of course also fail five times or I can succeed immediately or I can succeed after the four fourth time or in this case now I don't succeed at all so this can be quite useful if you have error prone functions just add retry on top of them and they're going to be executed multiple times so or until they succeed or until you exceed the retries and of course you can also add some stuff like uh delay up here so you can say delay is equal to two and then you can say if it fails which you want to do is you want to say time sleep delay this would also work right so you could have some mechanism for slowing down and in this case now let's go and say delay equals 2 and you can see now it's slowed down a little bit in this case it doesn't really make a difference because it's just Randomness but if you have some API calls this can make a lot of sense so this is a very useful decorator in my opinion because we don't have to constantly reimplement this logic this process of trying something again a certain number of times we can just Define what it looks like once we can parameterize it to keep it flexible and then we can just reuse it on any function that is prone to errors so this reduces the amount of duplicate code massively now let us move on to the next decorator which is the type checking or I would rather call it type enforcing decorator so in Python of course we have type hinting so I can define a function add I can say a is an integer I can say B is an integer and this is typ hinting I can then return a plus b but of course since this is python nothing prevents me from calling the function on the string 10 and on the string hello which is going to lead to a string concatenation this is valid python code no problems uh because this of course is just a hint a type hint now of course I can use tools like mypi to enforce them but I can also enforce them actually in Python code so that's it's impossible to actually run this uh with incorrect types and for this we can write our own decorator so let's call this decorator uh type check and the idea of this decorator is that we're going to pass a list or a collection of types and these should match the types of the arguments so I'm going to say expected types with an asterisk to say that this is uh of variable length then I want to have a decorator inside of here that takes the function as a parameter and then I want to have a wrapper again with arguments and keyword arguments and then I want to say that the goal is to iterate over all the arguments and see if they have the correct type so I want to say for argument and expected type in zip the arguments list so zip basically arcs and expected types together and if not is instance if the argument is not of instance expected type we're going to rase a type error and we're going to say that we expected and then we're going to say expected type but got type of the argument and otherwise we're going to just do nothing in the end if we get out of this Loop without raising an error we're just going to return the value of the function call so calling the function on arcs and quarks of course with the asterisks and then we return rapper and decorator so again running this without decorator results in a valid function call if I now run this with type check as a decorator up here you're going to see that we get uh actually we have a problem what did I do wrong let me read the error message again takes one positional argument but two were given oh sorry I need to of course specify the types in here I need to say that I'm expecting actually so the type hinting is relevant I can call this AB but I need to specify that I want to use the type check decorator actually with the expected type so I want to have int and int and when I run this now you're going to see that I get a type error expected class int but got class string this also is the case if the first one is actually an integer so if I change this to be the correct type this is still going to fail because the second one is not the correct type but if I change this as well to 20 then I'm going to get the function call so this is a way to enforce the types with a decorator so if you're building a framework if you're building some package or if you're just building an application that wants to en Force types for some reason you can do it like this just build this decorator raise a type error if they're not consistent and then you have type enforcement in Python so the fourth decorator that I want to talk about is quite basic it's the debug decorator for this we're going to use the same function here add and the debug decorator is quite flexible you can add whatever you want to it basic idea again is you define debug taking in a function and the wrapper in here takes in again arguments and keyword arguments and now you can do anything you want around the process of calling a function so you can say uh for example some lock message like calling function and then we want to say function uncore uncore name uncore uncore um with arguments arcs and keyword arguments quarks and then I want to actually call the function so result is equal to calling the function on arguments and keyword arguments and then I want to say result was result and of course you can do stuff like timestamps and date and whatever you want to have as a debug information you can log anything here you can print anything here you can do whatever you want but this is just a thing that you can use in your applications uh with the goal of turning this on and off so right now I have my ADD function works okay if this is some complicated function I can add a debug tag on top of it and of course this is a problem for what reason exactly yes of course because I missed the asterisks so like this like this it should work there you go and now I get the function call with some printing here calling function at with arguments 10 20 and keyword arguments nothing result was 30 and then I get the result so it doesn't have to be this specific decorator that you end up using but just the idea of having some decorator debug that you can comment out or you can add and remove the functions to run them in debug mode to get some more information about them this is very useful now of course you can also replace this with IC so with ice cream uh I have a video on that on my channel if you want to check it out but this is a very useful deckorator as well and finally I want to get to something I would call it more exotic not necessarily exotic but maybe not always needed but it's a rate limiter so let's say you have some function that you don't want to call too often in a certain period of time you can limit the rate of this function you can limit how often it can be called in a certain period of time by using a decorator called or whatever you call it but by using a decorator that reprs a rate limiter so let's say I have some function here fetch data and this function what it does is it download some data so [Music] fetching data from an API or something or using some system resources some GPU resources something that you want to limit and let's say we simulated this takes a second to actually be done so maybe received data now what I can do here of course is I can just call this function many times and I'm going to be able to do that fetching data receive data fetching dating data receive data and if I want to limit this what I could do is I could keep track of the function calls and the time and everything but let's say I want to do this every time I call this function so what I can do for that is I can Define a decorator rate limiter which is going to take calls and periods as parameters it's going to have a decorator again that takes function as a parameter and of course a rer that takes arguments and keyword arguments and then we're going to say that we want to keep track of a call list of a list of calls so let's say that we're going to Define that list up here last calls is going to be equal to an empty list we're going to make this list non-local so that we can actually access it into wrapper so non-local last calls and we're going to say right now we have the following timestamp time time and we're going to say that we want to get all the function calls that are in a certain time period and this time period is going to be the following call time for call time in last calls if the difference between the Tim stamp right now and the timestamp back then is smaller than period so this is all in seconds so if I say period is five that means I have 5 seconds and I can say okay if B basically this list comprehension is going to give me all the function calls that happened in the last 5 seconds and what I want to check is if the length of that list is above the number of calls so greater than the number of calls I'm going to raise a runtime error saying um limit exceeded or let's say rate limit exceeded try again later and if I succeed in running the function I'm going to of course append this function call the current timestamp to the list so I'm going to say last calls append now and then I'm going to return the function call I'm going to actually execute the function call and return the return value and here I'm going to return the wrapper and here I'm going to return the decorator so if I run this now with the decorator active by saying uh what was it rate limiter let's say I want to allow for two calls in 5 seconds so let's say calls equals 2 period equals 5 and then I want to activate this rate limiter if I run my script now you're going to see fetching data receive data fetching data receive data and then boom failed because I'm trying too often I'm trying to do this uh within 5 Seconds too many times so I have to raise the exception if I say that I want to have two calls per second it's not going to be a problem because I am doing this to slow so I can keep going keep going keep going nothing is going to limit me because I am allowed to do two calls per second now if I change the sleeping time to 0.1 this is going to become a problem because I'm going to exceed the rate limit that's the basic idea of this this makes sense if you have some function that uses some resources or uh yeah uses some resources that you want to limit this can be quite useful even though as I said this might be a little bit more exotic and less Universal so that's it for today's video I hope you enjoyed it and hope you learned something if so let me know by hitting a like button and leaving a comment in the comment section down below and of course don't forget to subscribe to this Channel and hit the notification Bell to not miss a single future video for free other than that thank you much for watching see you on the next video and bye [Music]