Writing Python Functions Like a Mad Scientist

Key Takeaways

In Python, you usually define functions use the def keyword like I'm doing right here. You've seen this probably a million times. And on top of that, it's a really basic function that just takes a name and then prints out a greeting to that name. So, when I run this code, then this is what we get as a result. But Python actually has many ways to create functions, some practical, some questionable. Learning about them gives you a much deeper understanding of how Python actually works under the hood. So today I'll walk through a bunch of them, starting with the ones that I use every day and ending with one that's completely bonkers. Trust me, you're going to want to stick around for that one. The first way that I want to show you to create a function is a lambda function. And this is a oneliner function using the lambda keyword. As you can see here, a lambda function is a very simple oneliner that takes one or more arguments and that then evaluates an expression. And you can actually store this function in a variable even though uh that's not really a common way to do it. Of course, if you need a named function, then you should just use the dev keyword. But lambda is great for little anonymous functions that for example you might want to use for simple throwaway functions that you need to pass to functional programming tools like map and filter. Now there are some limitations to lambda functions. uh one is that it can only contain one expression, no statements, no multiple lines, etc. So don't use these for complex logic because it's going to be really hard to read. But ultimately, this code results in exactly the same thing as the very first version that I showed you. The second way to create a function is by using partial function application. I've talked about the fun tools package before, but in particular, funk tools has a partial class that allows you to take a function, apply already some of the arguments, and then it provides you with a new function. So in this case, I have a power function that takes a base and an exponent and it does base to the power of exponent. Very straightforward. Now, what you can do with partial is that you can actually take that function and apply already some of the arguments. So let's say I define another function called square that is always of course an exponent of two. So then I use partial to supply it with the power function and the value of two for the exponent argument like you see here and it gives me a new function square that I can then call with a single argument and then that's going to compute the square of the number. When I run this then this is what you get. partial function application is actually quite nice especially if you need to for example preconfigure some function and you don't want to write a special wrapper for it. Uh it can help you reduce some boiler plate. If you're writing higher order functions in other words functions that get or return another function as a result but if you're not familiar with partial function application it might be a bit harder to read. And of course this is not as flexible as defining a another full regular function. So take that into account when you use this. But in my opinion, partial function application is actually a nice tool and it's kind of underused in languages like Python. A third way to create functions is by using decorators and that allows you to define functions at runtime. So with decorators, you can dynamically modify the parameters or the result of a function by wrapping around it. And this is basically what that looks like. So what I did here is I created a decorator called print result or actually print result is a function that creates a decorator. Print result gets an argument in this case a formatted string and then it defines a decorator that basically prints the formatted string and provides it with the result of calling the function that it wraps around. If that makes any sense. Well, basically here is how you see uh how it works. So we have a double function that takes an integer, returns another integer and then performs some computation. What it does exactly doesn't really matter but I use the print result decorator supply it with an argument so that I can uh print the result of this function and then when I call this then it's going to print the result for me which is nice. So when I run this then this is what we get. But the nice thing is because now this is an argument I can now change this to something else like so for example and then when I run that again then it's going to use that formatted string instead. So decorators are pretty powerful for modifying behavior of functions that we use quite a lot for uh caching values for logging things. Now an issue with decorators that it's can be hard to follow. it's kind of hard to debug especially if you use multiple decorators in a row. It can introduce some unintended side effects. Uh it can mean that you do complex computations even if you're not aware of it. So that's another thing you need to be careful of with decorators. Now designing your functions carefully is really important. Not just what they do, but how they behave, how flexible they are, and how they connect to each other. If you want to learn more about designing clean and scalable Python code, I have a free design guide. you can download. It covers the seven steps that I use when designing new software from scratch, including how I think about designing functions and responsibilities. You can get it at r.co/design guide. Link is also in the video description. All right, the next way to create a function in Python is by using a class with a call dunder method because in Python basically anything is an object. So a function when you define it it's actually an object with a call dunder method and you can actually just define that yourself. So uh if you have a class in this case I have a greeter class and I define the call dunder method that means that an instance of this class is now callable and that's what you see happening here. So I have my called dunder method. I've supplied the implementation here. is the same as in the previous examples but and then in the main function I create a greeter instance and because it has the call donor method implemented I can actually call it as a function in fact it is a function in in Python this is actually how a function is defined it's it's an object with a call down method so this is how you would do that and let's also run this and as you can see we get exactly the same result as in the first examples now there's some interesting consequences of this. This means that functions can actually store state. So you can for example use it to keep track of calls or something like that. In fact, you can even take a function like this and actually assign values to it. So I could do something like main blah equals uh whatever and then I can uh print that and then when I run this then you can see it actually uh prints this as a result. So I'm storing now values in the main function kind of because it's an object just like anything is an object in Python. And you can even add methods to a function which makes zero sense but you can actually do it because it's an object. Now one reason why you might want to do this is that you may want to define a certain type. For example here I've defined a greeter function type that takes a name string as an argument. If you would normally define a function in Python, you would probably do something like this. My function equals and then we have callable that takes a string and that returns a string and then of course collable we need to import from the typing module. So that's how you would normally do it and then you can use this type throughout your code. But the disadvantage of defining a type this way is that you can't supply names to these things. So we don't know what type of string this is. Whereas if you do it this way and then you can use for example a protocol class or something like that then you can actually specify the name of the argument. Now it's a kind of contrived way of figuring out why this would be useful because of course this does add a bunch of boiler plates. This is much shorter obviously and defining a function this way is way less intuitive but you know there is some argument for doing it in this way. Now, we're already getting kind of in weirder territory. Now, the next few ways to define functions are still sort of normal, but keep watching because the last one is just completely crazy. The next way to create a function is by using XAC, and that's what you see right here. Now, it's kind of cheating because of course I'm defining or creating a function in the regular way except that I'm calling XAC, which calls the Python interpreter within the Python interpreter. So what happens here is we simply execute this source dynamically and then uh we can uh call that function because we defined it right here. So when I run this you see that this actually works. Now should you define functions in this way? Uh no probably not but it does work. It can be kind of useful for meta programming. There are some use cases where you want your users to be able to define custom code. For example, in an analytics dashboard, you may want to have the user write custom code to visualize data or something like that. But generally, it's not really recommended because this is pretty hard to debug and quite risky. Also, it's slower than a normal function definition because you're doing interpretation on the fly within your script. So, it's not really recommended, but it does work. The next way you can create a function is by using Eval. You can use eval for simple inline functions like what I'm doing here. Again, it's kind of cheating because I'm just defining a lambda function here, but I'm using eval to actually run that code. So, I'm evaluating this expression which is a lambda function and then I can store the result of that in a variable which is then a function and then I can call that function and that gives me a particular result like so. So basically the same pros and cons as with exac apply here. You need to be careful with these things because there can be security issues especially if this function is not defined by you but by the user of your application. The next way you can define a function is by using new class from the types module. And it's getting a bit weirder already now. So instead of writing a regular class with a call donor method, you can actually dynamically create one at runtime. That's what you see here. So I import the types module and then I'm calling types new class and I'm going to call that dynamic function. Uh there are some other arguments that you can supply here. Uh like the basis of the class if you if you need a superass for example. You can also specify the arguments. Uh and there is an exac body which is a piece of code that you can define where you specify what should be the body of the class. And it's a bit complicated how you have to do that. So uh of course because this is not like a normal way you would define a class or a function. But basically what you need to do is supply a function that then does something. In this case it updates the class object with a call dunder method and that refers to a lambda function that then does a certain multiplication. And since this is called dunder method that means that dynamic function is now callable when we create an instance of it. And then I can call that as a function. Complicated. Let's run this. There we go. It works. It's very contrived, completely useless, but you can actually create a function in this way. Now, maybe there are some situations where you need to do this. for example, if you need to generate many similar callables programmatically or something, but I'm not really sure. Uh, in any case, I definitely wouldn't call this readable unless you really need this type of behavior. Um, so if you know a use case for this, let me know in the comments. I honestly don't except for some really low-level internal thing. But this is child's play compared to the final way to create a function that I want to show you. Before I show you that, if you're enjoying this video so far, give it a like. It's a small thing, but it helps me a lot if you do that. It really does. All right. So, the final way you can create a function is by doing it manually from a code object. So, normally when you write a function in Python, you just use defaf, right? Easy. But deep down, every function in Python is actually made up of two parts. There is a code type object which contains the compiled byte code and all the metadata and there is a function type which wraps that code and it provides a callable interface. Now technically you can create a function manually by constructing a code type and passing it into a function type. Let's see what that looks like. So here I'm importing the types module again. Show you later on why that is. But what I'm doing is first I create byte code that loads a variable then loads the constant 42 adds them together and then returns the result and then I wrap that bite code inside a code type that is actually what is happening here very complicated finally I'm using typesf function type I suppli it with the code type object with globals. I give it a name and then we're going to store that in a variable. And now we have a callable Python function. And then when I call this, this behaves exactly like a normal function. And it what it does is that it adds 42 to the input. Let me run this. As you can see, it works. But there's a caveat here. This only works in Python 3.10 10 and earlier. In Python 3.11 and newer, Python's internals changed completely. There's new resume op codes that are required at the start of every function. Many instructions now have inline cache entries, and if you miss even a single bite, Python is just going to crash your program. Believe me, I tried creating a version of this code for new Python interpreters. I failed. I couldn't get it to work. If you figure out a way to do it, post it in the comments. But unless you want to learn more about how Python works internally, please don't do this. Leave raw bite code writing to the C Python developers or stupid YouTubers like me who accidentally fell into a Python bite code rabbit hole. Now, this was not an exhaustive video. For example, you can also define a function by importing code from a pickle file. You can directly import compiled pyc files. And maybe there are other options. I'm curious though, how many of these ways to define functions that you already know? Do you know any other ways that I didn't mention? Does this video entice you to start writing some bite code in your Python script? Let me know in the comments. Seriously though, when you're writing functions, there's actually a lot to consider. In particular, you need to design your function signatures correctly. I made a full video covering everything that you need to know. Check it out right here. Thanks for watching and see you next