Expert Python Tutorial #3 - Metaclasses & How Classes Really Work

Key Takeaways

hello everyone and welcome back to the Python expert tutorial series what we're gonna be doing in this video is covering metaclass is a fairly complex but very interesting topic in Python so we're gonna start from the very beginning and start understanding how classes are actually created instantiated the way they work on a lower level in Python and then from there we're gonna move into what medic classes are how they actually work and why we would even want to use them now I'm just gonna preface this video by saying meta classes are an extremely complex thing I'm only gonna be showing kind of the basic implementation of meta classes here if you really do want to use these you're gonna have to look up the documentation and implementation details on your own and there's very few instances in which you need a meta class a lot of what I'm about to show you can be accomplished in other ways and typically you're gonna want to use another way other than a meta class just because these are seen as very complicated and almost sometimes bad practice depending on what you're trying to do so these should really only be used by experts I mean I'm showing them to you guys you can get an intuition on how they work and if you see one you'll understand what it does but you shouldn't really be putting these into your own code and using them unless you absolutely need to so before we get into the content again a quick thank you to kite for sponsoring this video series they have a download link in the in the description down below you can download them for free they are essentially a very good Python autocomplete so you can see if I start typing some things like this we get a kite completion thing popping up and anyways that's what they are they're free you can download that from the link in the description all right so let's go ahead and get started now the first thing that I want to do is really discuss how classes work and what classes are so we saw in the previous example and I believe two videos ago I did something crazy like this right and I made a class and I defined it inside of a function and then in that function I actually just returned the class itself and this is totally fine this is valid syntax you can see I just ran the code down here we don't get any errors that's fine the reason we can do something like this in Python and not in other languages is because in Python classes are actually objects now the property of an object essentially means that we can interact with it at runtime we can pass it around through parameters through variables we can store it we can save it we can modify it we can interact right that's what object it's now a lot of people are gonna say well how is a class an object I thought a class created objects for us well that's very true a class that does create objects for us but that doesn't mean that it's not object itself and if a class is an object if you believe what I'm saying and I'll prove it to you in a second then that means we must have had some higher-level class that created that object for us right just like if I make an instance of hi can I do something like that that means we must have this class high such that we can make an object you know and have the instance for it so what we're gonna be getting into here with meta classes is the basic idea is a class defines the rules for an object right defines the attributes the parameters the methods the things that are allowed the operations that can be performed that's what a class does for an object what a meta class does is define the rules for a class so when you create a class you will use the meta class to create it this happens automatically don't need to type anything specifically but we're gonna hook into it later and see how it really operates and that meta class defines how this class is created and that's the basic concept here so I'm gonna make another class and I know this is confusing but as we go through you're gonna start to slowly understand what these examples and all I'm gonna do is just print an instance of this class so let's look at it we can see this tells us this an object main test object at some gibberish location so now let's print above this just the class itself okay look at that class main test now the reason we're able to do this again is because this is an object so I know this doesn't explicitly say object but this class is an object now I want to look at something else so we know we have this method in Python called type and I can look at the type of a class or an instance or an object I see what it is so let's look at it when we do the type of test we get that is in class mean if I were to print say the type of an integer right like the type of - let's have a look that's class int right and we can print you know the type of whatever we want when you print the type of a function so we say define func like that and then just we'll just say pass and and we do funk what's that that's class function right so that exists in a function so this is an object this functions an object everything in Python is an object and that means they have a type and neighbor created somehow so what is the type of a class well let's look at it so we'll delete all this and then with this print the type of and I somehow get insert accidentally there of test so what do you think this type is is it of type test is this class of that type no it is of class type now I know this seems ridiculous this means nothing to you right now we're gonna dive into what this is but essentially this is what we would refer to as the metaclass so this type is what essentially defines the rules and creates this class for us we type this class syntax we will call a type constructor using the different things in our class to make this class object that we can then use to make objects and use to interact with in the program so what is type how does this type thing work well if I told you that this class all this is doing this syntax is just calling this type class then we should explicitly be able to create our own objects without typing out the syntax like this we should just be able to use the type class itself which is true and what I'm about to show you so I'm just create a class test it has passed it has nothing in it right now I'm gonna show you how we can make this without actually having to type this kind of syntax and writing out class in fact and this seems ridiculous I agree Python should have done a better job when they define this but I believe it something to do with just backwards compatibility we can make a class by doing this and just watch and then we'll talk about exactly how it operates so this line of code is completely equivalent to this these two things are completely equivalent there's absolutely nothing different about them and I can use tests here the same way I can use tests here just like I can create an instance of tests I can create an instance of this other test so let's do it let's let's just prove it to you so if I print test and make an instance boom main test object works totally fine no issues with that what else can I do I can print the class again same thing we have that object totally fine that's because this function here essentially creates the class for us using these different arguments a name this is the internal representation of the class any basis which means anything that we inherit from just like a superclass or a parent class and then any attributes so let me start showing you what I mean and then hopefully this should make some more sense so I'm gonna set an attribute of five I'm gonna say key people's test make an instance and just print that attribute your like this of T dot X so just like this works on a regular class we can do that we get our value five now just like in regular classes as well I can define attributes outside of the class so they say something like T dot W why why not set that equal to let's just say hello want to print W why sure let's go for it oh t wy one second guys oh t is not defined so that needs to go above so I can explain metaclasses but apparently cannot just create regular classes okay so that's totally fine that works we have T equals test and then Twi equals hello we can print that out tribute that's totally fine so that's how we can create classes using type now what about methods what about sub I'm going to show you all that so we'll do that now and then we'll get into actually meta classes and how this is even useful to us so what I'm gonna do is make another class it's gonna call this class foo let's just make a method in here just called print actually not print let's do like show and then all we'll do in here is take one attribute we'll say self and just print hi why not right just do that so if I want to now have an inheritance what I can do is simply type the name of the class a comma just to make sure this registers as a tuple here and now test one hair it from foo which means that if I want to use the method show I can do that and I can run that like that and we get that now I shouldn't have printed that out because I forgot that wasn't a return value but you can see we get hi that works totally fine this is how you do this right now what if I want you to add methods right so to add a method is a little bit different but essentially what you gonna do is just define a function that will be the method signature so in this case let's say you know add attribute put self like that then here we just say self dot Z equals nine why not right we'll do that and then in here we'll define our attributes we'll say add underscore attribute cooling guess what we're gonna do add attribute no brackets now make our instance let's call T dot add attribute and then let's print T dot zip like that let's look at this and there we go we get the value nine that works fine so this is how we use type this is how we create a class and this is very important to understand as we get into the next part of the tutorial here because if you can understand how we use this to create our own classes then you're gonna understand the next part and why when we have a syntax like this all this really does is pass that information to another class called type that creates the class for us and returns an object that represents that class I know this was a lot I hope this you guys understand at this point now we're actually gonna get into metaclasses this was just the underlying information we need to understand alright so welcome to the next part of the tutorial we're gonna be doing now is actually getting into metaclasses and showing how they work so again the basic principle here is to remember that a meta class is above the classes you're creating yourself so if you create something like class dog right and you do that what's actually happening is all this information that you type out the syntax gets passed to some meta class the thing above it that actually takes that and returns to you an object that represents that class and that's the whole point here so what we're gonna do is actually make our own meta classes so rather than using that built-in type class and you can see on all of our objects when we looked at the type of them they were of type type we're simply going to make our own meta class that inherits from type so does a very similar thing but we're gonna change slightly how objects are constructed so you can really understand how this works now you might get lost a little bit the beginning when I start typing some of this stuff out but hopefully you should start to follow along as we go through and explain how all this works so let's get started I'm gonna start just by creating a meta class I'm gonna call this met now whenever you make a meta class what you have to do is make it inherit from type not necessarily but for our purpose that's what we're gonna say it has to inherit from type to make a meta class the reason for that is because type does some other things when it's creating an object right and if you don't inherit from this well your class is just gonna be not a meta class now because it's gonna be using type to create this kind of class if that makes any sense whatsoever but hopefully that does what we're gonna do now is I'm actually gonna define a new method like this and for those of you that don't know what new does as a dunder method or a magic method like we talked about before essentially this is called before the anit method so this is the first thing that is always called when a object is created or in stance or whatever you want to call it this is always called first so what you can actually do here is hook into this and modify the way an object is constructed by changing what happens in this new method whereas the Annette's all the init is doing is after this new method gets called and the object is constructed what it does is it initializes it changes the values that take some parameters in like that's what it does right whereas new it's just called before hand to modify the construction of the object okay so what I'm gonna do in here as a moment to take self I'm going to take class name and remember this is what's gonna be called whenever we create another class that uses this as a meta class or it's a class name we're going to save faces and bases is what we put remember in that type argument when we want to do inheritance we have the name we had the basis and we have the attributes and guess what this last one's gonna be it's gonna be attributes so essentially we're saying ok when we create this new method the construction of a class which happens in this meta class needs to contain the class name the bases and the attributes so that that way we can look at them we can do something with them and we can return to you a class that has those pieces of information right as an object so what I'm gonna do now is simply just return the type of in this case it's gonna be class underscore name basis and attrs now this is very simple we're not really gonna do anything other than this but I'm else we're just going to print out the attributes so you can see what they look like ok now what I'm gonna do is gonna make another class we're gonna call it dog this is gonna have a meta class that's equal to met like that and then in here all we'll do is just say X equals 5 y equals 8 just so we can look at some things when we print these attributes out and then we'll create an instance or dog so we'll say D equals dog like that and run the code alright so look at this what is happened well actually I'm even gonna get rid of this and just show you that even if I don't create an instance this still runs where you can see no instances this is still running the reason for this is because the default meta class of dog is type right that's what it runs and that's how class is created we've done is we've overridden that we've changed it to our own metaclass and essentially we've added our own little piece here that just prints out the attributes and then returns to us an object and member this is an equivalent way of creating this class by just literally typing this line and this is what happens this is what we get so we can see that we have the attribute module is equal to main the cual name dog so that's the name of this class right and then we have our attributes so X 5 y 8 that we've defined down below now what if we define a method call to low and we print say hi inside of here well then you can see we have dog dot hello at some memory location that's mapped to by hello and that's a function so that's how this works now knowing this information metaclasses can be very powerful because for example let's see like I can hook into the construction of this class and I can modify the construction of it if I want now this is something you would never do what I'm going to do is actually change all of the attributes that I have here to be what do you call uppercase so I'm gonna change this X to be a capital X this Y to be a capital y and this function hello to be capital hello so all capitals I won't bother changing I can't change these underscore methods but I'll show you how we do this okay so let's start programming this so what I'm gonna do is I'm going to make a blank diction I'm just gonna call it a there's notice down here that all of my attributes are actually in a dictionary right so what I'm gonna do is modify this attributes dictionary based on the attributes that we're passed in and make them all Capitol assuming that they're not they don't with two underscores so I'm going to say for item in ET TRS dot items and I'm not sure the syntax is 100% correct so there might be a mistake here we're going to say if item dot starts with and in this case it's gonna be underscore underscore then what we'll do is simply just add it back into the dictionary actually yeah we'll just add it back into the dictionary so we'll say a in this case item equals attrs and then item like that so essentially since this dictionary the way this is gonna work right is we're just assigning this item to this new dictionary it just literally the same thing right okay otherwise so if it doesn't start with a double underscore we need to change it to be uppercase so we're going to say a and in this case item dot opera-like that equals attrs item then we'll change the attributes here to be a which will be our modified attributes and that means when we return this dog class this should work assuming I haven't made a mistake which is likely but let's look at it and let's see what we got wrong all right so I guess I've forgotten the fact that when we do something like this I need to actually loop through with name value rather than just item so name will be the attribute name and value obviously the value mapped with it just means we're gonna have to change a few things here it'll say name so having to do that we can change that to Val you could say name and this yeah some name here this will be Val I think that's and I think we should be good on that front let's run that and let's have a look at this error named Val is not defined of course I've made it value let's run and there we go we get good okay so let's I'd haven't proved to you really what happened but let's just was step through this coax I did make a few errors so might be confusing so we started by printing the attributes we then made a blank list a blank dictionary story that's gonna represent our new attributes then we've looped through all of the attributes so we just do dot items to get the name and value and we said if the name starts with a double underscore just add the proper value back in otherwise what we'll do is add the uppercase attribute with that corresponding value so we've changed them to upper to uppercase essentially so let's actually print the values down here so the new value just have a look at the difference and you can see if we scroll up here that notice hello has been changed to cap hello X to capital X and y to capital y so now let's do another proof just to make sure that all of you believe me that this is okay to do and let's go down here and say D equals dog create instance and let's try to say see what DX is so just print DX well what is the error let's look at this here dog object has no attribute X what's but we defined X up here that's because we've changed it to be a capital X and notice that when I do that that's totally fine and same goes with below I can't access that but when we change that to capital below we're totally fine because again we've modified the construction of the object all right so now that we kind of understand how metaclasses work we've seen that we can hook into them we can change the way that objects are created we can force certain constraints I can change all of the attributes to be capital I can do whatever I want right I could change the basis I could remove an inheritance if I didn't want that to be allowed essentially this is why they call it magic because with this kind of hook into the creation of classes you can really enforce quite a bit of constraints on how classes are created so for example if you want every single class in a specific module to never be allowed to use a certain attribute or to only have or to have to have that attribute you could set meta classes for those are for that actual specific module which I'm not going to talk about now and then when that modules run it will check the metaclass right it will go through the metaclass that's how we create the classes itself and make sure that all of these classes kind of conform to that structure that you've defined now this is specifically nice for when you're writing kind of library code and you want your user code to be very specific let's say they're inheriting from a specific type you could check if they're inheriting from you know whatever type it is that you want if they do inherit from that type you can enforce the fact that they have a specific function or specific method in that class that's needed to make that work so that's some of the things that you can do with meta classes again they're very complicated there's not really like a need to use them so you don't necessarily need to implement them into your program but sometimes they're cool and understanding how this class construction works to me is even a better benefit using the metaclass itself so I believe there was one last thing that I wanted to cover here although it's not comment to me at the top of my head I mean I think that's pretty much it I don't know what more more I can show you guys you can have meta classes that inherit other metaclasses you can do very very complex things with that but I think honestly for now we're gonna wrap the video up at this I hope you guys enjoyed I hope you understand now how classes are created how a meta classes work and what the heck they even are and I hope that you understand that you don't need to use this and the point of this video is to introduce you the concept and make you more familiar with the way that classes and objects actually work in Python so anyways that has been it I hope you enjoyed make sure you leave a like if you did subscribe to the channel and I will see you in another expert Python tutorial