Learning Python Feels Easy. Until It Isn’t.

Key Takeaways

Python is by far the most popular programming language in the world right now. If you look at the tobi index, the difference especially since past few years is staggering. It's become the main language of AI, machine learning, data analysis, automation. It's getting very popular for backends as well with fast API due to its large ecosystem of tools. And on top of that, it's really simple to get started with Python. But there's problem though. In my experience, the learning curve for Python looks something like this. Learning the basics seems really easy and you can build up your skill really fast, but then you enter a sort of no man's land where you're basically on your own. Python has very few safety nets. Use type annotations or not. Use functions or classes or whatever you want. You can override dunder methods to your liking. And then there's advanced stuff like generators and context managers and protocols. Going from beginner to expert in Python is actually really hard, much harder than for example Rust where you know once you survive the borrow checker ownership and lifetimes well writing a full-blown CLI isn't that much harder than a hello world program. Now, in this video, I'm going to focus on how to become better at Python after that initial honeymoon and the exact steps I would take if I wanted to become really good at Python faster. Basically, the things I wish I knew 10 years ago. This video is sponsored by Data Camp. Talk more about them later. The first step, the most important one, is to master the core. Most people try to rush through the basics, but these are actually really important. you need to make sure that you don't have them wrong because if that happens then basically everything that you do afterwards is going to get even worse. What is the basics? It's things like conditionals, dictionaries, loops, functions, things like that. Things that Python is actually really good at. So let's say as an exercise you want to build a dictionary that maps strings to their length but only includes words that are longer than four characters. So here I have a simple main function that has a list of words and then prints out those words. Very basic, right? But already if you want to learn the fundamentals, you need to know, hey, I need a main function. And the reason you want that, you don't want to just put this at the top level in your Python script is that you don't want to pollute the global namespace because this words variable is then going to be accessible everywhere and you don't want that. So that's why you need a main function. This type annotation here doesn't return results. it returns none. And then we have this little bit here that makes sure that if you run this script that it's actually going to call this function because that's not going to happen automatically. So that's a basic setup, fundamental setup of a Python script. And then if we want to do this exercise where we map these words to their lengths, well then we're going to need a dictionary. Let's call that a length map. And this is a dictionary of well strings to integers. And initially we have it as an empty dictionary. Also here you don't strictly need to add this type annotation but it actually clarifies the code. We now know that the length map map maps strings to integers. That's the dictionary. Now next step you might want to do in this case to solve this exercise to make this mapping is to use a for loop which of course Python also has. So for each word in the list of words, if the length of that word is more than four, then we're going to add it to the map. And then finally, let's print this length map. So this combines a bunch of different Python fundamentals, right? We have functions, we have a list, we have a dictionary, we have a for loop, we have a conditional statement here. And when I run this then you see we get Python and refactor which are both words that have more than four characters and the other words it ignores. A different way to do this is to rely on a built-in Python feature called a comprehension. Now most people when they talk about comprehensions they only think about list comprehensions but you can actually also do it for dictionaries and for sets. Many people don't know that, but it's a core fundamental thing to be aware of. So instead of this for loop, what you can do is basically refactor this into a dictionary comprehension, which is basically just going to be a single line of code. So that's going to be a word to the length of the word. So that's going to be the map for word in words. if the length of the word is more than four like so. And now I can remove this entire for loop here. And then this is what we have. Let's run this. We'll get exactly the same result. So that's in Python a really neat way to solve this. Now you don't want to use these types of dictionary comprehensions or comprehensions at all if your conditionals are going to be very complicated because then this becomes quickly very hard to read. In many cases, I actually like using a loop. But knowing both of these tools and having them available to you as a fundamental skill is really important. Now, the next step is that once you know that core, then you want to make sure that your code looks and feels like Python. I already gave sort of an example of that where we want to make a switch from these for loops to comprehensions. Again, not always, but in many cases, they can be a great solution. But another example of where you can make things more Pythonic is let's say you want to print out these words with their index. So you might start with something like this. So now we have the indices and then we're going to print that index and we're going to print the particular word in the list like so. And when I run this then this is what we get. Now, this works, but it's not really the Pythonic way of doing this because Python has better support for this kind of thing, namely using the enumerate function. So, instead of using this, I'm going to use enumerate. Remove this parenthesis. And that's going to give me the index and the word. And then I don't need to access it like this, but I can simply print it like so. And this is going to give me exactly the same result, but it's more Pythonic. It's more the way you should do things in Python. And it's not just cleaner, it also avoids uh index errors. It's in my opinion improves readability and using these types of features also shows you understand the language really well. Once you know those fundamentals, once you know how to roughly write Pythonic code, code that works in a Python setting, then the next step is start building some tools. Don't immediately jump into giant web frameworks like Django. That's going to be way too much. It's also going to put you like in a box because it decides a lot of things for you. Just start building some tools. For example, you could write some tools that are helpful to you as a developers. Maybe a couple of scripts that automate some things like renaming files or scraping some data. These force you to use things like file IO, some standard library modules, error handling, that kind of stuff. For example, here's a very simple script you could write that you could run in a folder and then it's going to rename JPEG files from this to this. Building these type of things is nice because it allows to use path lilip which is a really good library. It teaches you some basic file operations. So building little tools like this is actually pretty nice and I use Python myself a lot in this way where you know if I have some folder I need to organize some stuff I just click quickly write a script to deal with that for me. For example, a couple of months ago, I wrote a simple Python script that takes my Final Cut Pro libraries. I'm using Final Cut Pro as a video editing tool for all the videos on this channel. But whenever I share the library with my editor, uh there's going to be some settings that are changed like the where is the cache folder, where are the assets located. So, I have a little script that basically opens the Final Cut Pro project files, an XML file, I believe, and then replaces those folder names with the correct ones on my machine. So, I don't have to manually click through the thing in Final Cut Pro. I can simply run my script and it does it for me. So, these type of small things are really nice ways of getting better at Python quickly without diving in too deeply at the same time. Now the fourth step is to learn more about how Python actually thinks. Python is based on a certain object model that's actually very elegant and consistent. But it's really important for you as developer that you understand how this works. Most importantly know that everything in Python is an object including functions. It's also important you know how scopes work, where are certain variables going to be available and how Python passes references typically and not copies of things. That can lead to problems if you're not aware of those things. For example, if you take a look at this main file that I was working on, let's say I add a function here. Let's call that list words. And that's going to get a list of strings. And what that will do is basically this part. So let's copy that right here. Now obviously I can simply call this function like so. And then when I run this, this is what it does. But if you know that functions are basically objects, well, you could also do things like this. We can in essence create a reference to this function and name it something else. But then we can simply call that as a function just like before and it's going to give us exactly the same result because functions are basically objects that you can refer to. And in essence what a function is actually an object with a call dunder method. So you could even create a class called list words and we could define a call dunder method where you can pass some arguments. Let's say in this case we want to pass the list of words like so. And then we're simply going to do this. And now we have an object. Oh, I actually of course need to copy the entire thing like so. There we go. And then once you have this class, you can create an object like this list words function. And that's going to be list words like so. And then I can call this as a function and pass the list of words. And then this is what we get. So knowing that this is how Python thinks, how Python works really helps you understand the language better and also come up with better solutions to your problems. Now you might think, hold on, that's all easy for you to say, but learning all those fundamentals takes a lot of time. In fact, to learn them efficiently, you need to actually practice them. Just watching tutorials like this one is good, but it's not enough. It's like you're trying to learn playing the bass guitar by watching videos and not by actually playing the bass guitar. In fact, many studies in education and programming show that passively watching video tutorials doesn't help you remember the information as well as using an interactive learning method where you program and build hands-on projects as you learn. Honestly, I recommend Data Camp for beginners learning Python because the platform is super interactive. It lets you code right inside their environment, getting instant feedback as you go. I'm excited to tell you about two of their Python learning tracks because they're genuinely something I wish I had when I was getting started many years ago. I've been recommending Data Camp for years. If you're just getting started, I suggest the Python programming fundamentals track, which is designed to teach you the basics from variables, functions, and all the way up to working with data. If you're looking to push things to the next level, get job ready as a Python developer. They also have the associate Python developer track, which dives deeper into realworld skills like working with APIs, data structures, and debugging. Both tracks are hands-on, projectdriven, and designed to actually help you practice not just passively, but interactively learning. If you're serious about learning Python the right way, don't miss out. Check out the link in the video description to get started for free with Data Camp. The next step in becoming a Python expert is to use abstractions and types to better understand your data. Python is dynamically typed, but that doesn't mean you should just write unstructured, loosely defined code. So, I've already shown you in previous examples in this video today that I'm using type annotations all over the place. For example, here I have my list of strings or here I specify the return type of a particular function. So doing those things really helps you understand better what the structure of your data is and what the contracts are, what methods and functions expect in terms of arguments, right? I can just look at the signature of this list words function to know, oh, I need to supply a list of strings because that's what this function needs. I don't have to look at the body to understand that this is what words is. On top of that, using type annotations like this forces you as a developer to think about the structure of your data. In my opinion, that's the most important role of type annotations. Now, you don't have to go full-on Java style object-oriented programming in Python. Dock typing is actually how the system works. And that means that it will look more at the structure of the type versus having to explicitly define it everywhere. For example, let's say I want to turn this into a bit more objectoriented code with typing. What I could do is I could create a class called let's say transformer and that's going to have a method uh transform and that's going to get a list of strings like so. And it's also going to return a list of strings. Let's say for now what this does is uh return word. So now this just returns the identity. But let's say I want to reverse this. Just going to remove all of this. I'm going to create my transformer like so. And let's run this code. And now we have all the reversed words. And then let's say I create a separate function with the great name do something. Don't really call your functions like this by the way. And let's say that gets a transformer object and it gets a list like so. And then this is what it's going to do. Right? So this does exactly the same thing before. And as you can see I'm using type annotations everywhere. And actually it's not complete because do something doesn't return a value. [clears throat] So I also need to add a return type annotation of none here to be strict. All right. So these are type annotations. But you can take this a step further. For example, in this case I'm just iterating over the my list argument here. So it doesn't need to be list. I just need something that I can iterate over. So in this case I can also be more specific and I can import from typing the iterable type and then it's not a list of strings but this is actually an iterable and when I run this well of course the result is going to be exactly the same just changing the types but now what I can do is I can also turn words in let's say a set like so and then let's [clears throat] also turn this into an iterable of completeness and this also works just like before I get exactly the same result and I don't get any error here because I'm being more precise with my typing. Now another thing you could do is that the transformer in fact we can also make that more generic and that's by using a protocol. So let's first change the name here. So this is actually going to be a reverser. reverser. Weird word. But then what you can do is create a new class transformer which is a protocol. And this is going to have that method like so. And then in my do something function, I can actually say that I get a transformer here, but I'm passing a reverser, which is a specific way of transforming these words. And because I made the type more generic here, I can also create let's say a class uh doubler which has this same method. But what I'm going to do is return an fstring with the words twice. And now I can create a doubler like so. I don't need to change anything in this function. And let me run that again. And now we see we get this as a result. And so knowing your way around types and abstractions is a really helpful skill to have. So once you know the fundamentals, once you understand how Python works, once you know your way around type annotations and abstractions, then it's important to start thinking more like a software designer. That means you need to learn about things like separation of concerns, dependency injection, uh the keep it stupidly simple or the you ain't going to need it principles, things like this. And having all of this in the back of your mind really helps you write simpler code and code that's much easier to work with. And I've already shown you examples of this in the video. For example, the do something function has a transformer object that it gets as an argument. So in essence, this is dependency injection. We're [clears throat] injecting the dependency on this transformer object to the function. We're not creating it here. And by being mindful about that in your code, it means you set up things much nicer. You'll end up with probably a main function where you create all sorts of different objects, patch them up, and then you run your code. And that also makes things easy to test. For example, if I want to test do something, well, then I can just pass in a mock transformer and I can check that it actually does what it's supposed to be doing. Obviously, these are very simple functions, but in a more real life setting, this is going to help you out a lot. So, know all of those design principles and patterns. And actually, I have a whole playlist talking just about that with lots and lots of videos covering basically every aspect of software design that you need to know about. I've put a link to that playlist in the description. Now our next step in becoming an expert Python engineer is that you don't just write your scripts but you actually also write tests for them. If you want to go all the way you can actually write those tests before you write the code and you're basically doing testdriven development but just starting to write test. That's where you move from writing code to just writing code that's actually uh maintainable and easy to manage in a production environment. Now, I recommend you start with Piest that works really well and even though it's not in a standard library, it's much better than the built-in unit test library. And it's important that you learn how to use its basic features like fixtures, parameterization, mocking, those kind of things. Now, I don't have the time [snorts] to dive fully into piest today. There's um several videos on my channel that talk about piest. But basically, if you have a simple function like this is prime, you can start writing tests for this. A very basic thing you could do is write a test is test uh zero is prime for example. And there we're just going to assert that is prime zero equals false because 0 is not a prime number. Now you can also instead of uh writing it like this. I like this way of writing it for test by the way but you can also write it like so. And now let's run piest and we get that our single test passed. But piest has a lot of other features. For example, you could also write a more generic test is prime function that gets let's say a value of n and the expected value like so. And then we're going to assert that is prime n equals that expected value like so. and going to import my test like so. And then what I will do here is I will use parameterization. And that's actually really easy. I can just write park.parameterize and I want to parameterize the n value and the expected value. And then I'm going to supply the list of options. So for example two is a prime number. Three is also a prime number. So I can run this test. So if I have these two options already like so. And now when I run pi test I'm getting that two of these tests have passed. And obviously I can now add more options like uh 17 for example is also a prime number but uh 10 is not a prime number. And I can also test my special cases here like zero is not a prime number like so. And now when I run these tests, this is really easy to write more extensive tests. And if you really want to take this to the next level, there are also libraries out there like hypothesis that automatically generate these lists of tests to run that also focuses on adding the various edge cases. But already knowing these types of features is going to really help you write tests much faster. And having these tests actually really helpful because once you have them in place, you can refactor things and change things. And if you break things accidentally, then the tests are going to fail if you wrote them well. So you're going to feel more comfortable working on your code. Now finally when you're ready you can decide to go really deep and dive into the internals of Python and learning how those different dunder methods like uh it next call all work. Uh you can start writing your own context managers using enter and exit or with context lib. It's going to ultimately help you write better code and especially if you're building complicated libraries. Let's say you're building something like fast API. Well, then you're going to need type introspection and things like that. So, you really need to dive into the internals of Python in order to do that. Or if you're building something like Pyantic, you know, that's those are the type of things that you will need to know. You need to be comfortable with dunder methods and Python's internals. Now, there's one more thing you can do to become better at Python faster. But if this video helped clarify your path to mastery, hit the like button and subscribe to the channel. That way you won't miss any of my upcoming videos. Now what I've talked about in this video was to focus on the core, write clean code, understand the internals and build from that. But the final really important thing you need to do is to use the tools that Python already gives you. If you want real leverage in Python, you need to master the standard library. This is full of utilities that solve common problems without needing any external dependencies. modules like it tools, collections, phone tools, path flip. These are really essential. Just as a simple example, here's a script that can wrap long text and it uses text wrap which is a built-in library. It's from the standard library and it can already do stuff like this. When I run this, this is what we get. It just works. There's no need to add dependencies, third party packages here. You just need to know what Python already gives you for free. Now, if you want to learn more about that, I actually made a video covering 10 standard library gems that most people actually don't know about. You can watch it right here. Thanks for watching and see you next