Building a Python Debugger & Preparing for NumPy 2.0 | Real Python Podcast #189

Key Takeaways

welcome to the real python podcast this is episode 189 how does a debugger work what can you learn about python by building one from scratch Christopher Trudeau is back on the show this week bringing another batch of Pyers weekly articles and projects Christopher shares a two-part tutorial on building a debugger in Python creating a simple one requires less code than you might think we also talk about an article from Edmar Turner trow about how to prepare for the upcoming changes to numpy the new version is not Backward Compatible and will require some inspection of your project dependencies inmar includes advice techniques and tools for updating your code we also share several other articles and projects from the python Community including a couple of news items a discussion about managing advice as a new developer moving to python as a former R developer building a marov chain to to generate readable nonsense optimizing fonts to individual glyphs on your website and a project for working with units of measurement in Python this episode is brought to you by posit connect posit is dedicated to open-source data science tools and connect helps teams manage all their data science publishing learn more at POS os.it sreal python all right let's get started [Music] the real python podcast is a weekly conversation about using python in the real world my name is Christopher Bailey your host each week we feature interviews with experts in the community and discussions about the topics articles and courses found at real python. after the podcast join us and learn real world python skills with a community of experts at real python. hey Christopher welcome back hey there so we are starting with some news this week we also have a discussion which I'm excited to talk about let's go ahe and start with the news this week sure so we've got a couple quick items the first is a call for papers for D Jango Con Europe the Call's open until the end of February so you've got some time to procrastinate the conference this year is in Vigo Spain which seems like a much sunnier place than the winter misery both you and I are experiencing right now so I'm jealous yeah the second chunk is and this feels like a little bit broken record I feel like I've said this a few times before but python 313 Alpha and this is Alpha 3 this time we're getting to the point where almost every single episode I'm announcing another python coup week so so they're obviously very hard at work getting that stuff going uh so if you're if you're playing with the alphas go get the latest one yeah so there's a lot of code that's getting updated these days and the one that I wanted to cover actually I've got kind of a data science Centric episode this week in in some ways some of the topics that I got interested in this first one is from our friend edar Turner trro who we've mentioned multiple times on the show and the title of it is numpy 2 is coming and the after col little subtitle is preventing breakage updating your code it digs into multiple sort of pieces of advice here the first is ways that the new release might break your application then the importance of pinning your packages which we've talked about in the past but we'll dig into a little bit more here and then ensuring your application doesn't install numpy 2 until you're ready and then he kind of ends with some really great advice about upgrading your code to support numpy 2 and has actual suggestions on tools and techniques for doing that so the whole numpy 2 could break your code they're directly changing the way things are named and very specific apis so ways that it could affect your code is that you're addressing those apis directly and so you might need to kind of look through your code and look look at that another way that it can break your code is of course direct dependencies these are you know things like pandas using it and so forth um libraries that that use it becoming incompatible with your code and then there's sort of that third layer of indirect dependencies dependencies of your dependencies sort of turtles all the way down of what numpy uh where it could be in your stack there so he gives an example of pyit image which currently is incompatible um has a packaging metadata declaration in it talking about pinning again of numpy greater than or equal to 1.22 so that that actually would mean it would go right ahead and install numpy 2 then realizing that the pinning and that kind of idea of how to organize it and so he digs into that talking about need depend dependencies a them again we've covered multiple times but here's a very specific example here you can always pin your dependencies with like equal equals to say I want this very specific one or you can do things like greater than but I'm guessing a lot of people may not have done something like num bu less than two or something like that and he goes into talking about in your dependency list of like something like a Pi Project tomel file or a setup.py or even lock files and he gives examples of you know screenshots of code kind of showing you how to do that and ensuring that numpy won't get installed He suggests adding like a little hashtag comment there for now comma make sure numpy 2 is not installed and then numpy less than two that's step one he's suggesting for ways to approach this step two he talks a little bit about waiting for your dependencies to support numpy 2 and then step three upgrading your code in your dependency he includes a link to numpy's migration guide for numpy 2 and there are a lot of changes of like fundamental types inside there C float is now a numpy complex 128 eight and then there's a float which is now the numpy float 64 so very much specifying like 32bit 64-bit 128 bit things like that inside there so uh definitely look at it if there are types that you've been using numpy wise it's there's potential change and then there's a lot of sort of deprecation of things that I think we're you know np's been around so you know it was maybe python 2 and other things like that that could have been happening there the nice piece at the end of it is he suggests using the popular linter of choice today Ruff to check your code and we've mentioned Ruff multiple times it's a rust based linter that's a faster alternative to flake 8 and pyant and it also includes like features like black and iort and stuff like that he gives advice and shows examples of how to check your code with it it has a dash Das preview flag that you can use for checking for incompatibilities and he walks you through doing that and even has a feature where it can actually give you suggestions on how to rewrite it or fix it for you which is pretty cool so thanks again edar all right so what's your first one starting this week with uh something called using a Markov chain to generate readable nonsense and it's by Ben hoit Ben's been featured in pie coders before but I think this is the first time we've actually highlighted one of his articles in the podcast let me break the title down a little bit first off a Markov chain I'm going to quote from Wikipedia which isn't going to help but we'll start a Markoff chain or a Markoff process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event that's a mouthful and it boils down to essentially a state diagram where deciding on what the next state is is based on a probability and the current state so you're not allowed to use any past State information that they influence the decision these kinds of models come up in probability Theory a fair amount what the article describes is using this kind of state machine to generate text that is readable that readable doesn't necessarily mean sensible it ends up being a bit of a word salad thing right but with the right set of tokens it's a lot closer to readable than say randomly picking words out of a dictionary this might sound like a strange exercise but it's related to things like simple text prediction on your phone there are a couple different ways of doing that but this is one of them the program that he talks about takes a chunk of text finds pairs of words known as byrams and then calculates the probability of all possible words that follow the pair based on some input text or training data you can also do this with trigrams and other things but adding length doesn't actually make things better necessarily so byrams are often good enough as the first example in the article that Ben uses is actually the five of The Ten Commandments from abrahamic religions and the input here so that's you know starts with things like Thou shalt not kill Thou shalt not commit adultery Thou shalt not steal Etc a lot of Thou shalt Nots here so you end up with thou shal being one of the bogams and of course every word after it actually is not so that ends up being a 100% probability whereas a Bagram of shall and not so after thou uh it's followed by by different kinds of words kill Thou shalt not steal Thou shalt not commit as in commit adultery bear as in bear false witness and a couple instances of coveting everybody loves to covet so the frequency of those words is then used to calculate the probabilities in the Markoff chain so if you start with a random Byram and then get the next word then you use that next word to select another Byram and you keep going until you've got some stopping condition and if your input text is small like say five of Ten Commandments your output is going to look an awful lot like your input yeah so this is you know it it's kind of interesting when when you look at the example almost every sentence starts with the thou shal and then you it feeds things together and it almost looks like what went in but of course that's not where the fun is the fun is in starting to train it with larger data sets for example Ben ran the algorithm on his own blog posts and generated the following especially if you need to return a value but but are now very flag almost five times as fast so that's like I said that's nonsense but it's almost there it's almost readable and so you can see how this can be useful in things like text prediction and it's very very small it doesn't require a large training set it doesn't require uh a lot of processing power and this is why it's still used in things like your phones so like I said nonsense but still better than picking words from the dictionary at random of course because this is a python show the article has a python implementation of the chain and she owes different inputs as well he uses Alice in Wonderland and War of the Worlds from Project Gutenberg and you get different results depending on the input text um would you say people have done this as like a game like you you talked about your phone before yeah start with like three words and then just like hit the autocomplete every time and it building out sentences is yeah essentially that that would be you'd be you'd be continuing the Markov chain yes if you're if you're playing that little game to see what happens with the with your auto complete that's it's the same kind of thing right it's it's a it's a mad libs kind of yeah exercise yeah okay the problem itself actually comes up in Computing a little bit he gets into this into the article as well is because there are a couple well-known books that use the algorithm as a basis to teach different things like optimization or whatever else because the algorithm itself is relatively small but large enough that you can play with it right so uh so this is why you tend to see it in a few different places yeah so yep if you're interested in probabilities this is a neat little experiment or like I said if you just want to have a little bit of Mad Libs fun a way of uh whing away your afternoon particularly if you're stuck in snow Banks like we are uh you know go go generate some nonsense yeah the implementation is like what is it 24 lines with comments so that's a a pretty pretty quick uh implementation to get you going and checking it [Music] out pait makers of shiny and quto and formally called R studio is the public benefit Corporation dedicated to making great open- source tools for data science let me ask you a question are you building awesome things of course you are you're a developer or data scientist that's what we do and you need to check out posit connect whether you're building with streamlit shiny fast API Dash or quto make deployment the easiest step in your analytics workflows and ease your it teams workloads with posit connect you can try posit connect for free for 3 months by going to pos. slre [Music] python again kind of continuing the theme of uh data sciencey kind of stuff I think I've mentioned multiple times on the show that I started python right about the same time that I was in a marketing department in in a bank and they also were an R shop the r programming language and so I ended up kind of learning both around the same time I became a bit of a fan of R I like certain things about it I know there are a lot of people that are in one particular family or another often the biggest issue is you want to accomplish something that is only available in one or the other and so sometimes you end up having to kind of cross between the two this article by Emily rerer I thought was really great for anybody who is in the r world and is interested in coming to python it outlines a whole bunch of libraries that still have the r like sort of feel to it and I agree that R definitely has a very specific feel to it I'll explain some of that as it goes along if you're not familiar with it but I found this really useful and I think anybody coming from that area of data science who's kind of interested in Python uh this might be a way to help you transition and get more into the pythonic workflows here so she starts out with uh this area of her blog and her blog's actually really great too if you're interested in data science topics and I think I'm going to try to reach out to her get her on the show to talk about some of that stuff so his first quote from the beginning of it is learning a new language is easily enough done programming one-on-one Concepts like truth tables and control flow translate seamlessly but ergonomics of a language do not the tips and tricks we learn to be hyperproductive in a primary language are comfortable familiar elegant and effective they just feel good and I really got to agree with that that that's one of the things I really liked about certain aspects of the r language that I was able to get to a certain point where when I was doing these cleaning of data and I could visualize the code in a way that I still sometimes struggle with python on that I can't just like in my head like immediately see the code right away whereas the way this tidy verse stuff worked I could very quickly elegantly sort of see what was happening so that I I can kind of agree with that I like that she has this uh call for truce avoiding a flame War little section titled what this post is not basically trying to avoid anybody saying well you know one is better than the other and so forth and I agree that that's common in some worlds but the meat of it is that she suggests this stack of tools she talks about setup uh the something we've talked about a lot installing python right installing R is really easy there's one source you get it from one particular way and the same thing for IDE you get R you get the um IDE and you're good to go whereas installing python there's a bunch of choices she is suggesting Pi M which I know a lot of people are fans of I have little niggling things about it my little problem with it is that it doesn't install everything always which is something that is sort of surprising to people when they're like hey wait it doesn't have the gooey tkinter stuff no it doesn't so there's things you got to watch out for with pyam but otherwise I think it's a really nice tool especially if you have to go across multiple types of python and it also has things that help you with you know getting your environment set up and so forth so not a bad choice vs code is her choice for IDE I think it's kind of like the sort of de facto winner in this case vs code with all the features it's adding and the way it's growing as fast as it is and the fact that it's free is a really good choice also she suggests for doing your data analysis and wrangling of your data instead of getting into pandas potentially maybe look at polers polers which we've had on a few people on the show to talk about and we've mentioned here multiple times it has a functionality that's a little more familiar to someone coming from the r and tidy verse D plier kind of stuff um there's syntax like select and filter column selectors uh ways to express window functions and it's really fast I think a lot of people have Embrace polers for some of that stuff and again I think for like what I was doing in R of data exploration and cleaning I can kind of agree again compatibility things you might have to think about she gets a little bit into visualization the primary tool in R is this thing called ggplot 2 her suggestion is Seaborn its object interface seems to strike a balance between offering the features that you may want and things looking really pretty in a similar sort of workflow this is an area that again I've been out of R for a little while but one of the things I liked about it was the way that tables kind of looked in it she suggests a package called great tables data science you're showing off data all the time and it'd be nice if the tables looked great too and she includes this this package great tables which has a whole bunch of other stuff for labeling areas of your table that kind of get ignored by a lot of other packages multiple headers stub heads stubs multiple Footers footnotes Source notes span spanner labels for columns and all this sort of stuff so uh great tables looks like a really nice package for that she talks a little bit about notebooks and mentions of quto for doing computational notebooks and then a little bit at the end about environment management she suggests PDM which I haven't dug much into so I'm not going to talk a lot about it but I know that that is one that's growing becoming popular for that and for code quality she mentions rough which again is seems to be the one that is the refrain across all things Ruff includes not only you know linting stuff like flake 8 but also the code formatting stuff like black and isort um and it's very fast I thought this was a really great resource for again not only people that are coming from are but anybody who's looking for a nice set of tools to get going inside of python if you're coming in from a data science side of things thanks Emily what's your next one I've got something from Johan Becker and it's called let's create a python debugger together as the title implies this is a dive into how debuggers work focusing on what happens when a breakpoint is encountered the article starts off simply showing you how to use a breakpoint in an IDE like say py charm and then introduces you to the command line debugger pdb that comes with python just so that you've sort of got the background on what you're actually building and how it works with that understanding out of the way Yohan goes on to show you the code you need to write a very simple debugger starting with handling the breakpoint function which was introduced in Python 37 if you're not using this python 37 added a function you could call that invokes the registered d debugger before that existed you had to do the same thing however your debugger worked so now as long as the debugger supports the hook you can use the built-in function and no matter which debugger you're using that gets called if you're an IDE person and you're used to double clicking the line number there are situations that aren't that's NE not necessarily possible especially when you're say debugging on remote machines uh although newer IDs are getting better at that get off my lawn I mention this breakpoint because this is sort of the way things are are done now this function itself calls a hook and that hook is inside of the Cy module and if you're using an IDE behind the scenes it's doing the same thing it's just doing it for you graphically uh reasonably enough this Hook is called the breakpoint hook all one word and whatever callable you point that at is what gets called when The Interpreter encounters the breakpoint function so once you're inside your breakpoint Handler you're going to want to see where in the program you are and what your current state is and you do this by Examining The Calling stack there's another call inside of CIS which is underscore getframe and it Returns the current frame object which is a collection of data that is about where your program is it has references to the previous frame the code you're in your local variables Global variables line numbers and a bunch of other stuff all that is the information that's typically what you see once you're inside of a breakpoint in your debugger so essentially what you need to do is call this function and then display that information to the user the article goes on to show you how to build on top of the reple so that your debugger can interactively examine the state of the system and not that we did this on purpose but two episodes ago we mentioned Max Bernstein's article about customizing the Ripple and here's another interesting use case of that so you can actually see some of the stuff that Max was talking about in the article we talked about in episode 187 Johan's example goes on to use pigments to colorize the code being displayed to the terminal and then shows you how to build tracing capabilities as well this is a two-part article what I've talked about so far was just part one and then the second part goes on to show you how to implement conditional breakpoints and stepwise right so that step into step over that kind of good stuff so if you've ever wondered how your debugger works or you wanted to build your own this article is a great place to start and he's built the code as you go and then the code is actually available and working on GitHub so you can actually see exactly what he did not just the Snippets in the article so it's a good a good resource to learn about this stuff yeah we've done these dives into building these tools for looking at your code um the abstract syntax tree conversations I had last year this looks like a great set of Articles to get you going this week I want to shine a spotlight on another real python video course are you interested in practicing your python skills by building a game with an unbeatable computer player this course is titled create a tic-tac-toe python game with an AI player and it's based on a real python step-by-step project by core team member bartas jinsky and it's presented by video instructor Darren Jones who leads you through how to develop a reusable python Library containing the tic-tac-toe game engine creating a pythonic code style that accurately models the tictac toe domain you'll learn to implement various artificial players including one using the powerful Minimax algorithm you'll construct a text-based console front end for the game enabling human players to participate and you'll discover effective strategies for optimizing performance like most the video courses on real python the course is broken into easily consumable sections and includes code examples as you work along through the course all real python courses have a transcript including closed captions check out the video course you can find a link in the show notes or you can find it using the Search tool on real [Music] python. speaking of debuggers that's something that's G to come up in our discussion this week again something that I feel like beginners are not given the advice to use as much so but we'll dig in here the article we're kind of focusing on up with our discussion this week is from Hell Wayne he has a newsletter it's called computer things and a Blog that kind of goes along with it and the title of it is advice for new devs who've read other advice essays so uh I like that that there's a sort of subtitle from a person who really shouldn't be giving others advice I think that's because he notes in this first paragraph that he's only entered the work force 10 years ago and that he has readers of his newsletter and so forth that have been doing this before he was born and so maybe imposter syndrome kind of thing that happens sometimes which I think it's interesting but it's a list of 13 items advice for starting programmers we've covered this kind of stuff in the past feel like we give advice all the time here and definitely I think we've provided our own caveats but we have our own kind of interesting backgrounds I think number one is a really well suited starting point is number one is people don't listen to me because I'm a good programmer they listen to me because I'm a good writer the same is true of pretty much everybody you'll read and I think that's very true that the information that you're getting from somebody if an article is really well written it doesn't necessarily mean that this is the code that's going to work for you uh the person you know maybe is just an excellent writer and put things together and it was easy for you to kind of understand what's going on inside of it but it may may not be the Pinnacle of programming advice so it's something where you got to take all of this with a grain of salt and so the advice is to make sure that you're checking your advice as you go I think in some ways so I really like that one I'm going to mention a handful of these 13 and then I'll let Chris kind of take over and maybe build on top of it and let him pick a few and also kind of include some of uh you know his experience with this because I know that we definitely advise people to keep learning and that's partly you know anybody who's listen listening to a podcast about Python Programming is one of these people that wants to keep learning what's new what's happening in the world you know where can they learn new things there's so many these sources and making sure that you kind of keep a certain amount of skepticism of what you're reading and how it applies to what you're doing except with us oh yeah right right it was always perfect advice we're we're fine listen to us yeah just just be suspicious of everyone else so the second one is uh don't worry too hard about getting tricked or learning the wrong thing which I think is kind of interesting advice also again it depends on how how you're thinking about things I guess the the level of skepticism can be a little over the top sometimes and so you may want to kind of calm that down um there's lots of things to learn out there third device is about debugging which we've mentioned a couple times today he mentions this book titled debugging the nine rules just a side note uh I had Nina zacharo on in episode 71 and the title of it was start using a debugger with your python code and she's great and it's really solid advice that's not as frequently given as I think it should be uh debuggers are your friend they're a really great tool especially as a beginner to just understand how things flow if it's not a simple script that you're going through your code travels a very interesting path and the debugger will show you that and teach you what's going on inside there and it's it's a great way of learning what happens with your code definitely think it there that book looks really great um debugging the nine rules the site that he links to has a little poster debugging rules and I think they explain a lot already uh understand the system uh the second one is make it fail to quit thinking and look uh divide and conquer change one thing at a time that is really good advice keep an audit Trail check the plug get a fresh view is another one and then if you didn't fix it it ain't fixed number four which is called the right way to program which I don't want to dig into too much but it's interesting that you meet certain people and they want to teach you that this is the right way and there's also like what number six which I think is really great um is a is a link to Julia Evans talk about making hard things easy that she gave in at strange Loop in 2023 and just a side note wow she's great I really want to get her on the show very funny and laughs all the time maybe seems familiar to anybody who has listened to me she talks about anything that you've heard of as the best practice a thing where she wants to know what happened why what did what did Bash do to you you know what what what did HD TP do to you that you have this best practice now you know where did the code hurt you and I thought it was a really great talk definitely check it out she also mentions there's like so many things that are about computers that you're just never going to fit at all in your brain which I agree number eight on the list is take walks which I think is really great and then at the very end 12 and 13 I feel really kind of connected this field changes all the time yes you're going to have to keep learning in some ways but also along with that you really can't predict the future and what's going to be happening gosh ticked in the last year and a half into into account on that one so there's a Hacker News thread that goes along with this as what seems to happen often there they got stuck on a few items and I had a hard time going through all of it but uh the the right way guys definitely got uh stuck there for a little bit there which I thought was interesting what are your comments here what do you think Chris I well I I'm a big fan of walk oh yeah that's and it's something as both having been both a a manager and a programmer I've always struggled trying to convey to non-technical people that the sitting in front of the computer is the easy part It's thinking about the problem that is the challenging part yeah and so that whole idea of you know you know like I remember getting into an exhaustive conversation with a VP of of HR at a organization whose name you would recognize and uh you you know why why aren't your developers here at 9:30 in the morning I'm like well right you leave at 4:30 you don't know that they're still here at 700 why are we having this conversation right why aren't they in their chair well they're out for a walk and they're talking to each other and that's actually some of their most productive time right so it's uh there's there's value in engaging your brain in a different way and and daydreaming a little bit and and going for a walk can help that immensely yeah the other one here that I thought was kind of really interesting was I'm a big proponent of is do different types of coding early in your career yeah and I i' personally I'd extend that to use different languages as well one of the first places I ended up working uh well for starters my University had Co-op so I ended up working at different places even in the University so that that helps but even in one of my early jobs was at a Consulting organization and it taught me so much because each one of those gigs was like like I was working for the same place but their clients were different and some clients it would be you know you're on there for two or three days and you're trying to solve a very specific problem and that other clients you were on there for six months and it allowed me to see how different organizations worked it allowed me to see how different programmers worked it allowed me to see different programming languages and I I think it really influenced how I approach things can you think of a way that outside of an internship an unpaid kind of thing that someone can experience that more often like again I try all sorts of different things like conference talks or or podcasts or other types of ways where I can hear some of the stories behind things or kind of see through other ways people were working but I always kind of Wonder like well how can you get more of that experience if you're not necessarily the person who's like out of school and can afford to have an internship or whatever yeah I so one of those places that it's you know every organization has its it's good things and it's bad things uh larger organizations tend to make this a little easier uh because you might be able to work in you know shift departments or something along those lines in order to get a little bit of uh variety it's a challenging thing no matter what you do personally I try to like if I'm picking up a new language or I'm trying to go down that path I typically what I try to do is find some problem I'm trying to solve for myself that's toy level right like a couple hundred lines of code and yes you know okay because I've been doing python for 15 years I could crank this out in 10 minutes but if I'm trying to play in a new language and it's going to take me all day then you know I do it in that way in order to try and you know absorb that and go it's it's a hard thing right like it's you you need the time to be able to do it every language sort of attacks problems differently you know both Java and python are very object-oriented languages but they are are objectoriented in drastically different ways and uh try you know having a bit of experience in both I think makes you a better programmer in both you kind of see how you know what works in one place and works in the other and it sort of helps you sort of approach problems in a more interesting fashion in for larger projects particularly with larger teams as well there's also always a big advantage to being able to choose the programming language last if you're making a de ision about a project and we're all python developers and therefore we're going to do Python and it turns out what we're building is a goey we're going to run into trouble um not that python can't do guey but it's not its strength and so when you're trying to figure out what those things are being able to pick a language that actually fits your need can make a big difference uh in your efficiency so having that background and being able to sort of look at the different things can make a make a big difference to that and it's EAS easy advice to give and hard advice to take yeah I think about the the shadowing other departments and and that ability to move around inside of an organization well first off you got to have an organization yeah um yeah and so many of us work at home uh these days which is a massive change in in just a certain amount of time and so you don't necessarily get that Advantage which again you know pluses and minuses but I I did enjoy that part that that was something thing that a couple of the larger companies that I've ever worked for actually did go out of their way to do the school I taught at I would audit other people's classes so that I could yes teach their classes in case they were out ill or something like that and so forth and so I got to learn all kinds of interesting techniques there that was fascinating working at the bank I'd go around and meet these other departments and kind of find out what they were doing and again you would get to see the tools that they're using were potentially you know the people that are the end users of how they're using it and watching somebody work is just the most fantastic teacher you know like just watching them get to going you know and I think that's the strength behind uh Consulting yeah and it doesn't have to be Consulting Capital C go work at deoy right like you know your example the bank like if you're if you're working inside of Department that Services other departments to a certain extent you're Consulting right so you see how they're using the code you see how they're using the program you see what the different kind of problems are and you learn from that and I think that's where that's where expertise comes from right it's it's the oh I I've seen this a hundred different times I can now start to sort of think about it in a general fashion right and have enough experience to go oh this is one of those times where General isn't going to fix it so I'm gonna I won't use the general rule here and that that kind of comes back to the you know you commented about the the item don't worry too hard about being tricked into learning the wrong thing right you know there was a great comment inside of The Hacker News Post which was did I learn TD when it was hip yes do I use it no did it teach me to write better code yes so like you know learning tdd even if you don't uh you know even if you don't apply it religiously understanding the ideas behind it and you know what works for you it's not a waste of time Having learned it and read about it yeah the very first post in that thread was interesting to me because it was kind of focusing on what this person felt was like the wrong lesson to learn that this program was part of a team and the team I don't know in his mind really over engineered this solution for something they just added all this extra sort of infrastructure that I can kind of see as maybe a maintenance nightmare as far as like you know all the things that are included there and so forth when it could have maybe been something simpler for what the end result was was the person in the team learning a lot implementing all that sort of stuff yeah I mean you learn by programming and creating systems and that person learned well I don't want to do that and I I can kind of see where that's going to fall down and so like I don't again it's like don't take any of these things and feel like you know any of them are the end all and be all of advice and advice doesn't work for everyone context is really really important yeah yeah in response to that although it was like 12 Pages further down yeah uh but in response to that same sort of discussion there was somebody else commented you know don't make stuff more complicated than it has to be and generally that's good advice but as you said if you're using this as the learning case for attacking your problem then you it is a real world problem you might be you might be over engineering it because you're trying to play right right it's one of the things it took me a long time as a developer to really sort of absorb is I I always I I was always a big fan of oh I'm going to reuse this and you know reuse is good and and so I'm going to generalize this so that I can reuse it and now I try not to do that until it's the third time right like when I find I'm rewriting it again and again and again I'm like okay this needs to be in a library because if you do it up front you haven't seen enough usage pattern you often uh make the wrong assumptions about how to generalize it or or you spend a huge amount of time generalizing it when you didn't need to and could have just thrown off some one-off code I saw a response to this sort of topic where somebody said well we had a rule in our organization which was no single use abstractions so like if you're finding you know you've got a base class and it's only being inherited from from one class well then you've over engineered it so that's not allowed right clean it up so yeah yeah but I plan on doing yeah and when you and when you do and if it does then refactor at that time right it's it's yeah that's that's the general idea but why why do it right now yeah yeah yeah it's interesting like the other one other one that was in there was this idea that and and that kind of went off the Julia Evans thing is that all this stuff is so deep and has so many features and so many functionalities and you're never going to hold all of these commands in your head there's like the 10 to 11 ones that you're like yeah that's what I use this is the stuff that using and that's okay you know the thing that computers are great at is remembering trivia remembering all this kind of stuff and you can spend some time digging deeper on some of these topics and learning a little bit more and and and kind of the I do see this from time to time and it depends on a certain personality and type of learner where they're like I want to read the entire book and it's like uh okay well that again it depends on the person but it may make that person's head explode as far as like now I have to like figure out how I'm going to use all of this or I'm going to forget it or there's all kinds of weird things that can kind of happen there as opposed to well actually just start using things so then it's the whole idea of like doing projects and implementing things helps to make sense out of a lot of the stuff you know again depends on the learner but in a lot of cases then they can kind of go from there and say okay yes I can see where that was useful and again seeing code running is so different than just reading it in a book especially if it's your own code you know it's a good article uh lots of Great Links and additional resources and cool well that gets us into projects this week I guess do you want to go first on this one sure uh so I was looking at a project called fontomizer vintage Dave what this Library does is creates a optimized font file based on a series of usage files so David created it because he was cranky that the pages on his personal website were so large uh he used four ttf fonts one for headings one for text an italic version of the same text and a few typographical weirdnesses and all told the fonts took up 1.5 megabytes which is a lot of downloading for the first hit on an otherwise very simple page so you use use Font toiz library to read in all of your web files and then you call the optimize fonts for files function and what it does is outputs a new version of each font containing only the glyphs actually used in your site by doing this uh to David's own site he went from 1.5 megabytes to 76 kilobytes so that's a 90 some percent uh Improvement and makes a big big difference uh if if you think about it like you know the example he's got here where he's got he had a different font for titles he's probably not using all the alphabet in the titles and if you you know if they were in say small caps then you need like 20 some letters instead of at bare minimum capitals Smalls numbers symbols all the rest of that so all those extra things that don't get used essentially gets pulled out nice the project can be run as a library or as a command line tool so you can include it in your own devops actions or write a script if your needs are specific the output files have uh are are have different names than the inputs so you don't accidentally overwrite anything so it's a simple idea it's a powerful tool pretty useful if you've got a statically generated website and you've already gone to all that trouble to make it small and efficient and then robot robotto mono comes out and mucks that up for you uh so font I essentially helps undo the problem caused by those giant ttf files and it's worth checking out nice my project is called pint subtitle units for python it's a project by Hearn and Greco and this is from the read the docs page Pine is a python package to Define operate and manipulate physical quantities the product of a numerical value and a unit of measurement allows arithmetic operations between them and conversions from into different units it's interesting it mentions numpy it it supports a lot of numpy mathematical operations without monkey patching and wrapping for numpy so again caveat to the numpy 2 thing that I mentioned earlier it's not required but is supported uh so the idea is that you want to do something like meters per second to miles per hour you want to do even to something like furlongs and fortnite temperatures there's all the different scales of temperature Kelvin degrees Fahrenheit Celsius um Ranken all the different type of unit systems Imperial metric Atomic it's got them in there and so it's kind of nice that you wouldn't necessarily need to write a lot of that stuff yourself he mentions a lot of other libraries that have been around before in case you want to look around but I was impressed with it it it seems to have a lot of nice functionality the documentation really good so yeah if you're dealing with lots of different units potentially scientific or projects or different things you're working with uh check out pite there have been some rather famous software disasters because of a Failure to observe this kind of thing so getting it into the language using a library for it is uh is actually a very good thing because uh stuff blows up when you accidentally use meters per second instead of miles per hour or vice versa yeah yeah I think I've heard that uh some horror stories on that one too all right well thank thanks Christopher for bringing all these projects and articles and discussion always a pleasure and thanks again to posit posit connect helps make deployment the easiest step in your analytics workflows learn more about posit connect and try it for free for 3 months at pos. itre python I want to thank Christopher Trudeau for coming on the show again this week and I want to thank you for listening to the real python podcast make sure that you click that follow button in your podcast player and if you see a subscribe button somewhere remember that the real python podcast is free if you like the show please leave us a review you can find show notes with links to all the topics we spoke about inside your podcast player or at real python. c/p podcast and while you're there you can leave us a question or a topic idea I've been your host Christopher Bailey and look forward to talking to you soon