Creating Documentation With MkDocs & When to Use a Python dict | Real Python Podcast #118

Key Takeaways

welcome to the real python podcast this is episode 118. how do you start building your project documentation what if you had a tool that could do the heavy lifting and automatically write large portions directly from your code this week on the show christopher trudeau is here bringing another batch of pie coders weekly articles and projects we talk about a real python step-by-step project from martin bryce about mk docs the project walks you through generating nice looking and modern documentation from markdown files and your existing codes docstrings the final step is to deploy your freshly generated documentation to a github repository christopher talks about a pair of articles arguing for and against using python dictionaries the first article just used dictionaries pushes to keep things simple while the second article don't let dick spoil your code contends complex projects require something more specific we cover several other articles and projects from the python community including discussing the recent beta release of python 311 2fa for pi pi procedural music composition with arvo building a tic-tac-toe game with python at dekenter a discussion covering common issues encountered while coding in python a typesafe library that generates svg files and a lightweight static analysis tool for your projects this episode is brought to you by sneak sneak is the security platform designed for developers securing the software development lifecycle from the tools and flows devs use sign up for free at dot co slash real python that's s n y k dot c o slash real python all right let's get started [Music] the real python podcast is a weekly conversation about using python in the real world my name is christopher bailey your host each week we feature interviews with experts in the community and discussions about the topics articles and courses found at realpython.com after the podcast join us and learn real world python skills with a community of experts at realpython.com hey christopher welcome back hey it's been a while yeah i know um how's your summer been summer it's hot out yeah summer it's hot yeah thought we'd come back and do a little bit of news stuff just kind of a short set of almost kind of release notes kind of things it's just sort of top line stuff so did you want to start that sure the first ones we've come across a piece inside of one of the python mailing lists uh just a couple weeks ago that indicated that they're kind of behind on the release schedule for python 311 okay and on top of that already being behind they're not entirely happy with the stability of the current beta and so they've been discussing with the steering committee whether to add a couple of more betas and if that happens it might mean 311 won't go until maybe december okay i like the fact that they have timed releases it makes things predictable but i'd really rather stable than unstable so let's hope it goes smoothly from there and uh we'll get it a little later christmas present okay and it currently scheduled october so there's a little bit of play room in there correct okay yeah kind of on top of that there was a tweet and then a post on python.org about i don't know there's been like this whole trend of them calling the latest releases cursed just because of things that are happening during the releasing of it and this one they just had a real you know all caps please help us test this release thing and due to the modified release schedule and the stability concerns regarding the past beta releases please please please please please help us test python 311 by testing this beta release and they had this whole bunch of lists of stuff uh if if you maintain a library or a third-party package test the beta release if you have code that you maintain at work research center classroom whatever test the release if you're a multi-million corporation that uses python testers and even down to the single new person i'm detecting a pattern here test the release yeah exactly if you use python for work research teaching literally anything test the beta release so and it's not hard to do i know it's some work but getting the beta releases is not difficult to do well and particularly seeing as you should have automated testing for all your code anyways it really is just a matter of installing the new release and running your own unit tests so there you go should be on top of that the second news item in the python world is a change with pi pi and two factor or 2fa they put out a tweet thread and then there's some additional stuff on that they've begun rolling out 2fa requirement that kind of brought up some other stuff that we're not going to deal with here but they began rolling it out soon maintainers of critical projects must have 2fa unable to publish update or modify them and so along with that they have a post on pipi.org that one is to ensure these maintainers can use strong 2fa they're distributing 4 000 of hardware security keys those are usb style keys either standard usb i guess a and usbc style project eligibility is based on downloads any project in the top one percent of downloads over the prior six months is designated as critical out of what 300 000 different projects they're talking around you know just over 3000 or so anyway they've notified people via email and anyway you can kind of look at the information there i thought it was interesting that they chose hardware keys and they had some links to some information about that and also a really good faq about the program if you're curious someone of the details and and hopefully you can get some of your questions answered instead of screaming out into the void of like what is this and they had this link to getting factor right in 2019 on the trail of bits blog and what i found interesting about that is even back in 2019 they were talking about how what i use for authorization for most things which is tools like authy or the google authenticator app that use like a qr code kind of thing i guess there's some potential pitfalls there and so this article kind of gets into that so i'll include some of the links and i guess we gotta wait for the next level which is gonna be hopefully the authorization with our devices which is seems to be coalescing and coming together so i hope that will make it easier a little bit news at the top there do you have any other notes there and of course this is all to mitigate uh the potential of people stealing each other's repositories and we've talked about that in the last couple of episodes but uh there's been some issues lately so uh really as far as i'm concerned i know that you know anytime this kind of thing happens there's always a bit of a fuss but i think they're trying to be responsible and they internet will debate because it's the internet but uh i think they're they're trying to do the right thing right and the one percent of the one percent will you know be upset but hopefully that doesn't get amplified yeah well and you know i i think it should be constituted a bit of a badge of honor right if they want to ship you a key that means your your report your repo has been massively successful right so yeah yeah which is nice yeah that gets us into articles and tutorials this week i'm going to start with one from real python it's by previous guest martin bryce and it's a step-by-step project i think actually both of our real python ones are step-by-step things this one is titled build your python project documentation with mk docs things that i thought were really nice about this it provided a really good set of best practices for not only kind of documenting your code internally things like docstrings and so forth but also like what are best practices as far as the types of things that need to be created for documentation for your own packages like if you're going to put up something on github and you want to share it openly this project what are things you should be thinking about as far as documenting it and if you've done documentation within your code such as putting actual docs strings in with your functions and so forth there are tools that can help you create really nice pretty documentation kind of automatically or even better as you make changes update all the documentation for you too and actually output a set of html files and so forth that you can then host a github page also and this takes you through that it it starts with a really just total example project it's a simple calculator tool so there's the functions are like add and multiply and so forth but that's really not the point the point is to kind of get you into the mindset of using these tools there are some kind of big prerequisites like you should be at the point of thinking about quote unquote a project hopefully you're familiar with virtual environments using pip to install packages but also like freezing requirements and then how to organize your code into modules and packages yourself so there's some really good resources for each one of these i'm mentioning on real python that are linked also so there's good links to kind of get you up to speed and then fundamentals of code documentation and then kind of the basics of github and github repositories going beyond you know using git on your local machine but also pushing code in and out of repo and so forth in the tutorial you work with a tool called mk docs that produces static pages out of markdown and then it also can pull in documentation from your docs screens using another tool that's kind of related mk doc strings and then again lots of best practices stuff and then he's using we've talked about lots of sort of themes that kind of help you with sort of css and making stuff look a little nicer uh in this case they're using the material set of standards from google and so there's a package called material for mk docs which is sort of theming everything to make it look good and then at the end you're sort of hosting the documentation up to the github pages so i won't go all the way through step by steps but you basically are setting up the environment creating your python package again there's a sort of built-in one you're writing your doc strings and that is one of my first like kind of like takeaways is i didn't know there were so many standards for doc strings they mentioned three in the article and the one that i'm most familiar with is the google one i didn't know that it was actually labeled as that but that was the style that i've seen the most and maybe that familiarity makes me feel like that feels like the best one for me there's a numpy style but and then there's a sphinx style that's the ones that are mentioned and sort of linked there i learned there's three other ones that are also potentials when one's there and that led me to a kind of a cool takeaway is there's a vs code tool called auto docs string that you can download as you know kind of a plug-in for vs code and it'll help building your doc strings generating them for you and you can kind of in your preferences choose which style you would like it to help you kind of format in there which i think is really nice again things to help kick you in the direction of doing doing better documentation and that's the whole point behind this that whole area of doctrines you know creating them under your functions and then writing examples that you could potentially use testing through doctest which we've mentioned recently type hints and providing automatic type information and then kind of going further into like what kind of doc strings are inside of actual modules and then starting to use those tools using mk docs and building the documentation and then hosting it up in github so i think it's a good resource for anybody who's interested in taking this to the next level and diving in and wanting to make good looking documentation and not knowing where to start this i think will get you up and running pretty well and lots of additional resources and like i said best practices you were mentioning to me that that you're more of a fan of using the sphynx tool right yeah i've never actually compared it's just where i started and so i haven't had a reason to go do anything else but most of the libraries that i maintain i use sphinx and there's connectors to read the docs so you can push it all up and get hubble uh read the docs will pull it out of the github and and and maintain it up on the on the main site so yeah the the doc strings for the sphinx style i wasn't that familiar with with it has lots of extra colons in there i was just like that's looks different but it makes sense that you kind of have to preset a bunch of rules to kind of have it identify these elements to separate them out and put them together yeah that's it's really just another way of tokenizing things the the um i do find i i use it fairly often but it's one of those things i often seem to have to look up there's basic things like the parameter on the function or on a method i remember how to do that because i do it often every time i try to put in a reference to another class and i want that to show up as a link i almost always have to okay is it pulling in the backslash what what and i always have to look it up so yeah i think that's pretty common so what's your first article it's actually two just for variety's sake all right kind of a head-to-head thing uh the first one is called don't let dix spoil your code and it's by roman emcolob roman's premise is that as your code grows those simple dictionaries you use to store data may become harder to maintain and one of the examples of this highlights what happens when you have a function that takes a dict as an argument this can be really fragile right an upstream change to the dictionary say renaming a key or adding a key could impact the function and it's not going to be discovered until run time when the function fails and it's really hard to type this kind of stuff and catch that with my pi or something along that lines okay so roman goes on to describe the use of a good use of dicks which is what he calls the wire format and due to well the internet json has sort of become the de facto standard for a lot of data interchange and it only takes a single line of code to go from json to python dict so roman says basically don't stop there use the dictionary to populate a declared class and do so in a way that the keys and the dict aren't in the class they get ignored that way you're kind of fragile proof is really what it comes down to going that further step and getting into say a data class or something like that gives you the safety of more type info and someone mucking with your keys up straight won't break your code okay so he goes on to talk about you know data classes and pedantic both of which give you more structure pedantic is a data validation and serialization library that includes a bunch of type info i've actually been using it recently indirectly because i've been using django ninja and it's built on top of pideantic so a lot of the serialization pieces and things like that are built using that that's a tool for api design right yeah for api design that's right and then finally so roman also points out that you can also use the mapping object which is in the typing library and it essentially gives you a dictionary that's a little more rigid so you give it a pattern and then it solidifies the keys and then if you try to use a key that's not supposed to be in there it's disallowed so this gives you again more type safety right so remember when i said i had two articles yeah well the next one is called just use dictionaries and it's by david vujick i don't believe it was written explicitly in response to roman's article he doesn't mention it but it really could have been david's premises don't overuse classes dictionaries are often good enough and the first example he uses shows a data relation problem where you're mapping desks to offices and the class-based implementation gets messy quickly because you've got this nested thing you have to have an office type and a desk type and you know the relationship between them and the dictionary it's like i have a list and it's that's it i have a list and so of course the dick doesn't have the type safety but david's argument is you know what there's about a third the amount of code here and less code means less code to maintain and you're probably better off from that perspective right so he goes on to comment that pro as programmers we have a natural inclination to want to define objects for things our data often maps to physical things in the real world so it makes sense to want to encapsulate it and this goes double for those of us who've spent a lot of time in the java or c plus plus worlds because like it's ingrained in you yeah we're gonna make classes exactly right and uh he argues though that this really is just often overkill he goes on to talk about data classes and pedantic and he's it's not like he says don't use them he just says they have their uses and here's some examples where they shine but if you're not doing these examples is it overkill so fundamentally david's argument is the kiss principle keep it simple sam that's what the second s stands for right yeah silly sure uh so these articles have varying viewpoints something neither of them really talks about though that i find is important is scale and scope they both seem to be do it this way and my argument is okay are you maintaining a million lines of code well roman's got a solid argument for you are you writing something smaller well then romans overkill and david's approach might be best right that's what i was thinking about in that roman's approach is something where it's a bigger project and other people are going to touch it yes yes often all right even for yourself right there's a there's a maintenance cost a mental maintenance cost once you get past a certain size right yeah and i think what happens oftentimes is folks get bit by somebody else's code and then they start to become rigid and go well it should never be done this way because that cost me three weeks of my life right yeah yeah yeah i when i was a younger developer i i wrote a lot of code thinking i'll reuse this and that again is sort of towards that class mentality but now i tend to wait to modularize until after i've written something a couple of times so i kind of start with the simple and then i go oh which i you know is david's argument and then it's like oh this is getting bigger it's getting under uh unwieldy then i'll switch over to the other side when simple isn't good enough okay that makes sense i think i'd be right in the middle too it depends well you know i i think this you know this echoes some of the the the type safety stuff we've talked about before right like it's not that it's not a good tool but if i'm trying to write 100 lines of code and because my boss says so i have to add typing information on this it's like it's going to take me twice as long and why am i bothering right so right but in the case of you using pydantic in in this new ninja tool like exactly makes sense right yeah that's right yeah with a couple key bits of type annotation i've got the entire arrest api spit out for me being and there's an appropriate use right so yeah i know here i am arguing for reasonableness and uh i'm sure the internet will remember in episode 118 [Music] sneak doesn't just find vulnerabilities in your code open source dependencies containers and cloud infrastructure it also provides expert remediation advice and automatic fix prs so you can merge and move on knowing that your applications are secure and since sneak was designed for developers it works right from your favorite dev tools like pycharm git cli and more start your free forever account at sneak dot co slash real python that's snyk dot co slash real python [Music] so when you saw this article get in the feed you immediately were like all right i know i was gonna pick this one and you're right so it's titled procedural music composition with arvo and it's an article from serge etienne parrant and it's on a blog deepnote.com let me start here i have mixed feelings about procedurally generated music the compositions created with it often well i just don't dig the results they really kind of lack a lot of the humanity of music and kind of feel more like kind of mathematical proofs of what we could do with this i think they can be super interesting all of this though was super fascinating for me and has been a thing for me lately where the technology and the ideas that are shown in an article or tutorial or something like this gets my mind spinning and i start really digging if you will into all these other areas often the digging is just provides a bunch of dirt and rocks you know but occasionally there are some gems and i found some really great gems through this that are kind of related the article covers a tool called arvo and arvo is written it's a project that's by dr george dimitrov and the package is really kind of built on top of another package called music 21 which is kind of out of mit and it's that package is a python package that's calls itself a toolkit for computer aided musicology music 21 is like a set of tools that helps scholars really look at music here i'll read from it is a set of tools for helping scholars and other active listeners answer questions about music quickly and simply if you ever asked yourself a question like i wonder how often bach does that or i wish i knew which band was the first to use these chords in this order or i'll bet we'd know more about renaissance counterpoint or indian ragas or post-tonal pitch structures or the form of minuets if i could write a program to automatically write more of them so music 21 is kind of designed to analyze music but also potentially generate music out of it i tried to read the documentation and i had a real hard time with it it's real deep as far as what it does and so what i liked is the tool that was created from it this thing called arvo was taking it and sort of abstracting it again and allowing you to kind of create some stuff and it has a bunch of modules built into it and the the modules that are in arvo let me see if i can find their names again so the library has these built-in core modules one's called iso rhythm and they're functions for generating iso rhythmic constructions from pitch and rhythm sequences minimalism which are functions that generate philip glass steve reich inspired additive or subtractive processes and then tin tin bully which i don't know if i cannot pronounce that which is generating arvo part inspired these are based on like chords and kind of creating harmonies based on that and then then another tool called transformations that allow you to inversions transpositions retrogrades these other kind of musical tools that are could be really cool for again creating music that's in that vein it also contains helper module for scales and sequences and other kinds of tools and then it has a collection of samples the tool can output to either like sheet music which led me into this whole avenue of learning that there's this awesome tool for documenting music and kind of what cheat music can be in the world of the web today and the w3c the worldwide web consortium i think is what it is has created this thing called music xml and it's a way of sharing it it's i guess been around since version version one was 2004 and they're working on it still so like version our version one was 2004 version 4 was released in 2021 yeah i wasn't paying attention to this and even along with that one of the gems i found is this thing called muse score and you can get it at muse score score.org totally free open source notation package that is great i was really impressed with it i worked at music stores and i tried to sell you know notation software and it was multi-hundreds of dollars and super clunky to work with not compatible with each other and it was like all this kind of nightmares and this is kind of breaking beyond that which i was like wow this is really neat so one of the things that you can do is with arvo it can export directly to it so you can kind of see the sheet music and the kinds of things that it's generating but also can output midi the author of the package created a really nice youtube video so i think this tutorial that i'm referencing the procedural music composition with arvo that's by serge antion is good if you're into kind of reading and following along you know like a written tutorial and it has a sound cloud clips that you can kind of play to kind of get an idea what the stuff sounds like but the creator dr dimitrov actually made a youtube video of him showing you how to use arvo and i was really impressed with that and again i sometimes want to watch a video just kind of see the results and then in the end he's tying it all together and using this thing called ableton live to kind of get the parts and play with them i'm just fascinated by this sort of cross-section and it again i found a bunch of little interesting little gems out of it i think that one of the things that i think is really kind of cool with this music xml is that this idea that we could actually kind of build tools to read and write xml in python and kind of abstract some of the nonsense that xml sort of kind of raises the hairs on the back of your neck oh no you know what am i going to get into here and i think that there's a lot of these kind of tools that kind of help with it i think also music 21 is a really fascinating project because of the that it's you know able to analyze music but also output it so anyway a bunch of resources if this is something that you're into again it's a deep dive but i had a lot of fun kind of digging into it and i got to play with the packages a little bit and um i could see myself kind of using it as an idea generation kind of thing honestly when i was done looking at the arvo package i was thinking man you could build kind of a nice gooey front end for this and be able to kind of play with it so that's i'm thinking of a project for myself for it so we'll see i don't know if you're a big fan of uh uh these kinds of projects or do you listen to philip glass or things like that i i've um so i i've got history in it but i don't do a lot of it now i think it's what it comes down to so it's why there's a there's an interest in reading the articles because i'm like oh well that's neat and you could do this and you could do that and and then i look over at my piano in the corner that's slowly gathering dust and go and when will i do that so yeah well your next one's a real python one right it is uh and it's another step-by-step one this uh this one's called build a tic-tac-toe game with python and a tk enter and this article is all right mr bailey say it with me leodonis our listeners are going to start to think that uh he's bribing me or something uh i if you're listening leodonis 15 year old havana club grand reserva just so we're clear uh anyways as you might guess from the title this is a project-based article if you haven't played with tk inter before that's a tk-based gui library for python and tk is a set of underlying gui libraries that several different programming languages have built tools on top of i'm not sure whether this is a regional thing or where i just hang out but i've often heard tk inter pronounced as tinker which is like a purely dyslexic thing so excuse me if i slip into that here mine is t t kinter yeah i think yeah i think it's not meant to be pronounceable uh playing with this was kind of fun i had flashbacks to about 25 years ago i used to write tickle tk applications with c plus plus backends uh so it was kind of nice to see that tk's still around and that i can do this in python is even better because uh there's uh it's a pretty quick and easy thing and in fact a lot of the time when i'm messing around with css i look at it and i go but all you have to do is learn how tk put these things together and none of these problems would would exist this is much easier but anyways um you don't want me starting to rant about how the web is broken and the article is broken down into five major steps so step one is setting up the gui and the main parts of the tic-tac-toe board the second part is creating some game logic then third is processing a player's move which includes things like checking for winning and tying states the fourth is then handling the player's move in the gui so showing that stuff updating the display and then final is really some polishing addings a menu that kind of thing so tinker does come with python so in theory you don't need to install anything but to quote yogi berra in theory theory and practice are the same in practice they're not yeah the caveat here is the article uses features out of tk enter 8.6 and if you're running python398 or more recent you're good it's there if not you might have to upgrade your tk enter the installation to get it to work then there's some stuff in the article about how to do that depending on what platform you're on some pointers yeah it's kind of fascinating i think a lot of people maybe didn't know that this is one of those batteries that's you know in there in the it's a really big pile of batteries and some of the batteries do the same thing as much as we say that they're not supposed to but yeah there's there's multiple gui things built into python so yeah the turtle and and decanter is like right there there's some corners yeah definitely the first part of that project then is talks about sort of the classes and the widgets that you need and how tk puts those things together how you glue the buttons into the tic-tac-toe sort of design and at the end of that step you actually have a thing you can play with you can push the buttons the buttons don't do anything but you can push the buttons and then it moves on to start the construction of this is actually a two-player game right so you would click and then your opponent clicks and it shows you some code for figuring out okay how do i know what patterns on the board or wins or losses and then tying that back into the gui so that if i click now i'll check okay has that changed the state of the board to a win or a loss or a tie how all those pieces fit together so it's a nice you know if you if you're new to the gui world or you're new to tk inter this is a nice little way of sort of going through and introducing to it it's not exactly beginner's material so if this is your first python project you might want to start somewhere else you've got to be fairly comfortable with the concept of classes and list comprehensions some basic data stuff dicks name tuples but if you've got that this is a nice way to play around and do some new stuff and they've got a really good section on sort of the prerequisites that lists out a bunch of other articles yeah so if you need a refresher in any of those things that i just listed you can go off and click through on all of it yeah i'm liking that structure of in the articles lately for that the again like the one i was mentioning earlier yeah no it's a great way to do it so you can kind of know what you need and if you don't have it you can go click that first and come back so cool as expected all in all top notch article and they're leodonis there's some rum worthy praise if i if i've ever heard it so i'll send you my address all right [Music] this week i want to shine a spotlight on another real python video course and it's about a topic we're discussing this week when you're starting a project one of the big decisions you need to make is how are you going to store and work with data python provides a variety of built-in tools this course is titled dictionaries and arrays selecting the ideal data structure the course is based on a real python article by dan bader in the course my co-host christopher trudeau is your instructor and he takes you through what are the advantages of using the built-in dict type what are four other types of python dictionaries how list and tuple types are arrays what are typed arrays and how can they save memory what are the different arrays for storing binary data what are practical uses for the different types and finally how to select that ideal data structure for your programs i think it's a worthy investment of your time to learn the variety of ways of storing and managing data in your python programs the choice of a write data structure will affect the readability of your code ease of writing and performance and like most of video courses on real python the course is broken into easily consumable sections and has code samples for the techniques shown like all of our courses it has transcripts and closed captions check out the video course you can find a link in the show notes or you can find it using the enhanced search tool on realpython.com [Music] so we're back to doing a discussion this one is from matt harrison and he has been asking kind of questions to his audience on twitter and this one was what is the most common issue you have coding with python and i think there's lots of ways that you can take the word issue and people definitely did in the thread which was interesting he tried to answer some of them and then a lot of other people were trying to reach out and help too but i thought it might be interesting like i guess a lot of one of the ones that i saw that that i kind of related with was this setting with copy warning which i think i've already mentioned before and he was on the show talking about it also that there are ways to that you would never see that but that was something that literally that warning would come up in the code i was working on as i was you know beginning to work in pandas and people would just wave it away it's just a warning i'm like yeah i don't think this is good [Laughter] so i don't want warnings in my code i don't know if you want to get into it like like some of the stuff again are common beginner things that people have trouble with virtual environments the idea of packaging just some straight up complaints like pip install never works and i'm like wow okay i don't i don't know where that person's gone wrong yeah yeah i saw that one yeah i'm not sure what never means because i don't think i've ever had a problem with it i don't either yeah i mean i i had problems initially with like the concept of virtual environments or just forgetting to turn it on and you know just be in a hurry and like where did i just install this um i have solved that problem so um again some of the commonness of the issues have shifted over time i think probably one of the big ones for me is forgetting we just mentioned all the batteries of python but sometimes i just forget to use built-in stuff and and remembering that and it's almost kind of cool like i had an episode that just came out talking about refactoring and occasionally i would use a tool like sorcery and it would be like hey you know this thing exists and you're like yeah oops you're right that would have been better so that's that's a common one for me what are some of yours i i have a horrible habit of copying pasting a line either out of a function or a dictionary and forgetting the comma on the end and turning something into a tuple that isn't supposed to be a tuple and i know what someone's going to tell me well you know if you did type checking [Laughter] you wouldn't see that the other one i find which isn't really python specific but because i do a lot of web i'm back and forth between python and javascript yeah and so it's it's what ends up happening is you you end up like writing half of each language in the other language and it never works in either case so i'm often accidentally using def instead of function inside of javascript or putting a brace bracket somewhere inside of the python and going oh yeah that's right that's wrong language and a subtle version of that one as well is enumerate the order of the index and the value that comes out of it different languages have similar concepts but put it in different orders uh so i frequently will end up going what why why why is that got a number in it that shouldn't have a number oh right that's the index right so i'll get them backwards and you you mentioned the uh you know being in the wrong virtual space uh because i run on a mac the base python install is still two seven uh and so sometimes i'll quickly run a script in a screen somewhere that i haven't uh enabled the virtual end for and and it right just dies because there's syntax errors in it because it's uh python 3 right so yeah it's i i to me this is i very little of it i noticed as i was going through except for maybe the tuple thing as i was kind of thinking about it very little that it's python specific i think a lot of it is just programmer be programmer so yeah to kind of go over to again one of the common ones i saw in this list of stuff that i never have this problem and i just kind of get confused by it is the the nesting and indentation thing yes i did have that problem once working like through tutorials like i was doing stuff that was very very class heavy which was like working in pi game you know where i was embedding things and like making sure i had the indentation right but that isn't as much of a problem for me now i can see how that can be you know because it's like that's the crucial sort of structural element in python the one other ones that i kind of run into now are similar to what you're saying of like i feel it like it's my office you know like my coding environment and the way that it's set up i am constantly jumping around running lots of other people's code and i feel like often i just i need to just stop and i need to like spend some time on like getting this the way i want it and it's just this thing that it's gonna be unique to me because i'm kind of like looking at lots of other people's stuff often not only within an organization but also you know i'm researching projects and the stuff that we bring up here on the show and often we have this sort of meeting where we kind of decide on like okay what do we want to talk about this week and then we kind of are picking and choosing and sometimes that changes within the next day and we're like okay this did not work out and that's often because i'm not able to just even follow the way that they've described how this stuff is set up or there's like version or hardware conflicts those are the things that are causing if you will the issues for me now as just like you know getting things to the end point you know like getting past sometimes these projects and getting them going and so i don't know what my solution is there like i've thought about going beyond even like usually it's virtual environments which has been good but sometimes i'm doing bigger things and i'm wondering okay well maybe i should go have this on collab or some other kind of thing and so i'm not decided and the other one that is old that i had problems with was and i've mentioned this before in the show earlier on is in jupiter notebooks not knowing what the current state is because i was exploring and having to just okay let's just start from the beginning and run these cells you know yeah and kind of start over and so that was probably you know some of the pain points that i was feeling were there other ones in the in the list of stuff that he had there that you brought up that other people were feeling uh a lot of it's uh you know it's the internet so someone said python so someone says white space and copy and paste right that's kind of what it is and someone else says performance and the and the latest answer to any time anyone says performance is but 311 is going to be better yeah it's like yeah okay so you said python i'm surprised someone hasn't made a snake joke and since you're all very original there was a you know a little bit of a call back to your mk docs thing there there was a few places that were like very project specific going we should get the people from this project to write the docs for that project because these guys rock and that stuff is horrible right so there's there's a little bit of that sort of specific pieces i wonder about that because i i follow a few developers that are in the apple sort of ecosystem and they all just complain so often about the documentation that is part of like say swift or you know apple's apis for their hardware that it's just not complete and i wonder as a language i feel or even just the ecosystem like i feel like the documentation generally in python or at least stuff that i've chose or selected to use is pretty well documented but i don't know again i'm one of these people that my experience kind of starts at this sort of halfway point you know like i was doing programming before the internet and then stopped right and picked it up you know recently um do you feel that way i suspect some of it is age of language so because swift is still relatively new and because it's fed out of corporation your primary documentation mechanism is going to be that corporation yeah from them okay and because python is open source if you don't like the documentation contribute to the documentation and right and because it's been around long enough people will have done that right so yeah okay you know and that's it honestly has complete tangent but it's one of the easiest ways of contributing right so i i was uh i mentioned earlier i've been playing around with django ninja and there was that i needed to do something and i i posted and the the maintainers are fantastic they answered very very quickly and i was oh okay that's great and i'm like okay maybe i missed it maybe i missed it in the documentation and i looked and it wasn't in the documentation i'm like okay fine i'll here i'll write two paragraphs and give you a merge request so the next person doesn't have to ask that question right it's the easiest the easiest way of contributing to somebody's package right so and i think over time this is what happens is it gets built up right so yeah well i think that brings us into projects if i hadn't have enough fun with xml earlier i found a project that is probably one of the smallest projects i've ever talked about it's really simple and really small which i like about it it's svg.pi typesafe library to generate svg files why do i care well i was interested in svgs if you're not familiar with it scalable vector graphics i've always kind of been fascinated with vector graphics and i i think they're really kind of cool in the sense that you can scale them to whatever size that you want it's a very different sort of art style from raster stuff i never owned a vectrex video game system but i was fascinated with it anyway so svgs i had to do some work where somebody wanted me to create like a logo for their company and i'm like i'm gonna do this i'm gonna learn how to do this and and so i was working with different tools and so forth and i get hung up sometimes and the xml that is the svg sort of standard is another w3c thing it's just sort of complicated and having a library that can kind of turn that into python objects and make it simpler i think is really great it's by orcinium is what he goes by on github his first name is just graham that's all i see on there graham from the netherlands features of the package compatible with all svg standards 100 type safe pure python no third party runtimes it's the fastest thing i've pip installed in a long time it was super quick no deprecated attributes only what actually works the same names and structures are as in them standard if you know how to write svg files you'll know how to use this library and it's based on a schema that's part in the mozilla mdm reference the svg-xsd schema so that was a little rabbit hole i went down again lots of nice little examples to get you started and yeah uh i don't know if you're gonna need to output svgs but like they're really kind of nice you know they're super super small because it like literally outputs just text if you will and they can describe really kind of complex shapes and then again they can be scaled to it or size so i think of like background elements i think of just general shapes of other things they're everywhere but you may not know that it's possible to create them so and now that browsers support them directly yeah i'm finding they're becoming more and more common inside of web pages so you'll see them for things like logos or for icons and things that rather than using an icon library they just directly embed the svg right into the html yeah i'm wondering like some of those uh common glyphs that are in some of these other libraries that are out there i'm sure a large number of them are svgs but anyway i just i've thought about going in and working in the xml itself and then just sort of like shaking my hand and going nope nope and so this is kind of a nice way to kind of at least stay a layer above it and be working in python and has a nice way that you can kind of just pipe the output right into a file a dot ssvg so anyway nice simple little project what's yours well it's nice but wasn't simple this took a little bit uh so it's uh it wasn't too bad it's just uh it wasn't a quick install shall we say okay uh so this is a tool called semgrupp uh it's a multi-language multi-programming language semantic analysis tool for valued in code so if you've ever used something like sonar cube this is similar to that so it's one of these quasi-licensed things so it is open source but it's from a commercial company with a commercial offering so there's a command line interface that you use to evaluate your code and it does that locally so they swear up and down that they do not send your code up to the server but they do send a bit of metadata so depending on your comfort level with that you might want to dig into you know they've got a good privacy policy and everything but you know if you want to be paranoid you should know that so there are ways of controlling it but if their rule registry for what analysis to be done is actually on their servers so it you run the command it goes it gets the latest rules for whatever files you need and then runs it on your thing so i ran it on a couple different projects one was one of my own libraries and the other was a client project i was very impressed with the results i didn't see any false positives everything i complained about was something i probably shouldn't have done so that's always good okay nice the library that i was talking about is a set of utilities for django projects it found not only the python in the library but also some bash scripts and yaml files and did analysis on those as well warned me about a spot where i might accidentally be trusting some user input that i shouldn't be highlighted a spot where i was bypassing the password validation bad boy and uh you know leaves it to uh security flaws for insecure passwords and even inside a couple django template files it highlighted the fact that i wasn't using the integrity argument on a link tag which of course is dangerous for cross-site scripting attacks so it was pretty comprehensive from that perspective nice the client project is a single page application using django drf on the back end and vue.js on the front end so it ran analysis on the python bash the html django and javascript and it even found like some json data files which it validated this one got ugly fast it had 89 findings which is a kinder way of saying you're an idiot thankfully i had included some third-party libraries in this repo so my ego was soothed a bit by the fact that not all 89 were my fault they're not all my findings yeah on the other hand the dangers it was highlighting are still in the code and that was being shipped so that's always it's fun thing semgrip does support writing its own rules which is kind of cool it uses a yaml format with the code matching part looking like code itself so you're not having to write some horrific regex you so if you were you know the example they use on the website is you want to you want to catch print because you don't want print in in production because it spits stuff out to the logs essentially it says you know the the rule is print bracket dot dot bracket and it finds them all so nice and clean cool their rule registry is open for contributions and each rule gets its own license type so most of them are open source i think they're mit if i remember correctly and some of them are owned by the company themselves so you're allowed to use their rules as long as you're not charging for them so they don't want you setting up a competing registry but even they were saying like even if you're a consultant and you want to use it at your client and you're getting paid to do that they're fine with that they just don't want a competitor popping up on the web which seems fair right using all their yeah we sell a tool that does the same stuff okay exactly and uh so the one last but important feature to me is it supports and ignore pragma so put a little comment in your code to tell some grep you know yeah i understand this finding but i don't want you complaining about it yeah which is good because you you know sometimes you know you just want to ignore certain rules or maybe if if you get a false positive you want to comment them out because if you get 50 comment if you get 50 warnings and then you get used to the idea of there being 15 warnings you're not going to notice when there's 51 so you're better off you know getting down to zero each time you run a tool like this it's just better prac