R Package Review Episode 5: TodoR

Key Takeaways

okay here's my r script here's a to do i want to go follow that up i don't have to open up my r script like watch i can close that r script file i can double click on this marker and it opens up the rscript file and it actually jumps my cursor exactly to where this tag was [Music] welcome back everyone for those of you that are new here my name is brian jenks and on this channel we talk about all things programming tech and data analytics it's been a while since i've done an r package review but that's exactly what we're going to do today today i'm going to be covering the to do r package this package is a great way of following up on all those little comment notes you leave yourself such as bug fix me hack to do etc in your code base and pull them all into one interface and hop to them to resolve them all with one simple tool so let's jump right into rstudio so as with all these package reviews you can find all the materials talked about and covered in my git repository for this entire video series and the link to that will be in the pinned comment below so let's get started the to-do r package is something that i have actually used in production at my workplace and it's a very very useful package for just managing all the stuff in your code base to find all of your items that you leave yourself like comments to follow up on and handle later so first of all the first thing we're going to do is we're going to start with a brand new r session you can do i think it's on windows ctrl shift f10 but i'm going to restart my our session so we have a clean slate all right so brand new no options no anything set it's brand spanking new so i have a bunch of files in my current directory so in this directory i have let's see an html file a sweeve document.rnw i have our markdown document our html document and an r script a lot of these other things we can just ignore because they're not relevant to this entire subject so i have all these files and within them i have placed comments of different kinds and we're going to actually run the function to do r and see what sort of output it gives us so now we run this we get this whole list right here in a tab called markers and now some of these of this output is very interesting it'll tell you the exact file path to a specific document so it starts from your home directory and all the places uh leading up to the actual file and i'll tell you what the file is so for we have an r script and we have a single to do and it's actually a to-do item comment on line two now i also have another document here this one is for the actual our markdown document we started with and we have all these different kinds of notes we have to do's we have fix me's we have a bug and all these items are comments in the actual files so what is it actually doing what to do r does is it will find all these comments so when you leave a comment for yourself you don't want it actually printed in your output you don't want any code executed it's just something to remind you later but commons have a way of being forgotten just being buried under a large code base and we forget to follow up on these things or we forget why we put something there so to do r is a really great tool to make sure that you actually go back and remove and follow up on these items so in our markdown actually in the code chunk area comments start with a hash symbol so you can see right here i have one to do here's a comment follow-up note and we can look over in the markers and right here for the our markdown document we have to do here is a comment follow-up note but the to-do is actually pulled out and actually tells you what kind of note this is and then it actually tells you the content of the note so it actually pulls out that first you know category here which is how we have to structure it it has to be like the comment symbol to start it off and the beginning of the comment is actually the type that we actually want to specify it as so you can see right off the bat what occurred is that i ran the to do r function and then all of my r files and the entire project were scanned so to do r by default will work with an entire r project it defaults to the project level and says everything within this r project we're going to look at it and we're going to pull out all of these items so it pulls out everything out of our scripts and your our markdown documents but we also have some expanded functionality on what it can look at and we'll get into that as we get further into the video so let's say we have a very large file a lot of code a lot of items and we only want to look at notes for that one specific file and ignore everything else temporarily so we have the ability to do a single file scan with to do r and we do that with the to do r underscore file argument function and we're going to pass it a string argument of the actual name of the file now right now i am all i'm currently operating at the same directory level so that means that my current project and all my files in my project are at the same directory level normally in an r project you might have a directory you know with a capital r and you keep all your r scripts in there and you have an inst inst like an install file for your our your our markdown documents so you have differing directories normally in an r project so right now it's basically a relative reference to where this file is located it's at the same level so that's why it's just the name of the file but you will have to use actual file paths if you do different directories and keep these files in different locations because by default to do r is working at the root directory level of the r project which is why pairing it with the here package is a really good idea i think i've already done a review on the here package if not i will be doing one soon if i have already done it by this point i'll put a card up above but the here package is really useful to be able to say hey we're starting at the root level of the project now go to this directory now this directory and now this file and that way you're able to basically have relative references by starting at the root of your project level but in any case to do our underscore file i want to run it on just my r markdown document so currently we see my r script to do's and my our markdown to do's if i run this function now we only have the to do's for the our markdown document so this way if you have a very large file and you want to want to cut out a lot of the noise and focus on one specific file at that point point in time you can easily run this on a single file so let's say you have an r package an entire encapsulated package for r now maybe you don't have an r project file in this package and that would mean uh oh how are we operating at the at the project level if there's no dot r proj file so in this case what you might want to do is run the to do r underscore package function where it will scan an entire r package for all of your to-do's i'm going to run this and it finds nothing because currently i'm not running a package now if you did this and you actually had a package very useful function it'll tell you everything you need to know you're good to go one really great feature of to do r is that it also comes installed with several different rstudio add-ins so when you install the package with installed up packages you i don't even think you have to load it with library to use these add-ins you just run these add-ins so in our studio we have the add-ins drop-down for different packages and right here i have to do r and you can actually find all the active to-do's in your active file which is the file you're currently on your project or your package if i run file it'll just do the arm markdown file if i do project it'll pull out everything for the entire project which is currently the r script in the r markdown document and we saw earlier that package does nothing because currently this is not encapsulated within an r package it is a project so very useful features these are all add-ins in the add-in menu and sometimes i will just quickly run this if i want the output and it works great okay so we can find to do's you saw that i have examples of fix me and bug but what other things can to do are find by default without any additional functionality or hacking so by default to do r has several things it will find and look for just right out of the box we have to do fix me changed idea hack note review bug question comeback and temp these are all things that right out of the box to do our will find if you use this correct structure of you know start the comment either in this case the code chunk uses a hash symbol and then the actual word i don't even think you need it to be in all caps let's test this out and run one to do this is not in all caps now i saved the document you make sure if you're going to run this on an arm markdown document or something that you actually have to save your file first if you haven't actually created the file on your file system you try to run this i actually got some stack overflows i think on uh rstudio and i actually had to restart my computer so be wary and do not do my mistake so now let's actually run this on the entire project it does not find it so in this case let's actually change it to caps to verify save the document rerun it for the entire project all right maybe it was there hmm actually let me undo that save that and rerun yes okay so it does in fact need it to be in all caps so make sure that when you're writing these items that you always start the comment off and you write the tag in all caps as the first thing directly after the comment is started and then whatever else you put after it will be put into this markers pane to let you know what was i talking about what is this about and a little bit more information about it so that's a lot of different items that you can search for just right right as you start so what if you want to actually limit that down and you want to ignore some things like you really only want to look at two different types of tags and you only want to see what you're actually interested in so right here you can see in my our marketing document i have a tag for an item that's called bug well let's say i don't want to look at any of the bugs i want to look at to do's and fix me's so right here you can just use the exact same to do our function and if you pass in a vector using the c function and you pass in some comma separated strings all caps but you can actually specify which tags you want to find in this case i'm saying hey to do r look for only to do's and fix me's if i run this function you can see that bug has now disappeared from my markers tab for the r markdown document i only see to do's and fixmes and you can find as many of the items as you want but you have to specify them in a comma separated string and this will work for narrowing down which tags you actually want to focus on at that given point in time if you don't want to have to write out the entire list of items you want to do are to look for say that you never use anything other than to do and fix me and you actually want to define your own custom search value then you can actually use the options function and pass in to do our underscore patterns and then give it a vector of strings separated by commas and say hey to do r whenever you run without having any arguments so basically remove this vector when i just run that function to do r i only want you to ever search for these items and you can pass this and you can put this in your dot r profile if you wanted to so that that way whenever you run this function you never have to specify an entire list of stuff it's set once set it and forget it and that way you could say hey i only want to want you to look for twos and fix me and then with this option to do our underscore patterns you can actually define custom things you want to look for because let's say that you have i don't know a note where instead of using to-do's and fix me's you might want to use your co-workers names to make sure that that person comes back or they just use their names to tag these things in which case you could actually custom define the names of these people or whatever and then now it's searchable by to do r and if you use the same structure begin the comment all caps or even case if you specify it that way the actual tag and then the item to actually follow up on then this will work for that so just as a quick example i change this list hey fix me to do and my name and then i make a tag brian this is not in all caps so i i can save this i can run this on the entire project and right here at the bottom you can see now my name is now indexed as one of these search values so you can do this for whatever value that you want now after setting the patterns options argument i actually need to restart my r session just to make sure that i don't limit myself for future examples but that is all options and comments within an actual code chunk in the r markdown document what about other parts of the document what about other comment types so this will actually work with other document types and the way that they do comments so in the code chunks we actually still use the r syntax so the hash symbol for comments and there's no multi-line comments but in our in the actual our markdown space for markdown text where we actually use html syntax and html comments like this you can see that the bug option has been getting picked up so when i run this on project bug does get picked up and it does actually have the html comment tag so the the angle brackets the bang and the two dashes all of this does get picked up so in this case you can have excuse me markdown comments and they still get picked up for these tags so no matter where you leave these tags in these comments it will get picked up by to do r especially if you specify the exact tags you're looking for so far we've only been able to see the tags in the r script file and the r markdown document now i had some other r type r document types open here that we actually put some comments in so i have r html i have the sweeve documents of the dot rnw and you can see in the swede document it's just simple there's a hack tag here and it's a very empty document it's just that comment our html here's a bug you know it's in the r code chunk of this you could put it in an actual html comment either way but we haven't seen any of these comments from these two documents listed here what gives so by default it's not going to look at those document types it's only going to look at the r scripts and the r markdown document file types so we have some other options here to control the behavior of to do r so we also have to pass in this into options so again you can set this into your r profile and forget it and be done with it but we have dot rmd so the to do r underscore and then the actual name of the file type so rmd that's our markdown document and the default for this option is true so let's say you actually wanted to ignore the r markdown document you could set this to false by default it's set to true because obviously it finds them if you watch you want to search through sweeve documents you can set the underscore rnw option with to do r to true so it will actually find sweep documents i'm going to run that now i'm going to run the project and now we actually have the hack option or the hack tag here for the comment in the sweep document now sweeve is basically embedded our code with a specific syntax in a loftec document and in law tech we actually use percent signs for the comment so i can save this i can run that and with the actual law tech comment type with the percent sign it still picks it up so that's three different kinds of comments that it's just finding uh based on the options that we give it so the next one we have is the rhtml document i've never actually used this document i really wonder who's actually using these types of documents but in any case if you wanted to actually have your to-do items in this type of document found you could either use the html comments up here you could use the r comments in the co embedded code chunks right there and setting the our html option to true will actually let you have the option to find all the comments in the our html document right here so that's a really useful option now one thing i haven't said yet is we have all this list of markers okay and what if you have a very large code base hundreds of files lots of stuff to sift through to find what you're looking for that's a lot of things to go hunting down for each and every single one of these commented tag items what do we do so one of the very very useful parts of to do r is that once it builds this markers list of all of these tags that it's found in all of your files based on the options that you've given given it and how you ran the function let's say okay here's my r script here's a to do i want to go follow that up i don't have to open up my r script like watch i can close that r script file i can double click on this marker and it opens up the rscript file and it actually jumps my cursor exactly to where this tag was and it's a marker so it's basically saying hey that file that line take me there if the file's not open it'll open the file if your cursor is not there it'll take your cursor there and i'm right where i wanted to be on that item i can go to the arm markdown ones because now i'm in the r script but i want to go back i can go to this fix me i can go to the bug i can go to the sweep document the rhtml document so with this you could easily jump around and handle all the stuff that you want to do and i think that with the markers tab it'll actually work the same way with lint r to make sure that you're linting your code correctly and it's structured correctly but in any case this is an incredibly useful feature because not only does it find and scan all your documents to look for all these items but it lets you easily get to them and then fix them so let's say we have a project or a package and you're using an actual snapshot of all the items you have installed for reproducibility or a reproducible analysis you might use packrat for this packrat is basically a way of taking a snapshot of all of your currently installed and loaded packages so that you can actually say hey when i did my analysis here is everything i used and the version of it that i used this is a very useful system i haven't had too much experience with packrat but in really in any case all i ran in the in this entire directory which basically you treat it like a project or a package is you run packrat init so you basically init a packrat um instance pack rat and then let's do init and really it's just you know creates a project directory and makes a new packrat project in the directory so really it's just load all your stuff put it into the directory great so what happens when you just run to do r on the entire project or directory now it hasn't been popping up here but this might actually cause an issue later is that you if you do not exclude the packrat directory every single file in that directory and all of its to-do's from all the developers that ever worked on those packages that are snapshotted in there come up now by default the exclude pack rat is actually true which means that you normally do not have to worry about this for some reason if you wanted to actually see all the to do's in your packrat files that are saved you can set this to false the option to do our exclude packrat to false you can run that and then you can actually run to do our project and you'll see an abysmally large amount of all these items from every developer for every package and you can see that you know we have documents we have library frameworks our framework versions 4.0 so even then they are 4.0 resources we have all these to-do items in here and it's a lot i only have like maybe seven or so items but look at all of these so if you want to ignore these you just set exclude packrat to true or in this case you don't even have to explicitly say that it's but it's ignored by default but for some reason if this comes up for you and this is an issue you can set this option to false and or to true and it will ignore all those things and now when i run project it will ignore all the stuff in packrat last but not least is one more option to exclude everything from our files this is basically your r scripts if you leave comments in your r script files like for instance this one has to do do some data cleaning and use the pipe blah blah if we actually set this to true to ignore all those files that begin with or the end with the dot r extension capital or lower case either way and we run this on the package so right here it should be at the top we run project it's no longer located because now these r script files are ignored so i'm not sure why you might want to do that because that's actually one of the file types that i think you would want to actually find but for whatever case if you have a use case for that you can ignore all the comments in an r file this way so i hope you found this video interesting i'll be loading up all of these files like the our markdown document suite etc all into the our package review github repository where i keep all of these review videos and all of their contents and used files in there and you can easily go check that out i'll put a link to that in the pinned comment below and yeah i won't be including any of the pack rat stuff just because it's basically like a copy of all the libraries you're using it's a lot of extra data that i'm just not going to include so it'll pretty much just be all these files that were sitting at the root directory level and nothing that was involved with packrat but if you wanted to do that you could easily just install the packrat library load it and then run packrat in it and you're basically you just did exactly what i did and then you'd have the same you know issue if you wanted to replicate it yourself so without further ado thank you for watching the video uh if you liked it please subscribe check out the channel i make a lot of videos like this and thank you to all the patrons who support this channel thank you devon klaus alberto thank you guys for supporting the channel on patreon and i'll catch you in the next one [Music] you