How To Schedule Python Scripts As Cron Jobs With Crontab (Mac/Linux) - Python Task Automation

Key Takeaways

hey guys today i want to show you how you can set up a cronjob to automatically schedule chops on your system and then i'll show you how we can run python scripts with a crown job it's not too complicated but there are a few pitfalls where we have to be careful so these commands only work on mac and linux on windows it's a little bit different but there you can use a tool that's called task scheduler which is also not too complicated so you can still benefit from this tutorial by learning about cron jobs in general and the pitfalls when we want to run a python script so let's start [Music] so on mac and linux we have a built-in tool that is called crontab and we can list all the available jobs that are scheduled on my system right now with saying crontab minus l and then we see this syntax so we have five different options to specify the time and then we have the command and right now this is commented out like a python comment with a hashtag so right now this is actually not scheduled so before we learn how we can run python script with this let's first learn about this syntax and for this i recommend a very nice site that is called crowntop.guru so here you can play around with all the available options and then you get an immediate feedback when this should run so as i said we have five different items the first one specifies the minutes then the hour then the day of the month then the month and then the weekday and for each item we see all the available options down here so we can just use an asterisk for any value we can also use a list a range or a step so we will see how this works in a couple of moments and then we will see all the available values so for a minute we can use values between 0 and 59 so right now if i just use an asterisk then we see this should run at every minute and we also see the next time when this is scheduled and we can expand this and then see all the next times so this is really helpful so i recommend to play around with this so for example now if i use a 0 here then we see this runs at minute zero so essentially at every full hour then if i use a value for the hour so at every hour zero um then we will see that this runs at every minute for the hour zero then the next one specifies the day so here we can use values from one to thirty one so if i just use one then this runs at the first of the month then we can specify the month so here we have values from 1 to 12 so if i use 1 then it's january 2 is february and so on and the last one specifies the weekday so here we use values from 0 to 6 and 0 means sunday so yeah these are all the available options so let's do a few examples to get familiar with this so first let's say we want to schedule a job at midnight on every first of the month so for this we use a one here so for every first day of the month and then for midnight we say hour is zero and also minute is zero so don't forget to um replace this asterisk so a lot of beginners forget to set this and so this means that it would run for every minute at hour zero so yeah zero zero means midnight then for example let's say we want to run a chop at each hour on sunday so for this we specify the weekday so as i said zero is sunday and then for each hour then we have to set the minute to zero and we also see the feedback so this means at minute zero on sunday and then for each hour so if we expand this then we see it's at zero at one am two am three am and so on then let's have a look how we can use these list and range and step options so for example let's say again we want to schedule a job at midnight on the 1st of every month but also at the 15th of every month so again we use zero zero here and then for the day of the month we can use one and then a comma and then multiple options for example 15 or 30 and then we see this runs on the first the 15th and the 30th so this is how we use a list and then let's say we want to use a range for example we want we want to run this for the first seven days then we can say one to seven here and now let's say we want to run this every 10 minutes so then we use this asterisk and then a slash and then the step value so in this example 10 and then this runs every 10 minutes and we also get this here every 10th minute and then only for the hour zero and for the first seven days in the month all right so again i recommend to play around with the site so now let's have a look how we used the syntax and now schedule a actual python script with this so for this let's go back to the command line and now instead of saying crown top minus l we say crontab minus e this is for edit mode and now this will open up the default editor so if i quit this again so and echo my default editor then you will see that this is vim on my machine so if you're not comfortable with using vim you can also change this by saying export and then dollar editor and then the new value for example you can use the nano editor and now if you echo this again then we should see the updated variable and actually a mate aims take here so when we export this then we just say editor and this is nano and now when we echo this we have to reference this with the dollar sign so now it should work so now i change this back to vim so i'm not a vim expert but i'm comfortable with the basic commands okay so before we schedule this in our crontab file first let's have a look at the actual python script so for this i have two examples and what we do here is we import logging and then we set the file name and then we create a logger and we create a file handler so we want to lock this in a file and if you don't know how to use logging in python then i have a tutorial about this that i will link here so check that out and then we set up a simple function define do logging and here we simple say logger.infotest and we also want to say if the under name equals main then we want to execute this function so the first thing you should remember is to use this syntax and only run this as your main process and the second one is here to use the full path to this file so if you just say file name equals and then for example test.log then this will run if we execute this from in here so now let's say python tests dot pi and then we see this log file appeared here in the folder but if you run this as a cronjob then this file might actually end up somewhere else so we want to use the full path here to this directory so let's quickly print this whole file name and see how this looks so now if i print this again then we see this is the full path to this file so i recommend to use this so now let's see how we um schedule this as a cron job so for this let's go back to our terminal and let's clear this and then let's say crontab minus e and then go into insert mode and now let's specify the time so for this simple example i want to run it at every minute and then the command is python and then the full path to this file name so i copy and paste this in here so this is in my home directory slash code slash python task automation slash test.pi and now i say escape and then save and quit this and now it says crontab installing new crontab so now this should run so now let's actually have a look at this folder here and now it should lock into this file so now let's track the current time so i can go to time dot is and we see oh yeah actually already it already worked so we see the next logging event happened here at 3 5 p.m so now let's actually wait for another minute to pass and see if this works all right so the minute is almost over so now it should yeah it worked so now we see the new logging event appeared in our file so this works so now let's actually say crontab minus e again and then let's comment this out again and save this file so now let's have a look at a second example and for this we use a third-party library requests and then we do the same thing so we set up our logger and now what we want to do is we want to send a get request to the github api and this is by the way a public api so we don't need a token or anything for this and here we grab the latest events from my account and if one is available then we simply lock this last event and then again the same if name equals main then we execute this so now the difference here is that we use this third party module and of course we have to install this so now if i simply run this by saying python fetch github dot pi then this does not work so for this we get a import error no module named requests so now to install this i actually recommend to use a virtual environment and for this we have multiple options and i want to show you two of them so the first one is to use a conda environment and the other one is to use a vnf so for the conda environment let's list all the available conda environments i have here by saying conda and list and if you don't know how this works then i also have a tutorial about this that i will link here and now we see i already have multiple environments here and the one that i want to use now is this pi 3 8 environment so in here i already installed the requests module so now we want to say crontab minus e and then for the python command we want to use the full path to this environment so now let's go into edit mode and i'll actually copy and paste this in here and then i will show you how this should look like so you need a path to this environment and then slash bin and then slash python and then again the full path to your file name so now if we before we actually save this let's test this command so let's copy this and then let's go to our editor here to our terminal in here and now copy and paste this and now this actually worked so yeah we see our events file got created and here it locked the latest event from my github account so now let's delete this again and then in here let's um save and quit this and then again it says crontab installing new cron tab so now if we um have a look at this folder again and now if we track the current time so now we have to wait for the next full minute and now the minute is over and yeah here it appeared it took around two more seconds but that's okay so yeah this worked so now let's go ahead and again say crontab minus e and let's comment this out again and now i want to show you the last option so for the last option i want to show you how to use the vm so of course first we have to create one by saying python 3 minus m v and vm then we want to activate it by saying dot v and slash bin slash activate and now we see it got activated and now we say pip install requests and now this was installed so now if we say python fetch github.pi then this should work in here so if we go to the events.log then we see a new entry appeared so this worked so now again let's say crontab minus e and now in here let me copy and paste this command so now basically this is the same as this one so the same full path to our file but now here we used the path to our vm that we just created so it's the same full path to this folder my home directory slash codes slash python task automation slash vnf slash python and then the file name and now let's hit escape and save and quit this and then again it's installing this crontab so now again let's have a look at this file and have a look at the time so we have to wait five more seconds and now again this new entry appeared in here so yeah this worked so now you know how to do this so now let's again say crontab minus l and list all the current cron jobs that we have so yeah remember um to use the full path to your file and if you use a virtual end then you also have to specify the full path to this python version so that's what you have to remember and then you can go ahead and schedule your own chops with this crown tab so yeah i hope this was helpful and you enjoyed this tutorial and if you liked it then please hit the like button and subscribe to the channel and then see you in the next video bye you