Comprehensive Obsidian & Git Sync Workflow 🔄️ | Your Mind Under Version Control

Key Takeaways

[Music] welcome back everyone if you're new here my name is brian jenks and today we're going to talk about how i put my mind under version control clickbaity title but true so let's take a look at this i wrote a article on medium about how i put my mind or my obsidian vault under version control and i saw something done similarly by justin from effective remote work using a bash script and using git and github and i added a little bit more to that and put my own little spin on it and i detail in here if you prefer a written medium no pun intended then i give a comprehensive write up on all of this stuff exactly what the script does line by line and this is what we're going to go over in video form today if you prefer to hear this in video versus just reading it so we're going to go over this line by line and just jump right in and why did i even do this to begin with so first of all why did i do this why did i go to the effort of doing this so because obsidian is an application that is pointed at a vault of plain text markdown documents this means that it's plain text and we can subject it to version control so over the course of time i can say hey i want to look at a specific note in here let's do andy matushak and i can look at it throughout its entire lifetime in this vault and say hey what's the life of this note how what's what's happened to it over time and i can say let's look at the history and i can say every time i've modified this i can go back at that point in time and i can actually look at what that article looked like and what changes i made and every time that it's you know had an upload i can actually change and look at all this stuff and this is one of the critical things is that i wanted not only redundancy therefore my vault is not only on my local machine it's actually externally hosted and safe somewhere i can actually see the version control as i change these notes over time i can see what my old content looked like it's no longer like lost in in time and i can always revert back to prior versions or at least look at what i had before without any additional overhead i don't have to do anything i don't have to save copies uh v1 v1.2 v1 final final final final no i just have to put it in git and forget about it it's it's done automatically for me with the way i set up this script and i wanted version control i wanted redundant storage i also host my vault on my own icloud you know cloud hosting so i mean i i'm never gonna lose my stuff uh you know pending world catastrophe but i made this so that i could easily have everything synced onto github so that i can access this anywhere anytime from any computer and so i actually also included some of my workspace and configuration settings uh files so with this i can actually open up an obsidian editor anywhere and my exact editor even exactly the note i left off on everything is completely brought over it's like remoting onto my home computer and i'm working in the exact same session and i wanted this and now i can go i can go to work i can go on github i can just access my file change it on here and i'm good and i might do that by using the github code spaces feature i'm not sure if that's still like a beta thing or not but if i open this with code spaces i can actually edit this with a vs code cloud instance in github and if you're interested in things like foam or dendron you can actually use them in this github code spaces and then use obsidian at home if you wanted to so all the things that come with github makes this a very robust tool set that you can then add to your workflow with obsidian and before i forget if you want the link to this medium article or the gist where i store all the code for this entire script and the entire github and that'll cost in syncing of an obsidian bull all that stuff you want the links they're going to be in the pinned comment below with the time stamps and we're going to go over this step by step so i'm not going to edit this next section too heavily just because it's going to be a lot to chop out but bear with me so we're going to go over this line by line and see how i set all this up what i did and how it all works so the first thing is that you need to have git installed if you don't have git installed install git i'm on mac so you can do brew install git although i'm sure that you probably already have it if it didn't come already um basically for europe whatever your machine is get get so what first thing we have to do is we are going to permanently cache our credentials now you may not want to do this that's going to prevent you from really using this workflow the way i do and you might have to figure out some other stuff but really what this means is that i don't want to have to enter my name my password or any details anytime i want to push something to github to one of my repositories uh maybe this isn't the safest thing maybe this isn't the most uh well this is definitely a volatile thing so be cautious but what this is doing is saying it it doesn't prompt you for any credentials ever so you're going to do git config and i'm not sure why i had to do osx keychain but git credential helper store and yeah i'm not sure why it's saying.convig no so you're going to do that and then i think i actually have to do os x key chain for some reason because it's in my um saved keys here no stop and there you go so you're going to permanently cache your credentials which are in in my case osx keychain i believe that's what i had to do and then store is probably for people who are using password storage but in any case maybe this isn't the right thing i think i had to google this but if you need any help with that let me know if you run into any issues and maybe we can figure that out so first thing we're going to do is you're going to go to your obsidian vault you need to go to the vault and actually put a get ignore in there so i actually have my obsidian vault hosted on icloud my file pathway to there is ridiculous so i just wrote an alias that i can type icloud and it takes me there i don't have to type any of that because look at that that's that's a ridiculous amount of crap to type so i should probably zoom in too all right so i'm in my icloud folder and i'm going to go to my vault and i'm in my vault you can see i'm in a git repo obviously so we're going to actually create a git ignore so to do that we're going to say touch dot get ignore this is going to create the dot get ignore file so i can actually open up mine let's do bam dot get ignore so all i have in here is you know these four different things now in for this article all i did was the first three because when you're going to use code spaces on vs code you want to ignore the python environment it puts in there it's a bunch of extra junk you don't want so that's if you're going to use code spaces on github otherwise you just need these three things in here so from the command line if you don't want to use vim or whatever all you have to do is do echo open up a double quote i could do dot obsidian cash because you don't want any of that cashed information now we're still in an open-ended double quote so i can just hit enter for a new line in this uh file dot trash because i don't want any trash and then if you're on mac we have issues with the dot ds store files and we don't want those close off the quote and now we're going to push those changes or in this case overwrite in my case i'm going to actually append you can just do a single one because this is going to overwrite the entire file the blank and and get ignore file with the contents of this quote string which is these three files to ignore so i don't want to overwrite mine i'm just going to append but you should use a single angle bracket and we're going to put that into dot get ignore now what that looks like is i can open up my get ignore and i can see ah i have new stuff in here obviously this did not work as intended because i have my own file in here already but what that would do is it would actually just create these first three lines for you which ignores those things and that is what we want we want those things ignored so next we're actually going to make your repo a repo we're going to make your vault a repo so wherever you have this hosted it could be your local file system you could have it on icloud like me or a dropbox folder whatever you're going to do these three commands i already have a github repo so i'm not going to do this again but you're going to say git init which makes your current directory your folder a repo for a git repo you're going to say git add period which adds literally everything everything is added to your git repo and you're going to commit those changes saying hey i made changes the changes are i added stuff to the repo it was empty now it's not git commit commit dash m which is saying hey i want to add a commit i'm going to add that commit with a message the message is you know init you initialize this repo cool now it exists on your machine but not on github so what you're going to need to do is go to github and then create a new repository so if i'm on github i need to actually make a new repo you need to have a place for this thing to land so you can go here create a new repo and you can probably intuitively figure out that interface get the name and the uh your username and the name of the repo so if i'm on one of mine for instance it's tall guy jinx business card for me this repo is tall guy jinx settle costin so um at least i think so yeah yeah that'll cost them so in that case you're going to do in the command line once you've made your git repo your credentials are connected and everything's done you can actually push changes to github so what you're going to do after that is say git remote add origin and then the name of that repo and then it ends in dot get so what this is doing is this is saying hey i have a repo on github but my local repository the git repository we just made it doesn't these things are not connected they don't know each other exist so we're saying hey knock knock remote repo i have files here and changes i'm going to send them to you so you're going to do that you're going to say hey git remote ad origin this is the origin of your current local repository now we're pointing it at that saying hey that's where you came from now this is where you came from now so it's saying okay that's where i came from but none of my stuff's there yet i'm still on your local machine get push dash u origin master so this is saying is you're going to push your current branch everything that you have all your changes your new repo up to that remote repo your username slash repo name and set it to the master branch saying hey we got a new repo it's the master branch we are good to go we are quote unquote in production we're good to go so once that's done once that's done you can go to any place you're going to store this script so if typically if you're on a linux system which i think that really i think other people have adapted my workflow here to windows but i think i only made this really work for mac and linux but you're gonna go to your dot local share or not double local share dot local bin for binaries and you're going to create a shell script so actually let's go to my home directory cd dot local bin and then the script i named is just zk underscore sync that'll cost in sync touch to create it chmod plus x and the name of that script what that is doing is that is making that script executable it is something that can actually be run as code so if i go and actually look at my zksync script this is all it is that's it very simple and we're going to cover what every line does so what you're going to do first is touch make that file chmod plus x make that file executable and yeah so what we have here is some stuff that comes after so yes okay so we don't need to worry about these last two lines until we're done with the script because we don't want to add a cron job that the file is not doing what it's supposed to do so after you actually made your file executable we need to actually input all of this code and this is exactly the file i have but without all the comments so what we have here is a lot of comments um the first thing we have is the shebang what this is saying is hey i want this to run in you know the first instance of shell that you can find and it's sh for it to be uh posix compliant shell which means it is more portable than saying bash and it's user bin n for environment so that it font uses the first instance of this this is just to make it incredibly portable so it can work on multiple machines your zk path is a variable we are assigning and that is going to be the absolute pathway to where your vault is stored you can see that it says users my name library mobile my icloud stuff knowledge knowledge is the name of my vault if i went to icloud you can see that if i went to knowledge and print the working directory it is exactly what you saw there see so it is that complete absolute path we're saying hey this is where my repo is so what the script is going to do is in when the script is executed whatever instance of a shell that it creates for the script it's going to go to that path it's going to go to your vault the first thing it does is get pull this is saying hey if i was not at my home machine if i was over there way over there on a different machine making changes i'm on github using code spaces or obsidian on another machine and i made changes to my repo on github i want those changes give them to me onto my local machine sync them and bring them into your repo basically if i make changes and i'm on another machine or i'm uh if i'm at work and i make a quick change to a note on github to one of my notes when i get home and open up obsidian that change is going to be there waiting for me that's what we want so then we can also say hey pull any changes that happened while i was away from home but now what also runs is if there aren't any changes that happened externally nothing happens but if changes exist and what we're doing here is saying hey changes exist is going to hold a value what's the value we're going to run this sub shell and we're going to say hey git status dash dash porcelain what does that mean what does that do so we could do get what was that command really quick get status so git status dash dash porcelain and what is that going to do what that is going to tell you is a very simple line by line interface of what changes occurred if i just run git status i have a shortcut alias for gs git status you can see it gives you all this other output you know indented modified delete we don't we don't want that we want single lines why do we want single lines because we want to know that even if no no changes occurred what we want to know is hey porcelain pipe that into word count dash l wc wc is word count but we don't want word count we want lines so word count dash l what that tells you is that there are 13 lines basically 13 different changed items either modified created deleted there are changes that's what matters so the number 13 in this instance is stored in inside of changes exists because we're asking a question we're saying hey changes exist and so what happens is this if statement is saying hey if changes exists which is the number 13 right now is equal to zero then exit the the script doesn't need to run there's no changes we pulled any changes that happened remotely so if nothing happened here we don't need to do anything more nothing happened nothing to push nothing to pull we're done good so that's what this is doing is this saying hey if there's anything that happened locally now we're going to go and put that up to the remote repo but if anything happened on the remote repo pull that first but if there's any changes here push those next but we have 13. the the changes exist isn't equal to 13. it's not equal to zero so we're not going to exit we're going to continue on so next thing it's going to do is before i push anything to the remote and cause any git conflicts first thing we're going to do is we're going to get poll again we're going to make sure that changes exist make sure we pull the absolute latest changes because let's say something weird edge case i'm editing on github and obsidian at the same time or or if you work collaboratively on a vault with other people this will not prevent but help prevent that changes occurred both remotely and locally at the same time you're saying hey give me remote first then push local and so we're gonna pull again then we're gonna add everything add everything in the entire vault except the things in the dot get ignore because we're ignoring them which is the stuff we don't want cash trash and whatever else so we're going to add everything all the changes and then the commit message we're going to do dash q because we want it quiet i don't want to see any output from this script it's quiet it runs in the background and the message is last sync and then i put input a timestamp because i really like my timestamps and so anytime that there are changes whatever files are changed you're going to have a commit message update of lastsync and it's going to be that timestamp and then a quiet push we're pushing all those changes to the remote repo and that's it this is all that this script does and so this whole thing it works i could just run this script every time i want things updated but i don't want to have to do it manually every single time this is where cron jobs come in cron is something is a command line utility that will actually let you schedule scripts and jobs to run on a chronological timer so there are several resources to help you with this and i will put them in the link below and i've also put them in the medium article and i had some helpful suggestions on additional resources as well but cron is really useful for scheduling scripts to execute on an interval and how we get there is if you're on mac at least what i found out with mine i bought a brand new mac i already had cron installed so you might need to install chron maybe not but when i was on arch linux i installed crony to do this um crony and that's what i installed on arch linux to do it but once you have cron or some cron job system installed i'm going to open mine with cron tab dash e which is edit my crontab i only have one job in here which is you know syncing my my script using my script and syncing it so what is this all doing i'm not going to go over cron jobs in detail there are resources and helpful links for that but really what you can infer from this is every 60 minutes every hour i run my script my script is located at the absolute path users mine my username dot local bin zk sync it runs my script now i set everything to dash queue quiet but just in case there are any issues or anything like that i set this up in a way and i use it in a way where i don't really have any issues with anything but i don't want to see any output i don't want any weird aberrant behavior so i'm sending any output to dev null basically saying hey if there's any output send it to the void i don't want to look at it and it sends standard output and all i forget exactly which of the streams that this is referring to but basically it sends all of my output to dev null i don't want to see anything just execute in the background on a one hour timer and then once i save and quit editing my cron tab no changes but if there were changes it would say hey new cron tab installed and you're good that's it now you can go about your obsidian editor you can make changes to your notes i can say hey let's go to my home i'm going to make a change to my index and i'm going to say hey i'm going to add a line here hello youtube and now i can go back to command line i could say hey i'm going to go to icloud i'm going to go to my fault i'm going to say git status and index is now modified awesome now i'm not going to wait for a timer because obviously i'm filming right now i'm not going to wait so i'm just going to run zk sync and it's going to give me see it's giving me output i don't want that so that's why we send it all to devnull but it's going to run all of my my scripts and now you can see hey it was yellow because there were changes it's green because now there's no changes get s no changes there now if i went back to my zettle constant repo and showed you what the files look like in there if i go to my index file somewhere wherever it is index you can see now hello youtube and just like that everything's backed up now for those of you who are not as technically inclined or don't want to have to go through all of this work or figure out all this stuff or deal with it there is a github not github there is an obsidian plugin now that i believe they said they were inspired by my article but they actually referenced my article but in the developer plugins now this is an insider install by the time this video comes out it might actually be publicly available we'll see but obsidian git what this does is this actually lets you do exactly the same thing i'm not i haven't set it up myself simply because i use my workflow obviously but this is a way of installing a plugin to help manage this in a way that uses a little bit more of a gui interface so i can go to obsidian git let's turn it on obsidian git and it'll say hey what's the interval you want to what's the the commit message date format commit message branch pull up on start all this stuff to help you manage your syncing of your vault to github so multiple ways to do this whether it's tech friendly or if it's just a plugin whatever you want to do multiple ways to get the job done and in the end of the day what i cared about is i wanted my content saved backed up under version control and with plain text we can do that and it's awesome so links to all of these materials my article the code the script everything is going to be in the pin comment below with all of the timestamps for this video so do check those things out and yeah i hope you enjoyed this and it was a fun project to make and it works beautifully for me i edit my files on my remote computers when i'm off off away from home i can edit my files come home my changes are already there and i can just pick up right where i left off and it syncs my workspace pane layouts and even what note i last had open on any of my machines because of the way some of this data is stored in certain files so basically i can open up a session leave my computer go to another computer and open up that session it's awesome if you'd like to talk more about this type of stuff or connect with me in any way i have a discord server link is in the description below i do stream on twitch and if you want to support the channel you can do that by watching the video using my affiliate links below to buy anything from amazon uh you could be a patron one-time donations buy me a coffee whatever you feel like at the end of the day anything is appreciated so a quick note before i go thank you to the patrons who support this channel devin ed klaus brendan alberto and john thank you guys for supporting the channel and i'll catch you all in the next one [Music] you