Setting Up a Full Bitcoin Core Node with Docker and Ubuntu 22.04

[Music] all righty guys what is up de daddy here today and um this is a video that I've been wanting to make for a while it's something that uh I work on um you know pretty closely to one of my research projects and you know in the last couple of days Bitcoin has made uh 52 we highs uh as far as price goes so that it was appropriate to go ahead and and make this video uh for several reasons but basically what we're going to be looking at today is how to set up uh a full Bitcoin node and uh what that is is a node or essentially extension of the Bitcoin Network that validates uh and keeps transactions basically keeps a copy of The Ledger um like on that local device or on that that node shall we say so so this is different than mining you're not actually you know uh using compute to uh go through and actually um you know put transactions onto the network or or or mine or put I guess best way I could say it is put blocks onto the network um trying not to get a rabbit hole with all of that but uh it's different in the sense that you're basically just keeping a copy of the network and you're allowing uh other people to in distributed way get a copy of that from you so that everybody has a copy of the agreed upon um Bitcoin blockchain so um that's basically what it is and the next question is why would you you know why would you want uh to do this and one reason might be that you just an alteristic person you want to contribute to the health of the network Because unless people do this uh you know the network basically falls apart um another reason is you know you uh like enhance privacy so for example if you have your own node um you have a full copy of all the transactions um basically you don't need to go to any other thirdparty server to uh validate transactions you have your own full copy so there is no reason to go to any other server than your own so um you know for example you wouldn't have any logged requests other servers nobody would know what you're doing um so that's one reason people choose to do it um it's more reliable because like I said you always have a copy on hand you don't have to depend on other nodes being up and running um you know like I said supporting the ecosystem um you know for learning education or research purposes which is kind of what we're will be doing it for today um and then basically some people just like having you know control over their um you know own node and kind of with that you can uh have your own wallet and you can manage all of your own personal transactions through the wallet that's attached to the Bitcoin core um so you know there are some other you know really interesting things that uh can be done with your own node but for us today what we're going to be looking at is we're going to be looking at um you know setting ours up for basically research purposes and for my own purpose I would like to have a full copy up-to-date copy of the uh entire Bitcoin Ledger locally available at all times so that's the that's my main objective um and I'll explain that why I I I want that a little bit later um but for now let's look at the requirements so you need any OS is fine pretty much there's a lot of different options Windows Mac Linux um you could run it on uh run it with arm so run it on a Raspberry Pi if you could you know uh if you can make that work seems a little bit you know um resource light to me but that's you know beside the point um you need seven gigabytes of free disk space which this also seems low I think this is if you're running a pruned copy um which means you only have like the some of the more recent um blocks available you don't have a full copy of The Ledger um you also need to rewrite speed of 100 megabytes which you know most people would have these days um um you need two gigabytes of memory which again most computers are probably going to have these days um you do need a reliable internet connection um and at least here in the United States this should be you know definitely something most people have however this does use a lot of data and um you probably are going to need some kind of unlimited plan otherwise you're going to especially at least in the first IBD or the first part which is the initial block download uh it's going to be pretty uh data heavy and you're gonna end up going over for maybe a month or the first month while you're um getting most of the blockchain downloaded um and then also one thing that's nice is if you could keep it on six hours a day that helps contribute that's kind of the altruism thing where um you know you're not only helping yourself but you're helping others because they can access your node to um you know get a get a copy more or less all right so that's the the main things here before we really jump in um so what we're going to do here is we're going to start I'm just going to show you the node that I have up and running so to give you some context this is one of the main server I have downstairs um and we are running a container on it it's been up for about three hours um and let me just get rid of uh let's clear this out um and and inside so if we go look at the other we're actually now inside of the container so this is what everything looks like uh inside and um if we look at the blocks we can see that it is just in distributed way downloading all of the um blocks from from all the peers shoving them in dat files and then they're all ulating um you know in this directory so the way that this works basically is um other peers on the network will have a copy uh of The Ledger and you are basically accessing as many peers as possible and you're saying hey I need blocks and they're just you know packing up and send packing you up and sending you the blocks you don't have uh and then you're storing them in these basically storage files called dat files so every time that you fill one up 128 megabytes you'll move on to the next dat file and so on and so on and so on and these are basically just great big old binary storage devices so that's how this this kind of works it's almost like a bit torrent where you're just you know downloading kind of in a distributed fashion from from peers on the network uh it works a little similar uh in in that regard but anyway so this is how it works and if I keep refreshing the page here you're going to see that this number keeps growing and growing and growing um but these these block.at files BLK do some uh five-digit number. this is what actually houses the uh block header and all the transactional data um it is in binary so so for me I I have written a parser that actually goes through and parses out all of the binary data into a Json format but you could use any format that you want XML whatever um and this way it's readable and you can actually go in and you can start you doing some some analysis uh of the underlying uh transactions on the network so just a a validation of why this is important or why you might want access this information um that's why I I do it why I'm setting it up so let's um get into it as far as what you need to do to do the same so I I've typed out all of the commands what you need to do first is um and we'll swap back over to the larger container because we actually you know ideally we need to start up another uh container I've already got this one running and it's it's downloaded quite a bit of data so I don't want to reset it and because of networking I don't want to have to reset up a whole another container because it would be a pain so I'll just explain all of this um and at the end I'll show you how to start and stop and and get everything set uh get everything going and then I'll also put this in description so this should give you you know pretty much step-by-step instructions of how to set everything up but what we're going to do first is the we need to create um a Docker container so the first thing we need to do is we need to make sure that our container set up correctly so um the command here is we're just going to run a Docker container we want to run it detached we want to run uh with interactive shell we want to also map a volume so for example this is kind of where it comes into play You're if you especially if you are not using a pruned version you're using the full version of The Ledger you really need to think about where you're going to store this because for me the reason I have everything here in this LTS this is long-term storage I have about 35 terabytes worth of storage here so more than enough to handle you know the probably 600 gigabytes right now that the blockchain is so anyway you need to plan a little bit and as far as where you're going to be storing this because it's important if you don't you might run into a case where you're not even able to download the entire chain or you know in a year it might might be you know something where you have to figure out how to increase your storage so anyway just keep that in mind and this is the reason why I've set mine up this way um and then it's never a good idea to to keep your your information in a Docker container I always like to have information in and underlying uh volume on my server so that way in case something happens to the container nothing happens to uh this data and I'll have to redownload all of it so anyway uh this is just a volume that's this is the phys physical machine and this is the container and they both share this particular uh file and then for me I like to map some arbitrary port to port 22 so I can SSH into my container uh personal preference there and then I also this is the mandatory mapping so this is 8333 uh which is the port that Bitcoin uh nodes um share data through so this needs to be open and available um and since we're running it on a Docker container we need to map it to uh some external port on our server um in this case I chose 5833 um because I don't like to really map uh for a number of reasons I don't like to map uh the port to a its exact port number on my uh actual external server that this container is hosted on um and this case it probably would be fine I just like to use these ports I know that I'm never going to use for anything else um it gives me more flexibility and sometimes you'll run into issues where you'll have two programs that have to you or use the same port and one is really difficult to change and I've had that happen before so you know these situations I always just like to map it to some arbitrary higher Port that I know isn't really going to be used for a lot um and also for security I can map this all the way through to the uh to the router and I don't have to uh you know use the exact same port it's a little bit less obvious for security reasons so anyway not really any real reason it's much easier just to map it to the exact same port and in fact you can totally do that you can just take the five away and map it to the same port and I'll explain a little bit better later with a diagram but one of the main reasons I think it's good to use this in a container not only for security but also for resource resource management um you can set the number of CPUs and the memory that you're going to use this container which will um you know make sure that it is constraining itself to a certain amount of resources on your local machine um you know and depending on what the resources you have at your disposal this is you know more or less important um and then here we're just setting some kind of uh name that makes sense that we can remember so I don't well I'll go ahead and leave that so I'm GNA have to change it to run it uh and then risk using the regular iunu uh $22 .04 image which you can pull down from Docker so you can really lit write the command Docker pull bon2 22.4 and as long as you have Docker installed correctly it will pull it well and you spell auntu correctly it will pull down um that command or pull down that image and then Ben bash just allows the container to stay alive uh basically so that you can can uh work on it just opens a shell in the container um all right so the next thing that you would do is you're going to run pseudo appt update pseudo appt upgrade um install pseudo you're going to install these uh you know optional here you're going to install these uh basic packages and then you're going to create yourself a user so for me I created um just uh user oh sorry I were so let's just say that you created this you created the docker container and just for um to show you that it works go ahead and we'll change these and we will go ahead and run it there we go so as you can see it's up and running now it's basically a copy of this just one port higher and uh named a little bit differently so I'm going to go ahead and kill it because we don't need it all right so now let's hop back into the actual container so this is BTC core container we've got going and so in in this container you would run these commands and then you would uh create yourself a user so as you can see here we have user um the password I've set to this you know set set however you want um give it pseudo permissions um okay and then I always install open Sage server you you know feel free to do whatever you you'd like um and this is just basically setting up uh changing the shell the default shell to bash and then installing SSH keys so it's easier to access but again these are personal preference you can also um just go and attach a shell to Docker and you can work through that it really makes no difference um okay now that we're in the container here what we need to do is we need to come down here and again depending on what you're doing if you have your setup same as me where you're just using some volume um this is the volume that I picked so everything is going to need to be in in this BTC core uh folder so CD into BTC core and then run WG so this is going to pull down uh a zi version um or a tar file a zip tar file of the Bitcoin core uh 25 five so then you can unzip that with the following command um and then inside of here you can see the bin with all of the uh the CLI different utilities you know all kinds of stuff uh a config example configuration file a read me you know all that good stuff then what we're going to do is we're going to take um basically this this Ben so all these executables here and we're going to do what's called install which basically just means hey put this here and assign certain permissions um so that you know that they can be run so this is the the uh new place where the copy is going to be and your user by default has access to the spin so you should be able to run uh basically any of these executables that are in the bin here as the user named user and you should be able to ACC you should be able to run them um so that's the beauty of this install here all right next thing we need to we need to address here is we need to look at the firewalls um port forwarding and we need to look at uh address reservation so this is the networking here is really I think probably the one of the the most challenging part um so sorry let me pull up my whiteboard where I have all this drawn out and here we go just give me one second to load all right now one thing I did forget um unfortunately and I'll just draw it in really quick is there's a f wow that was bad there is a firewall that sits in between this port forward so this is with u FW uncomplicated firewall so we'll address in just a minute how to how how to set that correctly and then the other thing we need to be aware of is we need there we go green uh is we need to make sure that this internal IP address that we have for our server or whatever machine is hosting this container we need to make sure that we set uh an address reservation otherwise when we do go through all this trouble set the port forwarding your DHC server on the router May uh you know if you turn this computer off or reset it or whatever it may assign a new IP address and then this for forwarding will no longer work so let's go ahead and let's walk through this um step by step it's not really it looks complicated but it's really not um so you have the public internet right and you need to interact with the public internet to get information from the peers on the Bitcoin Network about um about the the state of the blockchain you know about if there's any new blocks available um and you need to pull those into your own system so you can have a complete copy of The Ledger so um what that's going to look like is you have a public IP address so if you do something like type in what is my ip.com um then it will show you your public IP address and then the other peers are going to basically find you and reach out or be able to communicate with you through Port 8333 or they're going to look for that port to be open on your home router and that's what you're going to send and receive traffic through so the first step is going into your home router uh and then setting up a port forward to whatever internal system or whatever internal IP address is correspondent to the Home Server or home machine that you will be hosting this container on so uh for example in this case the server uh so this guy right here this for 4 6x1 server that I have here um is hosting this Ubuntu container so in this port or in this container what we're going to do uh is we're going to tell the router say Hey whenever you receive traffic on Port 8333 we want you to forward it to this internal IP address and this is just a made up one but 192.168.255.255 this particular Port so 58333 so okay so basically all traffic that comes in is going to be sent to your the think about the outside of your machine so your external machine which is the server itself and the server is GNA say okay we have an internal Docker container running and whenever I get stuff on this port we're going to send it in to uh 8333 8333 and this is normally Docker uses an internal networking structure of like 17223 like whatever so just to kind of map out what it would actually look like but what that's going to look like is they're going to you're your external server is going to receive it on this port and then it's going to send it uh internally because we set up our Docker container sorry we set up our Docker container with the following constraints we said hey when you when the external Port so the external machine receives stuff on 58333 send it to the internal container Port of 8333 so um that's what the the uh server that's running on the container so if we go back one more time to this inside of this there's going to be another little box if you think about it where there's going to be basically the the Bitcoin server so if you think about this is going to be BTC that's running in here that is it's going to be looking for data that's coming in on this port 833 so all of this convoluted stuff we just did is to get the data from this port on your home run router all the way to this internal port on Docker container so that the server or the service I should say that's running inside of here the hosts Bitcoin core can get the data from uh the external Port of or the the docker container Port 8333 so know a little bit complicated um and again one one thing you could do is just make make everything Port 8333 you know there's wasn't really a reason for me to do this uh you know I just sometimes I like to make life more complicated but for all intents and purposes you could just make it Port 8333 8333 instead of 5833 and then just have everything fored all the way through and it'll be the same thing so and the address reservation so this is going to be at your router your router has or most people's routers these days have their own DHCP server um and you need to go in and basically say hey uh do what's called setting an address reservation and that will basically say every time that this machine uh so that's just too complicated basically this will be the IP address for this particular machine forever and always so you're assigning IP address to a MAC address and regardless of whether you reboot your machine or whatever happens that that IP address will always remain um uh that machines so you'll have to worry about it it changing or being reassigned um so and do that indefinitely Or do it for like whatever the longest leas period is so you don't uh forget and you know uh you know like a year's time it it it it resets but anyway um the next the last piece is we need to talk about this ufw um or uncomplicated firewall which may or may not apply depending on whether you have a firewall set up um but for me so back again to the uh server so in the diagram it' be this blue guy and there's a firewall that's around it and what we need to do is we need to make sure that we allow um so let's look at pseudo ufw status so as you can see here we have um TCP or sorry I'm I'm allowing in uh Port 22 so this is basically SSH so I can SSH into the server uh and then I've also allowed in Port 58333 58333 um which corresponds to our picture here 58333 otherwise um any if you don't open that up this firewall is going to block any incoming traffic that goes to that Port so just be aware that your firewall if something's not working your firewall actually may be an issue and you might need to make an exception uh in your particular firewall to make this work so all right cool so that I think addresses uh the networking stuff I know it looks complicated um it's really not that bad um one thing I do want to mention here is I I wish I could give you guys more detailed information about how to actually set up the port forwards and the address observation and those kind of things but this is really um machine specific so for example my Deco x55 router has a app that I can go in and go into advanced settings and I can you know do uh address observation and port forwarding all through the app pretty easily um but you know some have uh like a basically a web hosted server inside of them and you have to go you know interact with them through that you know this really depends on your server so my best advice here is just look at the manual for your particular device and there should be either the instructions there or maybe a good YouTube video on how to do it um you know so give it your best shot and if you need some help just leave um a comment and I will do my best to to help you there um okay so next thing we need to do is we need to make uh a directory called Bitcoin so uh by default everything that the Bitcoin core does is through the whatever user that you're you're using so I know it's kind of confusing I named my user user but you know whatever this is whatever you're signed in as um it's going to use that as the default location to look for um this folder and then the configuration file lives inside of it so what that means is inside of this Bitcoin this Bitcoin folder there's a bitcoin. comp configuration file so what we need to do is first we want to create this directory and then we want to copy from this thing we downloaded and extracted we want want to copy the configuration so we want to copy this from here into this directory and then we also want to update uh one value so we want to update the data directory and this just specifies basically what folder all of the data is going to be stored in so in our case we want it stored on um you know here in this Mount BTC core right because on the underlying server this is in long-term storage where we have 36 terabyt of memory or sorry 36 terabytes of storage um so in this case I I know it's going to be safe and I know it's going to have enough enough room to have house all of the data um so anyway once you set that up the rest is pretty easy the next thing we're going to do is we're going to run um Bitcoin D and just to show you what that looks like I'm going to go ahead and uh so I have it running in the background right now so I'm going to go ahead and stop it so the command is Bitcoin CLI stop so we'll go ahead and we'll stop this guy and we'll just give that just a minute and then what we'll do is we'll run it in the foreground so we'll run it in the terminal and it will get you know it'll show what the spool up process is like um so you can kind of see how all that looks so let's see if it's ready so there we go it's spooling up um you it's running all of its checks checking for a cookie file um there we go all right and then it's actually going to start connecting to the peers again and it's going to start downloading uh and grabbing data so the initial process is a little bit slower because it needs to to to validate um basically every every block it goes through like a validation process for every block before it actually starts starts ingesting them so it could take a couple of minutes before it actually starts the first time but once it does it it's it's pretty quick you can start it and stop it pretty easily um and then um so one thing we'll do here is while it's running we'll go and do some other checks so one thing you you want to check once you have everything set up is run this command and this is going to go out and connect to uh some of the peers and it's going to grab some blockchain uh info excuse me all right so here we go we can see that we're still uh in the initial block download so there we go um so this is the current best block hash the current difficulty uh what chain we're looking at um so you can see a lot of different stuff and it's just it's grabbing this it's grabbing this and presenting this to us um the next thing that's important is we want to check our the network and here cool we've got 10 connections that means we're connected to 10 other peers um and this number if you want to be an active participant uh or I guess you could say an extension of the network this needs to be greater than eight otherwise all you're doing is you're just you're just uh you're just copying information to you you're not actually allowing anybody else to use you as a uh AER so you're you're only a one-way player so to speak um so you're not actually helping the the network proliferate at all um so anyway just one thing to say and you can set set things up so that um you just have eight connections and which means you'll only be receiving incoming data uh and that's fine that's actually for some people probably better from a security perspective it just depends on what your goals are but for me I I don't mind being a um you know a a contributor to the network decentralization and security so that's you know why for me I'm I'm okay with leaving it open um okay this means that everything's running correctly or running as it should um so at this point what we can do is I'm going to go ahead and stop so one thing to know is always try to use the Bitcoin CLI if you manually stop it with like controll C um sometimes it will be hard to start again so we're going to go and stop that and then if you notice it should interrupt uh this process here and eventually it'll shut down gracefully give it just a minute there we go all right so now what we're going to do is now we've made sure everything's working and all the setup is correct um we'll go ahead and we'll start it back but this time we're going to start it back as a Damon and all that means is it's just going to run in the background so we don't have to keep this terminal alive in order to keep the process running so we'll go ahead and start that and then basically the process is going to start and it's just going to run um in the background so another thing we can always check again we'll give it just a minute but as soon as it starts running again we should be able to run these commands boom and [Music] boom see it only got seven connections this time because it's having to go back and and remake those connections with with peers that it broke when when I killed the process but anyway you guys get the idea so now everything's running um and you know you should be able to sit here and look and be able to basically refresh wow it's down a a lot already um but you should be able to sit here and hit the refresh button and notice that the numbers keep increasing um I forget what the last thing we had downloaded was let's check it out all right so block 0477 so let's see how quickly things are changing [Music] 0477 oh well I guess I'm not changing that that fast but anyway you guys get the idea if you sit here and you do that long enough eventually um you know they're pretty big files 128 megabytes a piece but uh eventually you'll see that change um so anyway guys uh hope that you find found this helpful um you know I didn't find a lot of good stuff on this on YouTube so I figured I would I would make a video myself um I hope you guys enjoyed it if you have any questions just leave in the comments and I'll do my best to address them uh and I'll see you guys in the next video all right guys brief reminder here if you enjoyed the content please consider giving me a like And subscribe so I can continue to grow and produce better and better content for you if you really enjoyed the content you might even consider buying me a coffee and the link for how to do that will be in the video description below um if nothing else please just give me some feedback and the comments and let me know how I'm doing uh if anything's unclear or if there anything uh that I can improve on thank you again guys and have a great rest of your day bye-bye