How To Secure & Harden Docker Containers

Key Takeaways

[Music] you can register for part two of this series by clicking on the link in the description and after you provide your name and email you should be able to get access it's absolutely free to watch uh part two of this series on demand on the on24 platform what we cover in part two of this series is you know controlling container resource consumption with control groups we also talk about implementing access control for containers with app armor and then we talk about limiting container system calls and performing vulnerability scanning for docker containers and we finally end by taking you through the process of building secure docker images so if you want to register for part two of this series all you need to do is click on the link in the description and you should have access immediately hey guys hackersploit here back again with another video welcome back to the docker security series in this video we're going to be taking a look at how to secure and harden docker containers during runtime um so if you want to watch part two of this series you can check that out by registering in the link in the description uh the you know the second part is on demand and free it's hosted on linux on 24 platform and we've partnered with them to bring you this series so in regards to what we will be covering in this video the first thing we'll talk about is how to run containers with unprivileged users or to set up your containers to be run with unprivileged users so that you know you can limit access to privileged users as much as possible in regards to limiting access to the root user we'll talk about how to disable access to the root user directly so that even if a malicious attacker knows the password of the root user within the container they aren't able to actually switch to the root user and actually have a session we'll then move on to preventing privilege escalation and this will primarily cover the uh the the process of uh essentially ensuring uh that even if there are suid binaries in place that you cannot use them you know to elevate your privileges so that's also very important we'll also talk about limiting container capabilities uh through the kernel and kernel capability modules uh we'll then move on to file system permissions and access and we'll talk about making the container read only so that you know any malicious attacker or any malicious script will not be able to write changes to the container you know so on and so forth and then finally we'll talk about disabling inter-container communication so this video is really not going to have much theory because everything will be covered in in the practical area or the practical section and we will actually sort it into parts so that you can actually go through each part that interests you with that being said let's move on to the practical section okay so the first security technique that we're going to be taking a look at is going to be the process or the practice of always running or launching containers with an unprivileged user now the reason you need to do this or you you know it's actually recommended that you do this is to prevent privilege escalation attacks right so in order to to actually run or a particular container from a docker image as an unprivileged user you need to control the build process for that particular for that particular image so that's another security best practice that you should incorporate and that is to build your own images all right and this gives you the ability to set up the environment based on the parameters you're aware of so for example in my particular case i can build a simple you know a simple docker image using one of our our docker images that we created it's called the bug bounty toolkit so we'll just use the dockerfile here you can see it's based on ubuntu 18.04 and we don't set any particular unprivileged user here so we are going to be doing that so you know i'll just um we can just copy this here i'll just copy the first few commands because the rest of the commands essentially set up directories and install tools so what we'll do is we will begin the setup process so i'll just take you through the process of what you do when you're running a a standard container or a standard image so if i list out my docker images so say you know if i say docker images you can see i have an ubuntu 18.04 docker image now you would typically run it by saying you know docker run and then i you know say we want to use a terminal and i want it to be i want to get rid of this container when i exit and then we specify the image id right so we can specify the image id and then i can use a shell so i can say bin bash hit enter you can see that it logs us in as the root user now this may not this may actually seem like a trivial thing but it actually isn't recommended would you give root access to anyone or would you give root access to a person on a server the answer to that is no even if it's running services so you need to take the whole concept of host security into the containers so that's the first thing we need to do now when it comes down to creating these users i'll just exit from this particular container and if i say docker we can say docker psa uh you can see we we don't have any of these containers running which is excellent so as i said we'll be building our own we'll just create our own docker file so we'll say vim docker file and we'll just hit enter of course this is going to be done locally and we'll just copy this initial configuration here and we'll i'll just insert and we'll put that in there fantastic so in terms of the environment variables i can set my own environment variables right so i can set up for example we can say home and we can set up the name of the user that i want to create the unprivileged user that i want to create which is going to be alexis and remember if you do require root privileges you can actually just specify that under uh you can actually just specify that within your docker file so you know you don't need to actually have or you don't need to specify permissions for that user for your docker file to be built right so that's in terms of installation it's just when you're running the container which is the important bit here so to add the user all we need to do is say run so we're you know actually running a command here and the command we want to do is or that we want to run is going to be called group ad so we're going to be saying group add and then we specify the user which in this case is going to be alexis and again you can customize that to whatever you want so we can say alexis and then we're going to say user add and we're going to specify the group so uh user add r and g and then we specify the group in this particular case we're not granting alexis any any particular privileges or any pseudo privileges but we could have done that i can say you know for example sudo and then say alexis but in this case i'll just add it alexis to his own group so alexis alexis and he is a fully unprivileged user so that means he can't do anything that requires root uh that requires root privileges so now that we've done this you can see that immediately below this we can begin the configuration if if we actually want to do that but in our case we're not going to install any particular piece of software but you know just for instance or for the sake of this particular video i can say you know run we can then say you know apt install apt install nikto or just any commands over here but we're not going to do that so i'll save this docker file and to build this particular docker file into an image we're going to say docker build and we then specify this current working directory which is where the docker file is being stored and then we specify a tag and we'll just call this let's just call this particular image test and we hit enter and it's going to begin the build process and you can see it's complete and now if we list out our docker images you can see it's going to say um it's it's actually just going to say test that's the dock image that we created so now when it comes down to running this particular image um you can see that we can approach it two ways we can say docker run and i don't need to specify a user and i just i essentially just launch it and it'll actually use the root user however when it comes down to as i said running dockers securely you want to make sure that you use an unprivileged user so if you want to use a a user that you've created or in in your docker image all you need to do is say r and then we specify uh we all we need to do is say we specify the user so we can actually use you here and then we specify the username or the user id which can be obtained using the etsy password file so the user is going to be alexis and then we specify the other run options so for example docker run user lexis we can also just say we're running this and we want it to get deleted and then we specify the image id right over here and we'll put that in and then say you know bin bash for a um for a terminal session and there we are so we're now logged in as the user alexis and remember even if we have the the root user added and we haven't disabled it but we actually can do that i'll show you how to do that shortly we can't actually switch to the root user because we didn't set the root password so if we give this particular image to someone or you know if we give this particular container to someone or give them access to this container they can't escalate their privileges to root and they can't do anything that would potentially compromise the container or the services running within the container or the host itself if there are vulnerabilities within the host so you know the user can't install new software uh they can't muck around with the services running inside they can't compile anything so you and and they cannot execute anything that uh again may damage the system so we can then just hit exit here and we can close that out so again if i just list out my docker images i'm just going to get rid of this particular docker image that we created so i'll just say docker docker rmi so docker remove this image and we'll just put in the image id there and that's going to get rid of that particular image and then of course as i said we can modify the docker file to lock or block access to the root user completely so for example i can say immediately after this i can say you know run and i can you know say uh for example we can echo out a command or we can modify uh or you know prevent access uh by using the change shell uh command and then we can change the default shell for the root user and prevent uh you know we can actually prevent a user from logging into the root user account even if they have the particular password so again we can essentially just specify the uh the terminal for the root user as user s bin no login similar to what you do when you are securing uh when you're securing the root account or disabling the root account on linux so again all we need to do is just provide the the command so we can say change shell and we say the shell is going to be uh user uh user s bin no login right and then we specify the the user that we want to the user of whom or the user the terminal for the user that we actually want to switch and then um we just need to hit save and then we can build this particular image so we're just going to run the docker build command here there we go and it looks like that is done and if we list out the docker images now all we need to do is say we're just going to run the docker command again with the user alexis and we're just going to substitute the the docker id here so that we don't have to type that in all over again so we'll just copy that in there and put that in there and hit enter and now if we you know try and switch to the root user and even if we add the correct password and we know yet you know for example test it'll actually prevent us from logging in as i said we haven't set a root user password but i can actually demonstrate this to you so for example if we go back into the dockerfile we can actually say you know just before this or even after this we can actually just say um run we can then say password and then we know we we can then provide a password for the root user but again that's not recommended but this is a great way of blocking root access or providing a user who is trying to escalate their privileges with no terminal that they can log into all right so that is essentially how to set a you know an unauthenticated user now to block root access for your particular images and containers so again it's always recommended that even if you're using a third-party uh docker a docker image it's recommended that you get the docker file and then you know change or customize the configuration based on the parameters that you feel are necessary or you can secure it as required that's the great thing about docker so we're now going to move on to the next step all right so the next security protocol or the security feature or the security practice to bear in mind is you know the ability to run docker containers in privileged mode and of course you then have the ability to restrict that so it's always recommended that you never run a docker container in privileged mode and you might be saying well what exactly do you mean here so if i just list out my docker containers let me just check check if i have one i don't and of course we created the image previously so we're just going to be using that particular image that i created so what i can do is i can say docker run right and um i can then say you know these the typical commands i was i'm likely to use we want an interactive session and i want to get rid of this container when i exit and then of course i can say privileged right i can actually use that particular i can use that particular option and this will run this container in privileged mode you never want to do that unless it's explicitly required so when it comes down to a new type of vulnerability or vulnerability that many of you who are security centric and i've actually you know performed some penetration testing you'll know of a atypical or a a very very very very simple concept in regards to set use id binaries and the fact that you can escalate privileges or abuse the set use id binaries to run commands as a as another user so for example if within our particular container if we if we had a set use id binary that allowed us to to actually run a command or run a particular binary with root permissions we can actually leverage that right so as an attacker we can leverage that to run you know various other commands as root so if you want to disable this uh what you need to specify is the security options so we can say security um security opt which is which stands for options so again we say is equal to no new privileges so very very simple so no new privileges uh so that's privileges and we then specify the docker image um the doc image id right so i'll just copy this because i don't have my docker image id at hand and i think i actually should have shouldn't have cleared that so if we list out my docker images here and we again just go back to the i'll just paste in that particular command and we then specify the docker image id here and you know we just specify we're just going to use bash by default that is going to give me access and remember i can then specify to use the on uh the unprivileged user that i created so let me just do that as well as an added security feature and of course you can automate these uh the execution of these particular of these particular containers and so we'll say run and then of course we specify the user in this particular case is going to be alexis so we actually specified run but rather just a user and then we say alexis and hit enter and we're now launched in as the user lexus so now if i had a a set use id binary here i would not be able to actually escalate my privileges and you know as it is already i can't i can't run anything or do anything that requires root privileges because i did not assign root privileges to this particular user so that is in essence how to disable or how to work around you know privileged and unprivileged containers and how to configure them to run as either privileged or unprivileged all right so now let's take a look at a security practice or a feature with within docker that allows you to specify uh kernel level or kernel functionality uh kernel capability uh and you know in the previous section we talked about you know running containers in privileged mode when we say uh privileged mode what exactly does this mean well when we specify the privileged flag uh what this means is that this will essentially give that container all linux kernel capabilities now if you've never heard of what capabilities are in the context of a linux kernel let me just introduce you to that right now so i'll i've just opened up the capabilities man page which explains it really really well so again capabilities you can see for the purpose of performing permission checks traditional unix implementations distinguish two categories of processes now when we i just want to intercept there when we talk about modern operating systems and their permission and privilege architecture you're going to have two modes you're going to have privileged mode and unprivileged mode and again as it says here in regards to the processes privileged processes whose effective user ids is zero also referred to as super user or root and unprivileged processors whose effective user id is nonzero right so privilege processes bypass all kernel permission checks while unprivileged processes are subject to full permission checking based on the processes credentials so what this means is in the previous section when we specified the security option that essentially prevented us from upgrading or or actually yep upgrading our credentials you know in the event we were trying to abuse a set user id binary um what that means is if we set that up or we set up a um an unprivileged user and we assign the privileged flag to that to that container at runtime we essentially just use serping or getting rid of the of the fact that we have specified an unprivileged user because that container will still be able to bypass all the kernel permission checks and will be able to essentially run all the various capabilities that are listed right over here so in terms of the the various capabilities implemented on linux you have the you have them sorted out here and they all they all have a prefix uh you know they all prefixed with cap or capability and so for example the capability audit control enables or disables kernel auditing so you can see you can specify the capabilities that you want and that's the great thing with docker i can run a container in privileged mode which gives that container all in kernel capabilities or i can specify an unprivileged user and then specify what capabilities i want that container to be run with which is fantastic so we can do this really really simply by again if i'll just say docker run and before i do that let me just list out my images here so we say docker run and then i say i can then specify the capabilities that i want to use so you know i can say capabilities and that's done by using the cap option so capabilities i can then say add or drop so for example what is recommended is that you drop all so you say drop all and then you add the ones that you want so you can say docker run cap drop all so capabilities drop all and then you can say capabilities add and then we can copy the name of a capability here and you know we can just search for one that would make sense in this particular case because they're all very very powerful as you can see so for example let's see if i can find one that makes sense so we can use net admin which allows us to perform various network related operations like interface configuration administration of ip firewalls etc etc so that seems like a piece of functionality that you might want to assign to your container so we can say capabilities add and then we say we provide the capability name and then i can specify you know the other the com the other runtime options that i want and then we can specify the user as alexis and we then specify the image id here and we specify the shell finally and you know i can say bin bash hit enter and we now have uh firstly we have dropped all other kernel capabilities secondly we have specified the the kernel capability that we wanted to use uh and of course uh you know i i really am not going to be demonstrating what we can do with this particular kernel capability but you can use the kernel capabilities list here to specify the functionality that you need even though you want to run the container in an unprivileged mode or with an unprivileged user so that is the process of using or specifying capabilities based on your requirements or based on your container requirements and as i said there is never a reason to actually run a container in privileged mode because that's probably one of the biggest the biggest mistakes you can make when it comes down to running containers all right so i'm just going to exit for that con from that container and we can move on to the next step we're now going to take a look at how to you know prevent a container from writing any particular changes to the file system or how to set up a temporary file system so you know this is a it's a piece of functionality that's very very important when it comes down to restricting your containers or restricting a service account or uh you know for example the root account so what do i mean when i say you know restricting access to the file system well what we can do is we can actually specify and say that we want this container to run in read-only mode which means that that any user on that particular container any service cannot make any changes or cannot write any files to the file system now you might be saying well you know this doesn't make any sense because my container my image requires me to install particular files or i i you know my web server will be dynamic files will be changing what do you mean i can't make any changes well again this is very important in relation or in context with other with other you know file system commands like the temporary file system flag or the temporary file system option which allows you to to actually save something temporarily or to save a file temporarily and you can specify the directories that you want so for example if i wanted to run my um if i wanted to run the docker image that we created in this video and i can say you know docker run and i can then start specifying for example read only right so you can say read only and you know i'll just run it as as is so we'll say user is going to be alexis and i'll just show you what's going to happen here we'll say we also want to get rid of the container when we're done so the rm option or the rm flag really doesn't correspond to the file system to file system permissions in regards to writing uh to writing files or to making changes this is in in essence for managing containers and the fact that when i'm done with this container or when it's in an exited state i want that container to be deleted so that's just a personal preference because you know i'm demonstrating something here so then i can copy the image id here and we can then just use a bash right so we can log in so remember i'm currently i can actually go into my uh root uh well actually i haven't set up the environment variable that but i did i did actually create the the home directory so we can say home uh do we have that directory created um if i just say cd home and we have the we actually don't have a directory there right so if i say you know touch test for example you can see it's going to tell me cannot touch test or cannot create the file test this is a read-only file system so that means i cannot make changes irregardless of what user i am so you might be saying well that's weird what if you're the root user so let's try that out so i'm just going to exit from the container and instead of specifying the user this time i can just specify or i can just leave that out and log in as the root user and i'm logged in as root and i can say touch test and i hit enter and there we go so it says hey you can make any changes to this file system so this is a great way of as i said locking down access around particular users or particular services or you may just have a service that you know you want to lock down in regards to what files or what directories you can that particular service or that particular container will be able to make changes in so for example if i wanted to use a temporary file system i can say we can say we'll still use the root user i can also combine the read only flag with the uh the temporary file system so i can say t tmp fs so tmp fs and of course sorry we need to actually just say tmpfs and then we specify the directory under which we we can make the temporary file we can make the temporary changes right so we can specify the opt directory for example and hit enter and now if i print my working directory i'm currently in the root if i you know if i say something like and you know as the root user we actually have the permission to do this if i say something like um sorry touch test and i hit enter it's going to say hey you can't make these changes because this is a read only file system however if i go into the opt directory which is the temporary file system that we have specified and i say you know touch test and there we go so we can actually make changes within this directory so this is a great way of locking or you know setting up restricted access even though you may already have privileged users or you may not want to use a privileged user so you know there's tons of ways you can set up security with your docker containers at large so that is how to use or how to set up you know the file system type to use and how to set up uh you know your temporary file system all right so we're now going to take a look at a very very uh interesting aspect of of container security now if you remember in the introduction i you know i briefly outlined the differences in in terms of architecture uh you know from containers and virtual machines right and you know one of the questions you might have if you've used virtual machines before is with a virtual machine setup i know that i can isolate each virtual machine from one another or i can i can essentially separate and isolate a virtual machines based on on their functionality based on their purpose and the way i can do this is by using various and networking modes well your question might be can we actually do this with docker can we do this with docker containers can i isolate a docker container and the answer to that is yes now you know previously with docker you had the ability to you know outrightly disable or set inter container communication to false there was an actual flag for this and many of you have used docker for a long time actually knew about this but now it's all being handled through a network so uh what we can do now is what we're going to be taking a look at now is how to create our own bridge network and we can then specify the options that we want to modify one of these options is going to be the intercontainer communication all right so firstly what we want to do is let's take a look at the networks that we already have firstly let me just check if i have any containers running no we don't okay so to inspect networks or the networks that come by default is we can just say docker network and ls so that's going to list out all the networks and you can see they are sorted based on the type of network so you have your bridged mode which again allows uh all all containers that are running on that particular bridge mode to to communicate with each other now by default all containers will use the bridge mode and i can i can actually demonstrate this to you or demonstrate this for you um so for example if i want to inspect this particular network i can say docker network inspect and i just say bridge right and i hit enter and it's going to give us the subnet right over here which again is 172.17 so let me just keep this here let me just note this down so i can actually prove that to you so that's the subnet so that means we'll have an ip within this particular range now if we take a look at the other options for example enable ipv6 that's disabled that's fine and of course you have the the options here that i'm referring to so we have the if i can find it there we are so we have com docker network bridge enable icc icc is a an abbreviation for intercon intercontainer communication so by default it's set to true so we want to change that value to false now the way we can do this is by creating our own network and i'll show you how to do this in a second so what i'll do is i'll just create a quick little container just to show you that it is using the bridge mode so we'll say um docker container so we'll just say sorry docker run because we don't have one currently running and we'll just specify you know the test uh the test image and just say bin and uh we'll just say bash here okay so we're currently logged in as a root user if i just say you know apt app update and well we actually don't need ifconfig uh ad uh we use ipa s let's just use ipas uh we don't have ipaddr uh so we just we can just say i uh apt update and apt install let's see why and we'll install net tools so we'll just install that here and we should be able to get the ip then so so we'll just give that a few seconds here there we are so ifconfig like so you can see it's set to that particular subnet that we had just copied over so you know 17.7 172.17 so that means that if i create another container we will essentially be able to communicate with that particular container so for example if i just um you know i don't think actually demonstrate that but let's actually move on to the process of creating our own network so i can actually demonstrate this so what i'll do is we will just use tmux i think i have tmox here and we'll just say container one container one and we'll create a new session here and i'll just rename this to container two right so container two all right so within container uh within the um sorry within container one uh which i'm actually active on right now we'll just create a container similar to what we just did so we'll say um docker run uh it and we'll say you know we'll just leave it as is and we'll just run the test one so test and then we will say um pin bash right and uh we'll just hit enter so we have one container running we'll then open up the second container so i'll just move to my second section to my second session and say docker run same thing the exact same options here and we'll just say you know bin bash and obviously gonna have different ips so um what we can do is again just say sudo apt we'll just say apt update because we don't have sudo su apt update and apt install uh why we say net tools and i'll just copy this particular command and run it in the first container as well so we'll just hit enter go back into the first container run that particular command as well and remember all of these containers are running simultaneously so we'll just let this complete and now what we'll do is just wait for that to complete here and there we are so if i say ifconfig we have the ip here and let's check the container we can actually run an nmap scan but we're going to we're not going to be doing that so ifconfig and if i say you know if we want to ping this particular address we can again just copy that and if we go back to the first container and say you know ping uh ping that particular container address so it's apt install ping because we don't have that i can't believe that's not there already uh apt install net tools we've already installed net tools here do we need to do that again do we need to install net tools again we already have net tools so we should be able to ping um ping is not found so apt search ping how come we can find the ping package here ip utils ping all right so apt install iputils and we say ping iputoolsping like so and we can then say ping that particular ip here and you can see we can communicate with that particular with that particular container so that's uh in essence how the bridge how the bridge connection works now if we want to disable uh you know inter intercontainer communication all we need to do is let me just go back to my uh to my second container here so i'll just exit and we'll close this particular container in this particular session and i'll just leave this within tmox and we'll just say we'll just call it at term one okay so we'll exit from this particular container as well and uh we our job now is to create a new bridge network that prevents inter intercontainer communication so the way we do this is by saying uh docker network and then we say create of course and then we provide the the driver that we want to use which is going to be bridge so we'll say the driver is going to be of type bridge right and we then specify the options that we want to use in this particular case we need to copy the the additional option i think i'd copied it the option that essentially specifies whether or not we want to enable we want to enable the inter the intercontainer communication so i'll just copy that here and uh we should be able to paste it directly so there we are um so we'll just add the the quotation marks so that was the option here and we've set it to false okay which is very important and then we provide the network name so i'll just call it test net all right so we'll just call the network test net and we'll hit enter it's going to create that network so now we say docker network ls and we list out that particular test net you can see we can then say docker network inspect and then we say testnet right to just list out the information we have not specified the subnet information or the any of the ip information that's up to you but the option uh that we're looking for here is you know the uh docker network uh bridge uh enable icc and we want to set that to false okay so now if we want to run a particular docker image or a container with a particular network configuration we all we need to do is just say docker docker run and we just again specify the flags that you like or that you're comfortable with in regards to the configuration and then we say network and that's going to be testnet we'll run it with the test network and then we just specify the name of the image which in this case is just going to be called test and we'll just run this with bash here and if i now say you know apt uh install or app update let's just run that particular command so i have to update and apt install we'll just say if or i think it's net net tools and ipu tools iputools ipu tools ping i believe it was and we hit enter and let those packages install all right so we're just gonna accept that and we should be good there so if i now say ifconfig we you know we have a different uh different subnet but uh the main process or the main thing here is whether or not we have the ability to to communicate with another container you know we wanted to disable that if we want to isolate this particular container so i'll create a new terminal session here and we'll just say docker run and we'll run it on the same network or even we don't even need to run it on on the custom network that we created all we're trying to see is whether one is is whether one container uh can communicate with another and the one that we've isolated we want to make sure it cannot communicate with the other ones so we'll say docker run i t and i will just say you know bin bash and the first thing you'll notice of course we actually have to just perform this cumbersome installation every time so i'll just say app update apt install why net tools because we don't need the ping utility right now so net tools i have config so we've got the you can see it's on a totally different network but if we can also do it on the same but if i go back in here and we say ping for example paste in that here you can see it's not going to be able to get a response now that's on a different network if we put the if we put the second container on the sim on the same test net network then again it will actually be the same thing so we'll just exit from here and we'll then say uh you know docker we'll just say docker run and we then say you know it rm and we say we can then specify the network option which again is network and then we say test network so test net and then we say test and that's the image name and then bin bash hit enter app update let me just speed up this process app update act install y and then we say net tools i actually should have installed these packages within the image itself or within the docker file i should have specified these packages to be installed so that we save time but you know hey ho all right so there we go we'll just say ifconfig and now they're on the same they're on the same subnet so you can just copy that and we can go back to the first terminal here and say ping and provide that there and hit enter and you can see they can't communicate with each other i can't communicate the other uh the other container on the same network and they're all isolated so if you want to isolate your containers uh you can create your network configuration you can specify your ip configuration i'll have all of this documentation listed out within the documentation for this video so you can actually do that for yourself so that is how to enable uh you know or to disable intercontainer communication uh you know under the guise of isolation which is uh which can be very very important a huge thank you to all of our patreons your support is greatly appreciated and this is a formal thank you so thank you shamir douglas ryan carr sandor michael busby sits up doozy defean barry dustin on president michael hubbard your support is greatly appreciated and you keep us making even more high quality content for you guys so thank you [Music] you