Attacking Serverless Servers: Reverse Engineering the AWS, Azure, and GCP Function Runtime

Key Takeaways

hey everyone thanks so much for making it tonight I know that the world's kind of crazy right now so I'm glad that we can find the silver lining of coming together to talk about security and talk about a really I believe fascinating topic about scurrilous security and how to attack server list functions so I hope you will find that this presentation is rewarding for you as well so before we get into the presentation I wanted to briefly plug the course that I teach for sans which is SEK 540 cloud security and DevOps automation much of the material in this presentation comes directly from the course if your security professional and are not well versed in the ways of DevOps you are prescribing solutions for problems and teams that don't exist anymore SEC 540 gives you the context and the practical skills to play in that space the lab architecture is deployed on premises in AWS and the Nasher giving you real-world examples of how to utilize these skills across multiple environments while learning core cloud concepts for the more advanced students there are a plethora of challenging tasks that you can tackle in net Wars which runs throughout the week implement security controls in the cloud using the code that we provide and you can copy and paste that code into your organization's infrastructure and in the process win this awesome challenge coin also something that's brand new is we have a certification for the class called gia cloud security automation or GCS a if you'd like to learn more about more about it go to sans org slash SEK 540 or feel free to reach out to me either on linkedin or on twitter and my twitter handle is at brandon max Evans so a few details on me my name is Brandon heaviness I come from a developer background originally I've worked on a variety of teams that follow DevOps practices most recently I worked on a product development team at assure Ian that was primarily based in AWS I worked on a lot of cool things for sure Ian who allows me to also do stuff for Sam's so I'm very thankful to assure Ian for allowing me to have these multiple opportunities but at some point I determined that I like breaking things much more than I like building them so I moved over to the security world so as I mentioned I'm also a sans instructor my day job is assure Ian most people who teach for sans don't do it full-time and I'm no exception but I am a science instructor that has taken a variety of XI Act certifications a couple of weeks ago I took the beta exam for the GCSE a or G at cloud security automation which is once again the certification for 540 unfortunately because it's a beta exam I won't know my score until next month but considering the fact that I teach the class hopefully I did an alright job let's hope that I passed the certification so before we dive deep into the details of serverless I want to make sure that everyone is on the same page as to what serverless even is there have been several meanings of this term historically it's been used to refer to managed platform-as-a-service backends that you can leverage in your application without maintaining the underlying infrastructure typically developers would use these to create robust single page applications or mobile applications while writing little or no backend source code here are some of the services that we would call server les one that I'd like to highlight is firebase if you aren't aware of what firebase is I strongly recommend that you read up on it it is a very popular tool that allows you to connect to a central database directly from your front-end application if that doesn't scare you it should I find that firebase is oftentimes used for proof-of-concept applications but when those applications go to production teams struggle to untangle from it there's another topic we can talk about on another day but it's a server list technology that you should be aware of other services that are considered server lists are off zero and and AWS Cognito these allow you to authenticate your users without running your own authentication server so more or less the trend is that we're allowing developers to build applications while maintaining fewer and fewer servers this brings us to the current meaning of server less at this point developers have gone from running their code on-premises to the cloud they've integrated with various platforms much of which we don't manage the infrastructure for one example of this is I don't ever have to think about what type of ec2 instance my dynamodb database is stored on however with this infrastructure I still have to care about where my code is running so what if we changed that what if I could just write code have it run in the cloud and never worry about what server or container it's running on that's where the new definition of server list comes in using technologies like AWS lambda and that Microsoft Azure functions or a Google cloud functions I can define events that trigger code to be run and I can ensure that this code is run without maintaining any virtual machines or containers I can make a function that executes when an API is called I can have a service function that is called when a file is uploaded to an s3 bucket do some kind of post-processing on let's say an image file I can do all of this and much more this allows me to focus on what I is a developer actually care about which is writing code that solves a problem so if you are a security person you might have heard of serverless technologies and your first impulse might be to think that this technology is too immature and that developers are reckless for adopting it so I want to make my allegiance known right out of the gate I love serverless and you should too I love it as a developer who wants to focus on writing code to solve a problem I also love it as a person who never wants to wake up in the middle of the night to reboot a server and believe it or not I like it from a security perspective every security person should be happy about the fact that serverless shrinks our portion of the shared responsibility model server this means that we don't have to patch our underlying virtual machines every Tuesday anymore it also means we can restrict permissions much more granularly than we have been able to do so in the past and at least theoretically no long-running servers means that there are also no long-running command and control channels overall I think as service matures it will become the recommended practice from security teams however with any new technology there are bound to be security concerns and misconceptions in this technical demonstration I will show that in fact it's possible for you to gain shell access to a server less server so you might be wondering serverless server isn't that an oxymoron first of all who are you calling an oxymoron second this is a very reasonable thing to think serverless is a misleading term it does not mean that servers don't exist it means that we don't care where our code is running we just want to make sure that it runs on any arbitrary server we don't manage the servers our selves so despite the fact that we don't manage the servers I'm going to show you that it's possible to gain shell access on these underlying servers and execute arbitrary shell commands nonetheless now don't worry too much I'm not about to reveal that there is a secret SSH port open on these servers that's not what this talk is about so to understand this attack we first have to review two types of vulnerabilities the first is command injection injection is possible anytime that an application takes data from a user and sends it to a subsystem unless you're careful a hacker could trick the subsystem into executing the user provided data as code and there are many examples of injection attacks perhaps the most famous one is sequel injection but there's also cross-site scripting cross-site scripting is another very poorly named term it's really an HTML injection attack that may allow you to detonate JavaScript arbitrarily but it's also a form of injection the attack that we're discussing today is a command injection where an application has a flaw that takes user input and interprets it as actual shell code luckily shell command injection is becoming less and less common I don't see these kinds of vulnerabilities and functions because developers rarely need to spin up a bash program to get their work load done in this day and age however there is another much more likely way to get shell on a server list server and that is a supply chain attack I've heard a statistic recently that states that in nodejs in their ecosystem 97% of the code you deploy comes from open source libraries and frameworks so that that begs the question what are the odds that one of these open source libraries gets compromised unfortunately the answer is 100% because this has already happened with a popular NPM package called es Lent insolent is a very simple program it just looks for issues with your code style issues or syntax issues with with your code but the hacker changed the functionality of eslint to download arbitrary code from pastebin.com and execute it and the one piece of code that we're aware of that the person executed was code that stole npm GS org tokens which allowed that person to compromise other repositories which resulted in a huge outage because NPM J org had to revoke all these tokens so this shows that any component you install is a potential vulnerability and if that component secretly starts executing commands on your server lots of bad things can happen for this reason we should perform what is called component analysis this is where we use tools to automatically scan our dependencies for known vulnerabilities this is an important tool in our toolbox but far from a perfect solution these tools will not save us from a zero-day vulnerability now one of the critical components of compromising a system is persistence you might be thinking based on the description that I gave before that serverless is ephemeral and for that reason after each function invocation you will get a brand new container in which to execute which means that you would not have long-term persistence unfortunately this is only partially true in reality you can only get a new container when your function goes through what is called a cold start cold starts occur the first time you've run a function in a while or when you have enough traffic to result in concurrent executions although the details around cold starts are mostly unknown because the providers of these services don't advertise all of their internals many folks have done research on this subject it turns out that the cold start adds a tremendous amount of latency and that warm environments quote unquote warm environments run much faster developers know this all too well and in order to avoid taking a performance hit they will oftentimes purposefully keep their functions warm in fact there is a plug-in for the serverless framework a very popular framework for deploying service functions called serverless plug-in warm up that will invoke your function every five minutes by default in order to keep your function and warm you might be thinking that this defeats the purpose of the service model and you know what I probably agree with you however at this time the pricing model for lambda is so much better than ec2 in most cases that some teams would rather use lambda incorrectly to save money this would be all well and good if it weren't for the security implications if a developer or hacker keeps the runtime alive it is even easier to press it to persist your attack so now that you understand the attack vector we are gonna shift the conversation away from these details and on to the risk associated with these attacks in the remainder of this presentation we are going to simulate the process of compromising a surrealist server with a new tool called puma security serverless prey this is a suite of functions and tools that we can use to establish reverse TCP shell connections to all three of the major serverless function providers this was created in partnership with another sans instructor Eric Johnson who is also the lead author of SEK 540 and a co-author of SEC 510 which by the way I forgot to mention earlier is a course that I'm working on that covers a lot of this material so this project is sponsored by Eric's company Puma security so we have named each function after a big cat of prey so we have one for lambda which is our Panther function we have one for Azure functions which is the cougar function and one for the Google cloud functions which is cheetah so you can type this URL or scan this QR code in order to get access to the malicious functions and are associated research you can take a picture of this now or you can wait till the end we're going to show it off again at the very end so let me give you an overview of what surrealist spray is and what it does so the first thing you have to do is you have to upload the function itself to our cloud provider of choice so that can be AWS lamda it can be as your functions or Google cloud functions so I've already done that process in the background I'm going to be demonstrating the remainder of the steps next the attacker in this case I will be the attacker and the victim will open a netcat server on my personal machine so for those who haven't used netcat before it's a simple way of sending data back and forth between two machines via the command line in this setup we are going to have our functions connect the netcat server on the attackers machine in this case my laptop so my laptop doesn't allow inbound connections from the internet into any port that would be a good thing to prevent from happening from a security perspective so we are going to use a service called an grok which exposes the netcat listener to the public internet on a random port that n grok server one of the and rach servers selects next we will invoke the function with the connection details from n grok this pool result in the function taking those details connecting to the netcat server via n croc and then listen for commands to run we're going to type our commands into the CLI which will be supplied over the internet v the netcat server that the function phoned home to so TLDR serverless prey is basically SSH daemon for cloud functions this is obviously a simplification but it gives you an idea of what we are going for we want these functions to be accessible to run commands from our computer so that's what I'm going to hopefully be able to demonstrate to you in the remainder of this presentation so without further ado it's demo time so I have seven different tabs here I'm going to be exploiting AWS I'm going to be exploiting a sure I'm gonna be exploiting Google and I'm gonna show you what damage could be done once we are into the function so for starters I am going to run this handy script in order to automatically execute the steps two through five that I showed in that slide earlier so here goes nothing so that prompt implies that I am now in the AWS lambda container let's type ID to see what user I ended up being okay so at this point I know that we're in the actual server so you're going to notice this shortly but approximately every 30 seconds we will get a message saying reconnect it and there's an important reason for this functions do not run for that long and if they're backed by API Gateway you have a maximum time out of 30 seconds so every 30 seconds the connection is going to be terminated and our command is smart enough to automatically reconnect to the server so from here let's take a look at where we landed so PWD or print working directory so we're in the VAR task directory I wonder what's in there let's type LS so we can see two different files in this directory so first let's see what's in the handler dot J's file it turns out this is the actual source code for our lambda so this code is the Panther code that I advertised earlier so this is you know not particularly exciting but imagine if this was a real function there's a lot of implications to what you get from this you get intellectual property if you're hacking a particular corporation you get potentially secrets if there are secrets hard-coded in the application code and you can also learn about how the application works in order to execute other attacks so that's not great let's take a look at that other file in fig KS or JSON so we actually have a configuration file here that contains credentials for a database so you might be thinking okay I understand that this is going to persist for a decent amount of time but eventually I'm gonna find the hacker and kick them out of our network out of this function and or I could just change the credentials were patched the system and let the credentials automatically rotate because this lambdas using an IM role in those permissions are temporary but what's not temporary this password right here is never going to change unless you manually rotate the database password which let's be honest doesn't happen very often so even if you get kicked out of the function you can get credentials that live much longer so let's try a different type of attack next I'm gonna try to store some kind of shell script on this container to persist some kind of malware and I'm gonna make this really simple I'm going to make it a script that just says the word hacks whenever you execute it so I'm gonna let it restart real quick and then I'm going to type out the message so don't wanna be interrupts in the middle of typing okay here we go echo echo hat home malware dot Sh let's try writing to a file called home malware dot SH with these contents unfortunately with AWS lambda I am thwarted from this attack and the reason why that is is that the vast majority of this container is read-only however there is one big exception and that's the temp directory they give you that directory to work in in order to for perhaps process a file like that example I gave you earlier with s3 so let's try writing to the temp directory instead and you won't see it right now but it's working so if I make this an executable and I type here it's going to actually run that script so that's particularly worrisome so I wonder if this file is going to persist after that restart so let's LS temp it turns out that file is existing for multiple invocations of this function and this is all about that cold start warm start thing it's going to reuse the container until the function is inactive and then retire the warm environment and then the next time you execute it you'll get a new cold start but I'm keeping this function alive every single time that I'm restarting my connection so this malware is going to persist for quite some time ok that's a couple things we can do to this container let's take a look at the environment variables here so there's quite a bit that's exposed right here in the environment variables you get a lot of data on the project you get a lot of data on the ecosystem but what's most interesting are these tokens I'm gonna zero in on those tokens so we have a couple different credentials that are being used by the container in order to authenticate itself to other services so the lambda needs to access various other AWS services these three credentials are the keys necessary in order to actually access those resources so at this point I can actually exit out of the function I don't need to be in the function anymore from here I'm going to demonstrate that we can take these credentials and we can use them to access things that we otherwise would have access to so from here I'm gonna transport to this terminal right here so the first thing that I want to demonstrate is that I don't have anything up my sleeves I am in an empty directory right now an LS in the Dallas rectory there's nothing here but I'm going to show you that we can download things from s3 that we shouldn't have access to into this directory so first I'm going to start out with empty credentials so now I have no credentials associated with this session and I'm going to try to download the contents of this bucket that I set up Panther - and then some unique identifiers I'm gonna try that out and I'm going to get a message that says that there's an error I've tried to download this using a request that had a malformed authorization header it wasn't able to build the authorization header out of these credentials so what if instead I took the credentials that I got in the previous step and I set them in this terminal session so let's take a look so I just copied and pasted all of those credentials and I put an export up front now let me try to get the same contents and it worked now I can look at the secret file located at assets slash Panther now in this case it's just the Panther however imagine the implication here imagine what would happen if this file was actually sensitive what if it was a ww2 form what if it was a credit card number you could get a lot of things by accessing s3 buckets that aren't supposed to be public and if your lambda has access to those private buckets an attacker who gets to compromise the lambda will be in the same position so now that we're authenticated we can do some other things we can also get some credentials from the parameters store or AWS SSM so here I'm going to try to get the database user from the SSM service and it turns out that i'm able to do so and i have the value right here but it's encrypted because this is a secure string so the same thing is true for this password parameter that I can get here I'm not able to look at the contents of the secret unless I'm authorized to decrypt the secret using kms so I'm going to try to decrypt the password with using this flag with decryption so if I have permissions to decrypt the secret I will be able to see it in plain text and there we go this is the actual password and if you don't believe me I'm gonna do this with the username and you should see Panther underscore username or underscore user or is it it just slash user my apologies whoops with decryption slash user there we go Panther II user is the unencrypted string so we can not only get secrets but also decrypt them so that's it for AWS let's move on to Google so we're going to use the cheetah function this time the cold start is pretty painful isn't it so from here let's see what user we're running as it turns out that in the Google Cloud functions platform by default you are actually running as root and this is very problematic if you know things about container security basically if there was a container breakout vulnerability and the contents of this function could access the host system it would be able to do so with the root user they would basically get host root which is very problematic and there was a container breakout vulnerability just in February of 2019 so these things do happen so not a great thing to do I hope Google changes this to be a non privileged user in any case let's see what directory we're in in this case we are in slash server or / sr b / files let's take a look at what we have here we have a couple different files let's focus first on cheetah go this is the go function associated with cheetah so if I didn't mention this earlier nodejs is used for Panther on AWS coleg is used on cheetah for google so here is the contents of our function again it's not particularly problematic because this code is open source but imagine if this was your company's code you would really not like a hacker to get access to it again we can get secret files as well so cheetah Gamal has various credentials that are used for the application as well which is not great so the next thing we're going to try to do is we're going to try to persist malware so I'm going to first try to persist malware in this directory so once again not all of this container is writable only a portion of it is actually writable unfortunately for Google a lot more is writable than a single directory and we can see that by trying to write to the home directory and then we're going to try to make it executable and then we're going to try to execute it once again we were able to execute an arbitrary shell script on this container and in this case we were able to persist it into a directory and as far as I can tell we can persist set to any directory other than the source directory and that makes sense you should never have a function that rewrites itself that could lead to some instability but that is highly problematic nonetheless so let's type env to look at some environment variables it turns out there are a lot of details that are provided in the environment variables details about how long we have until the function times out in this case we have 60 seconds as opposed to API gateways 30 the amount of memory or using a lot of details about my project a lot of things that we really don't want to expose but fortunately unlike with AWS they are not exposing your system your service account credentials in this environment variable you have to do a little bit more work to get the service account credentials so to do this we have to talk to the Google metadata service which you can find here and this is basically a folder system so we can go from / - slash compute metadata and we can go further and further into the metadata service so we can go look at details on the instance shortly so compute metadata b1o it's very important that you have a slash at the end of your directories it's very temperamental so we from here can get the instance data whoops and that's a one tricky thing about whatever you have a restart oh it actually worked that time that's awesome it reconvened from where I left off I'm kind of surprised that worked I have to look into why that is so we can go deeper and deeper and eventually get to this directory the Scopes directory so this basically says all of the different services that we can interact with by default the function gets the execution role of editor and it's important to understand what editor allows you to do basically everything for the services that are in scope for your user it has all of the permissions of the viewer role plus it can modify resources this is the second most powerful role the only more powerful primitive role owner gives you the same permissions that an editor gasps plus it can manage permissions and set up billing Google's official documentation says that you can change the service account to limit or extend these permissions but if you decide that your Google Cloud function needs the ability to manage your accounts billing I'm not really sure where to say to you in any case we have the editor role in all of these scopes in putting cloud platform so basically we can do anything we want in this account using the service account credentials so now let's actually get those service account credentials from the metadata service oh one thing that you can't do is use JQ because JQ is not installed here so we're not gonna be able to see it pretty printed but here is the token so here's the token here's what it expires so in this many seconds so it might expire in the middle of this demo but we'll see what damage we can do in the meantime so I'm gonna once again exit from here and I'm going to take these credentials and bring them with me to my next session where I'm going to try to do things with Google on my host machine so here is a URL that I can query to get access to Google's cloud storage I'm gonna try to do that first and by default it's going to say this is an anonymous caller you're using an anonymous caller and you can't access this project which is reasonable I made this bucket a private bucket so in order to change this I'm going to export a Google token environment variable and I'm going to paste the contents that I retrieved in the previous session and then I'm going to try to query this bucket with thee or this account with these credentials and now I can list all of the buckets in this account it turns out that there is a bucket for the deployment and there's also this evil bucket that I have for other reasons and leave somewhere here is the assets bucket so I think this is the deployment bucket and this might actually be the assets bucket so this is some metadata about the account it's not really getting us the information that we actually care about which is the secret files but let's zoom into the bucket and get information on what files exist so here we can see we have a cheetah JPEG file that we can download from this bucket so I'm going to take my authenticated call and I'm going to specifically download that object and let's hope it works I'm going to LS we can see that there's a cheetah file here let's try to open it and there's our cheetah again pretend that this is an actually sensitive file moving forward we can also interact with the Google cloud secret manager which is a beta service but I mean basically everything Google provides on their cloud platform is considered in beta and I understand that's because they want to try to make things better and better but sometimes it means that they want to send out functionality that is not fully baked and I think the secret manager at least when it comes to automating the secret manager I've kind of felt that way about the secret manager but let's try to list all the secrets in this account so we can see that there's a database password in here so let's look at the metadata surrounding that password so we're gonna get the latest version of the password and this is just metadata if we want to actually look at the contents of the secret we have to do this access command and you don't necessarily have permission to access a secret if you can list the metadata so let's prove that out okay so here we have our data but it is actually base64 encoded data now if you know anything about encryption hopefully you will immediately say that's not encryption that's encoding anyone in the world can reverse this string and you're absolutely right so let me demonstrate that and now I have the plaintext password the same one that I got from Panther so that shows that we can get secrets and images from the Google cloud platform now let's move on to Azure so ash is pretty interesting in that you have two different environments for a sure you have a sure Linux and you also have as your windows or in some cases you have nothing because yesterday when I was practicing this demonstration there was actually a service unavailable error message that I was getting so you know buyer beware when it comes to these less mature serverless applications or honestly just buyer beware when it comes to service in general this is still a very much new technology and it's going to get better and better with time so let's go into the Linux container first for cougar let's type ID and this is really interesting I have a root user for a sure and that wasn't the case last month I don't really know what happened Eric and I are scratching our heads right now the only change that we made that we think could potentially be relevant is we changed the app service plan SKU from SKU from dynamic to basic and if you unknowingly lose security when you change your billing that's really bad so I don't know if this was an intentional change or if it was a matter of I changed my service plan but this was not a root user two months ago when I last presented this so let's see what directory we're in we're in the root directory so we can see a lot of different files in here let's focus in on this directory the WW root folder so this is a c-sharp function so it's actually compiled down into a binary so if I look at /bin slash star cougar star I can see the files necessary to run this application externally but I don't necessarily have the ability to read the source code and unless I use a decompiling tool additionally there's still a secrets file here if I go to the app settings file in this directory we can find some secrets associated with this application so that's also problematic we also can look at this asp.net folder which interestingly enough has some data protection keys I'm not someone who works that much in dotnet so I'm not positive with this file does but the file makes it very clear that I should not share the contents of it so I won't today at least there are some examples of the Mach data that you can get on our repository in our research section so now let's try to persist some malware in this container so when I first try to persist it to the www root folder and interestingly enough based on my notes at least this is going to silently fail so let me try to execute it or try to show the contents of it more likely and I don't see any contents and if I LS that directory now or just the directory yeah that files not there so it silently failed it didn't give me an error message but I can write to the home directory and execute it here I don't even have to make it an executable interestingly enough so I'm not really sure why but that's a pretty scary stuff that I can just persist malware just with a cat command and forwarding the output or an echo command and forwarding it to a file so for whatever reason it seems like as your functions live for a really really long time and I don't get that resetting that I get in the other two environs so let me demonstrate this file exists across sessions by control seeing and then creating a brand-new session with Cougar so home malware con Sh it's still there not good stuff so we can look at the environment variables here there's a lot of data a lot of keys a lot of seemingly Secrets so that's scary stuff but let's focus mostly on this msi secret because we can use that to look at the metadata service for sure so let's go to that metadata service and use it to generate a token for the storage service so one good thing about Asher is that when you get the credentials the credentials are for our particular service or a particular scope you can't generate credentials that work for every surface so here I'm gonna get them for storage for whatever reason I have to type echo almost every time that I run this command to get the results but here's the token for storage and now I'm going to run the same thing for the vault which is their secrets manager solution so now I have the credentials for both of those services so I'm going to copy that into my notes I'm going to leave this and go to my local machine once more so from here I'm going to demonstrate that if I make this request to the storage service with an empty bearer token I will get an error let's see looks like I got something different than I expected let's see okay that is oh this URLs malformed it has assets in two different places so whoops my notes have an issue my bad so if I remove that this will say the authentication failed because the authentication information is invalid so I have to change that authentication header okay so I'm going to take the token that I got from the storage service before and I'm going to export that value okay now I'm going to run the same thing and here we're going to get some metadata on these containers unfortunately they use XML they don't use JSON so it's a little bit harder to parse but I can use this tool called XML lint to parse the contents so you can see here we have a blob in this container called cooler JPEG okay so from here we can download the actual image itself so I'm going to do that here I'm going to download the Cougar file - cougar JPEG ok it's here let's hope this worked and success here's our cougar okay just a few more things and then we'll wrap up the demonstration I'm now going to try to get a secret from the vault soror list the secrets in this case but this is an invalid audience error because we have created a token the token that we exported was the one for storage not for vault and this demonstrates that you need to use the correct token for the correct operation so let me instead use the token that I got in the second command for the vault so I'm going to export bearer token paste this second token and try running it now and now we can see all these secrets we don't actually have the values but we can see the secrets themselves and now amou notice I'm going to specifically get the database password as shown here and here's the content of that password awesome or scary depending on your perspective the last thing I'm going to demonstrate is that we can actually get a command dot exe shell session to the Windows Azure functions so the azure functions that are hosted in a Windows environment we can run command XE which is interesting you will have functions that you're going to eventually hack if you're doing penetration testing that are using the Windows platform so it's important to know how to do this and serverless pray gives you the tools necessary to simulate these kinds of attacks so first I can type this to determine what my user is so I'm this workgroup user I'm going to type PWD to see where I am I'm in this directory right here I can look at the contents of this directory so here are all of the dll's for our function we can get the same types of details from this container as we did with the Linux container so this is the same thing as cat is on Linux so we're going to get the contents of this app settings JSON file and then once again has this database password we can also list the binary files for couger as shown before so here's the dll in the PDB and we can also look at all of these fancy asp.net data protection keys there it goes just took a little bit there it is there's the key we can persist malware in this case in a batch file if we so chose to so if I der slash home now we're got that there it is and we can use set to look at the environment variables so that's cool but let's say you are more interested in using PowerShell because you haven't used command exe in a while well you can actually pivot from here to use PowerShell vxe just by typing PowerShell dot exe and now we're in a windows powershell session so from here we could do some damage as a PowerShell user I'm not tremendously good at PowerShell so I don't have a lot of good demonstrations to show to you but that's the idea so we have compromised four functions one on AWS lambda one on Asscher functions or two on Azure functions one for Linux and one for Windows and then our Google Cloud functions so let's talk briefly about the remediation process so there's no way to completely stop this vulnerability we really can't prevent the attacks the best thing we can do is minimize the damage that can be performed through this kind of attack the number one thing you can do to do this is limit the permissions of your functions I am execution roll if an attacker gets into your function that's bad and they can do what the function can do but if that function has less permissions that means the attacker also has lesser permissions so in contrast if the function has administrative access that means compromising the function compromises your entire AWS account which is very bad and the same principle goes for Google and for Azure we can also do some protections at the network level we should not allow our functions to access resources over the network that it doesn't need to communicate and there's a lot of automation that we can use to audit our accounts for bad policies and bad configurations find components with known vulnerabilities like we talked about earlier and even block certain actions at runtime with runtime security solutions for server lists we should use all the tools at our disposal to limit the damage done here finally detection is a must prevention is ideal I know I got that phrase backwards but prevention is ideal detection is a must and we need to focus a lot of energy on monitoring these attacks are powerful but they're also pretty noisy if we can detect the attack we can take actions to contain the damage and remove the attacker from the system so you might be wondering how can I do that unfortunately this talk is mostly focused on the attack side of the spectrum but here's the good news the other author of this tool Erik Johnson and you know the author of several sprays I mentioned did a much longer talk on the defense of the functions at the RSA Conference so you can view that using this sans URL or using that you are code so if you want to learn more about the defense side definitely check that out additionally you should absolutely check out the course that I teach SEK 540 cloud security and DevOps automation as mentioned before a lot of this content comes directly from that course so if you want to go into more depth that course is a great asset to you additionally when the course I'm currently working on SEK 510 comes out you should check it out as well as it will cover similar topics to what we covered here it's going to go into a lot of depth about all these metadata services which is very important as metas data service attacks have becoming have been becoming very problematic very famous we all heard about the Capital One breach that happened last year and it's not necessarily capital one's fault specifically that vulnerability affected a lot of different companies so we need to understand those metadata services in order to protect ourselves so I actually plan on giving a course preview webcast in a few weeks from now so keep on the lookout for when we announced that webcast and hopefully you can see a little bit about what we're going to cover if you'd like to learn more about the differences between the big three cloud providers I actually already did a webcast for sands in January on that topic you can find that at this URL or this QR code and finally please check out serverless prey we put a lot of work into this project and we mostly built it for some presentations but we think it's very useful for service security awareness it can demonstrate the risk of these types of attacks which are only going to become more and more prevalent so issues and pull requests are always welcome so if you're interested in taking tech 540 but you can't go to a conference due to the horrible virus that's going around the world you can now take it as a cyber cast so here are some of the upcoming dates for the cyber cast that you could take a look at so what is a cyber cast it's the same quality training that you've come to expect from sans just online the same instructors who teach the class of conferences teach them on cyber cast I myself will be teaching a cyber cast of second 540 at the end of May this is an online live event and has the same lecture lab and network experience as you would have in-person if you have the training budget and you want to learn more about cloud security and DevOps I couldn't recommend 540 enough and with that let's look at some questions so we got a couple here so let's take a look at this give me one moment what are the prerequisites for the sec 540 course or the SEC 510 course I'm not positive which one you're referring to so I'll answer both so SEC 540 doesn't have any formal prerequisites but we have this new class I believe its SEC 488 which is an introduction to the cloud so if you haven't been introduced to cloud concepts in general that course might be better suited for you but you can take 540 afterward if you'd like typically speaking we oftentimes recommend that people check out SEC 545 if they're not used to infrastructure automation so 540 is very much focused on the automation aspect and it kind of skipped so it skips over on purpose the gooeys that you can utilize the graphical user interfaces that you could utilize in order to configure the cloud so if you're not really comfortable with the idea of infrastructure as code that might be a better a prerequisite so thanks for that question prerequisites for 510 are similar 510 we'd recommend taking 488 first to get an introduction of cloud security but after that you can jump right into any of the 500 level security courses someone mentions that they can't see my screen I really hope that other people could see my screen so I'm sorry if you couldn't is there a way to enumerate what resources the credentials have access to that's a really good question I'm not positive for AWS but we could see in the Google example that you could look at the different scopes that you have and I believe there's a couple extra bits of metadata that you can get from the Google meditativeness service and a sure you have to specifically get a token for a particular service so that's something that you have to consider as well but I'm not sure about AWS it's a great question so when attacking service servers are you attacking the AWS as your GCP side of be sorry I'm trying to read the entire question GCP side of the share responsibility and not the customer suck it absolutely not I am attacking the customers side of the equation this is not exploiting the underlying infrastructure or the let's put it this way I'm not attacking the lambdas service I'm attacking the container that the lambda service provisioned for me and you know it's kind of a gray area but all the ways that we can fix this attack come from the customers side right all these actions that I mentioned on this slide are things that the customer can do a lot of it comes down to your code not having the vulnerability the first place that allows me to execute commands on your server so I guess there's a little bit of a gray area but I would say it's more on the customer side so what is SEC 510 a great question it is a class on cloud security and analyzing cloud security so the title is still a working title but the general gist of it is we're gonna really dive deep into the internals of these cloud providers to understand how they actually work underneath the hood and not just assume that the documentation provided by the cloud providers is accurate we're actually gonna validate that some of the things that the cloud providers put in the documentation is not accurate and we're going to demonstrate how that lapse in understanding could lead to a security vulnerability the video if fully recorded hopefully it was fully recorded will be included at the end of this presentation shortly after they will archive both the slides and the video do you have any tips on the GCS a I just signed up for sacrifi 40 and I have not taken it yet so I think that certification is really really fair I think it's right down to the material of the class so if you are following the material in the class I think that you will succeed with the test the class is very heavy into labs which is great it gives you a lot of first-hand knowledge of how to actually use the cloud and that's kind of difficult for folks who have kind of pivoted into a more security or a more management kind of role but it's so rewarding and once you go through it you'll get a lot out of the process and I think once you've been able to practice those concepts you will understand them well enough to take the certification but there are many pieces of advice that I could give in general about diac certifications it really comes down to your notes and your indexed you need to have a lot of good notes to understand where all of your content is you can look up the right section at the right time so I've got another question about what about artifacts any eye sees out there IOC means a couple different things in a couple different contexts I am Oh indicators of compromise okay it's not an inversion of control I always mix those two up so indicators of compromise regarding artifacts like in terms of script persistence I would look at Eric's talk about that I'm not super knowledgeable on the monitoring side I'm doing more red team work in the context of this talk but he went into a lot of details on how to detect the compromises so there's a lot of details there that I'm not going to spoil and I don't remember offhand well enough to explain what ways to me monitor the AWS lamda I would highly recommend that talk it provides a variety of tools that you could utilize and ya know it's definitely an area that's not being covered enough and that's why Eric and I have been working on this subject it's a you know really uncharted territory I mean serverless has really been getting popular maybe a couple of years ago it's getting serious adoption now and like with any innovation in technology Security's oftentimes behind when it comes to creating tools that solve security problems with these new technologies but with regards to monitoring you can talk about monitoring your dependencies with dependency checking tools such as sneak or charge not checkbox checkmarks static analysis sneak or black duck or Gees I'm just blanking on a couple names it'll come to me I feel really bad because I have a friend who works for that company and the company does Nexus sorry that I'm blanking on the specifics Ross dependency check is a open source tool so that's definitely a great place to get started there are also a variety of tools that do run time security like pure sac or port a go or twist lock that actually prevents your functions from taking actions that it shouldn't be able to take thank you so much for mentioning Sona type I always mix up Sona type and sonar cube and I just wanted to make sure I got the name right so yes Sona type is the company that I'm referring to they also do dependency checking and they're also the owners of Nexus and maven central so thank you for that so there's a lot of free tools there's not a lot of free tools for runtime security that's kind of like the next generation you know rasp technology one-time application security protection and there's not a lot of f