Do you know this common Go vulnerability?

Key Takeaways

in my job I'm doing security audits usually for web applications and sadly oftentimes I don't get the source code and need to test the web application blindly as a blackbox but you can still get a lot of hints from the backend implementation for example through error messages stack traces unique HTP headers or simply how the API endpoints behave and feel like for example when you have a go back end but you don't know that and it's using the popular G Json library then the incoming Json request can have an equ sign at the end this Library will happily parse it while the built in Json Library would throw an error sometimes I test for things like this in order to figure out if maybe the back end is using go this kind of knowledge I think is very helpful because every programming language and every framework has its own typical issues and pitfalls so when you know the backend you can optimize your testing you can look specifically for issues in that direction that is common in that world and of course this applies even more once you get access to the source code and you have full white box test and you can see the whole logic and you see exactly the libraries and Frameworks being used and at this point it's very very helpful when you can obviously read that code and you are aware of typical pitfalls and programmer mistakes to look out for so let's talk about such a pitfall in go code that I was not aware of this video is sponsored by hex.io our online training platform but more on that later let's get back to the video at the 38 C3 hxp so that's a CTF team primarily from the Technical University of Munich they organized the Congress CTF and our team Alice didn't really play but some of us joined forces with flux fingers to help them and one of the challenge that I looked at a little bit was the challenge fuch sorry I had to use AI to pronounce that for me this was a go web challenge and according to Jet GPT this roughly translates to key value store and that is exactly what it was so let's get a quick overview of the challenge and see what it is about in this challenge we get access to the source code and when you unpack the files you can find the following there are two go files KV and front end and Docker files so that you can quickly run it locally in the docker file we can also see that the flag.txt is copied into /home CF so our goal will be to leak the content of the flag from here the docker file also has a lot of weird additional code which you might think could have something to do with the challenge but somebody from flux fingers told me that this is likely just code to harden the challenge container clean it up a bit at the end you can then see that it executes the front end binary which was compiled up here so the front end is our entry point with the docker composed file we can also easily run this code with Docker locally the challenge is running now on local port 8080 and when we then open this page in the browser we can see the message we booted a fresh web scale key value store just for you if we look into the front and go code we can see the message here so this is handling an incoming HTTP request and this message only appears if we have no cookie with a valid session set if we have a session then the request is actually passed into this HTTP proxy Handler so something else is then actually handling the request if we look around a bit more we can also see that the front end executes the KV binary and then it creates this HTTP proxy with a Unix domain socket called KV socket and if we look now into the KV program we can see that it listens on this KV socket and it also declares two HTP Handler functions set and get so it looks like HTTP requests incoming into the front end get forwarded here via a Unix domain socket to the KV process we can also see that if we try that out in the browser if we have a session we can call set with the name for example test and a value a short one will fail with a 502 that's because of a bug in the code but a long value will work and now that we have set this value we can now get the value with a call to /g get surprice we can set and get values it's a key value store now this feels weird right why use this front- end code that executes another go binary and then using Unix sockets to forward incoming requests to the backend KV process if you know how typical web applications are set up this is very odd and uncommon and suspicious and this could have two reasons maybe it's necessary for the CTF challenge in order to do user separation or this is part of the vulnerability maybe there's an issue with the exic or an issue with the Unix socket if we look closer at the code if we don't have a session we can see that the session ID for the cookie is returned by new KV and in new KV the KV binary is executed and a proxy is set up with a unique Unix domain socket connection file so this means for each new session the front end creates a new backend process and given then the session ID in the cookie it can get the correct proxy back end and Route the HTTP requests to the correct KV process by going through the Unix domain socket file okay this makes it a lot more likely that it's just a setup to do user or team separation for the CTF but you can never be sure and it really confused me a lot to the point where I was not focusing on the right path and I missed it completely because eventually somebody from Flex fingers noticed something very obvious VI there is an arbitrary file read you provide the name or the key that you want to read it's passed to OS open it opens that file and then it reads that content and returns it and if we try that that's how it works we can read Etc pass WD so can we now just read the flag.txt file unfortunately that does not work there's this check path function and it checks that the file path does not contain a DOT and does not contain the string flag so flag.txt fails both checks of course there were ideas if we can somehow get a siming to the flag or we could use the set function because obviously there's an arbitrary file right as well the name and value is just used to write file and very suspicious that threw us for a loop was these permissions 777 the file gets execution permissions so could we write a binary or script or overwrite something and then abuse the EXC from the front end or something else to get rce but none of those ideas really led to anything until someone had a somewhat obvious idea the proc file system it contains information about every running process including the file descriptors opened by other processes and those are Sim links to the actual files so what if another process has the flag opened then we could bypass the check paath function there won't be a DOT and no flag string in the file name but there doesn't seem to be other processes that open the file right right nothing in this code reads and opens the flag file unless other teams solve the challenge because if another team somehow opens and reads the flag with their process we can get the flag through the file the crypto Sim link to the flag.txt file this is of course not the intended solution this was clear to us as well and flex fingers even asked the organizers if it's okay to do that because it could go against the rule of not attacking the infrastructure but they said it's fine so we did that here's the code for it it also uses one more trick to optimize this you need to know the process IDs of other processes and you could of course guess them but you need to be fast when a new process is started otherwise it might quit again and the file descript is gone but you can use another useful proc file for that and that is located in proc sis kernel namespace last process ID this file gives you the last allocated process ID inside your name space and so you can use this with the arbitrary file Le to check if a new process got started and if yes you can then use that process ID to leak the file descriptors and maybe that one opened the flag so the script was running and now we just had to wait but it didn't take too long and we got the flag and most likely it was not even another team solving it because in fancy modern CTF setups you often have health checks for the CTF F challenge Services often times they are running the solution script against it this way that you can always confirm that the challenge is still working as intended so probably their flag Checker solved the challenge to confirm it's working and we leaked the flag from that but what was the intended way that's actually the whole point of this video right um in the end only a team solved it even though the challenge difficulty was estimated by the organizers to be easy well after the CTF I was very curious and waited in the Discord server for the CTF for the intenna solution and it turns out the vulnerability was right here do you see it obviously I didn't see it either it has to do with the differences of colon equal and equal colon equal means declaration and assignment of a value while equal is just an assignment and this is important go has built in concurrency features that makes go pretty unique and Powerful concurrency is everywhere and go and so you need to be careful with threat safe code and variables let's have a closer look at this code of course these HTTP request handlers can run concurrently multiple requests in parallel can be handled here so they can be multiple requests with get or set calls in parallel now look here do you see how this error variable is only assigned and not declared the return of check path is only a assigned to the error variable it doesn't create a new error variable but where is the error variable then coming from where is it declared vs code actually knows this if you rightclick the arror variable find all references then it will highlight all the uses of this variable and it shows all the outer ones so here the variable is declared and everywhere here it's just used the other variables that are declared and assigned are not highlighted they are a different variable you can also see that when you debug the go code we can set a breakpoint here and then investigate the variables dynamically first we see here the aror variable but let's single step forward until the next one this one is declared and now we can see that there are two error variables one without parentheses and one with if we look into the source code of the go debugger interface in vs code you can annotate shadowed variables in parentheses so this is a so-called shadowed variable variable shadowing occurs when a variable declared within a certain scope has the same name as a variable declared in an outer scope so if you debug the code you can also see that something funky is going on with the variables here and this is an issue remember this code can run in parallel so the error variable is accessed in parallel and if you look closely you can see that the check is done in two steps first the error variable is assigned and then it's checked so what if the check path is called with flag.txt it sets the error variable as expected but then in parallel another request sets the error back to nil then this check would pass allowing you to read flag.txt so that is what it was it's a go race condition and I agree with the organizers assessment that this is an easy challenge currency is fundamental to go if your experience with writing or reading go you probably should know this but I didn't and I'm very bad and inexperienced with go but thanks to the CTF challenge I now am aware of this and from now on out when I read go code I will pay attention to the Declaration and assignments especially of error variables now here is the final exploit code it's super simple we first get a session ID to initialize our personal key value store and then we just Spam requests to get a file that passes the check path function and writes nil into the error variable while we also spam request to read the flag.txt file this will return an error but if right afterwards the error is then reset to nil it will leak the flag successfully now we can just run it and depending on the networking and various other factors eventually we will get the flag by the way this is also the answer to the question why this challenge has this unique and weird back and front end setup if all CTF teams would share the backend binary or backend process then everybody would accidentally rise condition each other by spawning a unique KV process for everybody based on the session ID the rise condition only happens within your unique session nobody influences each other if you want to learn more about hacking check out our online training platform hex free.io we are still in Early Access so there are still lots of content that we need to create but I am still very proud of what we created so far also Google sponsored our Android application security course they have hundreds of apps in their back Bounty program and they identified the lack of resources or good education as a major issue that's why they contacted us they hope that we can teach the next generation of Android application Buck Hunters so give these courses a try they were produced by St smash and me and we did a lot of additional research to create them this is not just knowledge that you can already find online we also cover several vulnerabilities that we have found ourselves in various applications and we created test applications where you can practice what you learned and hunt for Flags so come and check out our courses on hex.io [Music]