Playbook for Security Onion

Key Takeaways

are you looking for the best in-depth training for your cyber defense team look no further than Sans blue team courses whether you focus on network or host data Windows Linux or even specialize in open source intelligence seam sock or defensive architecture the Sans blue team curriculum has the course for you from longtime Classics like SEC 503 Network intrusion detection to the newer SEC 530 defensible security architecture and engineering and SEC 487 open source intelligence gathering no matter what your specialty we've got you covered with an extensive Archive of free webcasts on the sand site and free online demos available for most courses you can easily check out the Sans blue team catalog and see which course is the best fit for you and your team check out the constantly growing list of available courses at Sans url.com slbl teamops this is the blueprint podcast bringing you the latest in cyberdefense and security operations from top blue team leaders blueprint is brought to you by the Sans Institute and is hosted by S certified instructor John hubard and now here's your host John hubard today on the blueprint podcast we have Josh Brower from security onion Solutions if you've seen Josh speak at any conferences recently you know he's been working on some key contributions to the new version of security onion in the form of the new PlayBook application regardless of whether you use security onion or not I wanted to dive into the design principles and workflow he planned out and coded up for the new open source Playbook app because I honestly think it's amazing work and there's something to learn from it for everyone on The Blue Team a lot of socks struggle with use case organization analytic documentation unit testing metrics creation and more and I think what Josh has done here wraps up solutions to many of those problems into one tightly integrated solution if you've ever been an analyst staring at an idler and wondering what to do next or a sock manager trying to generate metrics on your detection capability in an easy way this episode is for you combining Sigma rules attack Navigator The Hive and other open source blue team favorites the new playbook software Josh created was designed to address common sock problems and much much more stick around to hear the details coming up on this episode of The Blueprint podcast all right welcome everyone to the blueprint podcast today I have Josh Brower I had Josh on the podcast today because I'm really interested in his involvement in creating the Playbook functionality for uh security onion as someone who's always trying to organize analytics and use cases and things like that Josh came up with a really Innovative solution here and I think he's probably come up with something that might be more efficient than anything I've I've ever seen before and I love the way it integrates and everything like that so I wanted to interview him and pull out some of that very very useful information on workflow and the deep thinking I'm sure he did to build this application so welcome to the podcast Josh thank you so much John great to be here so before we get started can you hear a little bit about your background and uh how you got to where you are now sure sure so I actually started in it when I was a teenager at probably a lot of you the same where you fixed your family's computers and things like that and so very early on got into it and spent the first part of my career in operations more from a CIS admin and a network side of things and then got my first SS C back in 2008 and from then on really focused on uh security at that point and spent most of my career in the international nonprofit Community doing information security uh among that particular Community then about a year and a half ago joined security onion Solutions specifically focused on engineering of the platform and training I've run security onion and production for many years in my previous organizations and so it was just a really natural fit and it's been a really good year and a half with security onion working on the platform itself fantastic uh what was it that brought you uh specifically to want to tackle the problem of playbooks yeah that's a that's a great question so well first of all let's define this idea of a Playbook right so a lot of people in the industry will use different ways to Define Playbook and playbooks and for me I really look at it from the perspective of a detection Playbook we have plays that make up a Playbook and plays are just essentially custom reports or detections that you write and they make up your detection Playbook I really got into this because a few years ago ago again at my previous organization I was working on a particular issue where we had we had a technology stack that required a highlevel service account domain service account and I was concerned about monitoring that and I went to try to write an elast alert rule uh specifically figure out a way to write a rule for that so we can monitor any time we see usage of that account outside of where it should be and I realized that even if I could write this Rule and get this alert to generate the next person who sees alert is going to have no idea what this means we just know that this you know this account was accessed on this machine and there's no context and there's no next steps okay so I've got this now what do I do that was really the beginning of me trying to figure out a way in my work of how can I write detections in a way that would give context to what I'm trying to do and give the analyst next steps because whether the analyst is me six months from now because I've slept and I don't remember what I did or you know someone else on the team so that was kind of the original idea is building up detection plays and Playbook that gives context and uh next steps is that making sense John yeah absolutely um that gets to I think one of the biggest problems in in a lot of security operations centers especially ones with a lot of kind of retention or turnover problems is the tribal knowledge thing right you get new people coming in and they like first day on the job this alert goes off what do I do right and so having a documented place for those kind of things I think is a fantastic way to eliminate that and and standardize you know what is expected and understanding what a rule is where the data comes from what false positives might be created and all that kind of stuff and that's kind of how I look at playbooks too so that's why I was really excited when I saw like everything you had in here I'm like oh this is perfect that's super cool walk me through in terms of the new application for security onion like what is the workflow for someone who is maybe just thought of a brand new idea and is going to turn that into a rule as opposed to maybe an analyst that is going to be referencing it what what how does this interaction happen sure sure so Playbook is an application the back end is just a red M and open source like issue tracking and project management application and the idea is that the a person who has this new idea let's say really really basic good one here is I want to get alerted anytime there is a RDP connection from an external IP address to one of my internal endpoints right so that means that I have maybe an RDP listener from about desktop protocol listener uh publicly accessible and so I got this great idea where do I go well first I got to figure out where my logs are right and so I know let's say I have Zeke logs that are available for me to alert off of in security onion so I know I got the logs next up is you're going to open up Playbook and you're going to write a sigma rule for it Sigma is a language to describe detections think of Sigma just like we have syot and snort alerts for network data we have Yara for files we have Sigma for log files and so you're going to write a sigma rule that says anytime I see a zek connection record or excuse me a Zeke RDP record from an external IP address to an internal IP address I'm going to go ahead and generate an alert you write that Sigma Rule and then you can test it uh within the within the PlayBook application make sure it generates the elastic search query that you need and then you actually click on create play and that's going to create what we call a detection play inside Playbook and this play gives you some metadata about what's going on you can put you can tag the play with miter attack information and things like that but really the three components of the play which I already mentioned is number one context why am I doing this what am I looking for number two the elas search query that we need for the play and then third is the next steps so next steps may be is this a valid RDP listener you know do we have a terminal server on premise if we don't who should I be contacting next what should I be checking to make sure that this is valid or not once that is actually in Playbook then you can make that play active and that goes out and does some automation it creates in the last alert configuration so that we actually will now get alerts for that it also creates a case template for the hive The Hive is a case management application and so we'll talk about the analyst part of that next but it creates a case template next it also updates the application called the attack Navigator which gives you a visual indicator of your coverage across the miter attack framework that was kind of real quick overview but the idea is that you've got this idea you got to figure out where the log is you got to write the sigma rule uh for the log L and then create the plate from that and then put it into production does that making sense as we go along there yeah yeah definitely and that was one of the things I loved is like all of those automatic actions that happen with that paler has has made public like their ads kind of process and it's very similar right you come up with an idea you come up with like the core is it going to work you test it and all of that but what you did was take it a step further and say like we're going to push out a case template we're going to turn an actual Sigma rule into a rule that's active in the environment we're going to make it easy to see what you've done with the attac Navigator and so that I think is where it all of this integration becomes really really nice as a analyst looking at an alert that has gone off what is your kind of Ideal interaction with this environment and like how might that go if they're like they're open up the alert and they're like okay I've never seen this before I'm going to Playbook then what sure sure yeah so from the analyst side uh now that let's say I get an alert generated for this RDP traffic I pull open the alert and uh from right there I can pivot to Playbook there's a link you know I can pivot to Playbook and I can see context okay this play was written because we want to know when we have an RDP listener publicly accessible so okay I've got context and that's even without having to read the elastic search query itself if if you've ever had to I mean I think very normal is that when we're looking at Syra cotta alerts or something else because there's just no context typically to what's going on we have to spend quite a bit of time I'm not familiar with the title I've got to go research the rule I've got to go try to dig into it and figure out what it's doing why it's looking at it and so the point from an analyst perspective is that I should be able to quickly and efficiently pivot to the context understand what's going on why I got a an alert generated and then quickly and efficiently pivot to my next steps okay now I know I need to go check this system go check in with this person validate it and then escalate it to a case or you know suppress it or you know close out that alert so the idea is I can be able to quickly and efficiently get context on the alert and validate it from there very cool that's one of the things I'm repeating all the time in class is like you know in this triage stage right you have a whole mountain of stuff you need to know what's most important and when you pick one you know you need to see like very clearly this is something that is clear danger and and very high risk and providing that context and and what I didn't realize is it's linked directly from the alert so even better right it makes it very very easy to just say oh I don't know what this is one click you're there love that too that's really really important especially maybe the newer you are in infosec it really helps out in that respect as well as the you know the tribal knowledge problem let me just say John in relation to that when you click over to the play in Playbook you also get the history of what's happened with the play if we've made changes let's say we tweaked it because we've seen some false positives we also get commentary from other analysts on there and so like you said if you're new you can very quickly look at the history of what's going on and figure out you know is this something that's outside of Norm or is it something that's legitimately something I need to take care of so yeah all that tribal knowledge I think is more easily self-contained in something like a Playbook yeah absolutely I totally forgot about that angle too yeah that's a really important point is you know every analytic goes through many many changes and many people find issues with it and having a single place to say like like this Alert misfired in these situations have that tag to a time and a person that made that and then maybe when you make an edit have that timestamped and put in there as well uh that's a workflow I think works really well with a ticketing solution and you know just by chance I happen to use a ticketing Solutions in my past as well for use cases and found the same kind of benefits which is why I was really stoked to see you know that was uh what you were going with the other things I wanted to ask about the sigma piece Sigma is one of those things that I think is is catching on for for sure I'm starting to hear more and more about it see more and more support for it what has been your experience with the the sigma part and where do you think do you think and do you see that becoming a standard within the industry I do I do I I really I really like Sigma um so originally I was when I was working on playbook from an application perspective I started growing or using a homegrown solution for describing a play but I'm a really big believer if there's something out there that works and works well and has got pretty broad support and let's just go ahead and use that you know why create yet another quote standard um when there's something already out there and so that's one of the main reasons why I went with Sigma and um certainly there are some edge cases here and there but from what I've seen and used in production Sigma works really well again if if you're not familiar with Sigma you write your Sigma rule in yaml um and then you use the sigma converter which is a python script and you feed that converter your field mapping for your backend system as well as um what you want your backend system to end up being in our case in security onion it's elastic search but it could be Splunk or something else and then the sigma converter will then spit out the query you need for that particular Rule and so the issues I've seen are mainly around not having field mappings uh set correctly or you know things like that which inside again the platform that I'm working with security onion we do all that for for you you don't have to worry about the sigma converter or anything that's all done on the back end uh for Playbook so one of the things I noticed when you create new rules with a uh a sigma Rule and you make them active within Playbook is that it pushes out a new case template into the hive and for those that aren't familiar with case templates in the hive this is one of those things I guess this may be a little bit of a collision of terms here but a lot of people would call it a Playbook right but in terms of the hive it's a case template is a series of tasks that are expected to be done uh you know in a given situation and so you can take this play as we call it in Playbook and then tie it directly to a set of steps within the incident management system which I think is another really awesome thing how did you make that part work side not here I actually originally looked at creating a playbook in Playbook type thing inside the hive and I just couldn't get like you said the case templates wasn't quite there with what I was looking for and so that's why I ended up going with an external application but I really do like the functionality of having your case inside the hive and having those tasks that you go down and so the way that we do that in security onion and in Playbook is that you add an extra section to your Sigma rule called tasks and then when we convert that over and we make that play active there's some backend automation that will just take those tasks and make them uh part of the case template inside the hive and again the idea here is that when I escalate a an alert from the interface in security onion it will create a case inside the hive and there will already be pre-loaded next steps so that the analyst or whoever's working this can see okay I need to go you know check this out another let me give you another example of what this may look like if I have shared drives in my environment osquery at open source endpoint agent allows us to really get some interesting information about endpoints in the environment and one of those is shared drives like Windows shared drives and in environments that I've worked in it's really common for you know temporary quote temporary shared um drives to be made and then forgotten about you know backups or I'm just going to share this information but specifically backups is what I was thinking of and so one of the early on plays that I created was anytime there is a new share created in the environment but it has the term like HR or backups or temp or something as that share name or it's included in that go ahead and generate an alert based on that and then the case template next steps would be uh go and confirm you know was this a recently created share what are the share permissions they should be locked down you know go find out who is the owner of that of that system or whether or not this is a legitimate share can be removed so there's some very specific next steps that we could take in a case like that and that's where I think the power of adding tasks to your play is really important we'll be back after a quick break if you're enjoying this episode then you're undoubtedly interested in building the strongest security operations team that you can for those who want to go even deeper did you know that Sans has not one but two courses that cover security operations centers as well for the leaders managers and directors out there my co-author Mark Orlando and I offer 551 building and leading security operations centers this course covers building your team your physical and virtual workspace getting the right data into your tools and then focusing on security priorities through everyday execution of important security tasks and building the best sock team possible for the technical practitioners out there my course SEC 450 blue team fundamentals security operations and Analysis is designed to cover everything you need to jump in being the best sock analyst that you can be we cover important data types sock tools security logs malware analysis technique Automation and much much more in addition if you want to prove you can deliver the best on any security team both courses have an accompanying certification available from GAC that's the Gom for 551 and the gck for 450 check out both courses and free demos available on the Sans website you can get registered today for an in-person course at one of our many events or go to on demand and take either class anywhere at your own pace thanks for listening yeah definitely um one of the other things I was curious about is from a manager perspective you know having the guidance for analysts and all of that sort of thing is fantastic but what kind of of organization and like metrics and reports can Playbook produce for those who are maybe managing the sock and trying to understand the field of things that are available for detection and the state that they're in and things like that sure sure yeah great question so specifically right now there's integration with the um with the miter attack Navigator application which is put out by Navigator or excuse me miter and allows you to uh as I mentioned every time you make a Play Active uh so a detection is active if that detection is tagged with a particular technique or something like that then we'll actually get colorcoded cells on that on the application so from that perspective a manager should be able to quickly get a view of what kind of coverage they have um from a detection perspective you can also rightclick on one of those cells on one of those techniques and say view related plays and you'll be able to bring it'll bring up Playbook and it'll show you all of the plays that are tagged with a particular technique or tactic outside of that within the application itself there is ability to generate some basic uh metrics and those would be based on rule sets so let's say within the sigma Community repository there are over 500 Sigma rules that you can import and that we import automatically and those are tagged with like is this a Windows is this cismon related things like that and so we can do some metrics around that as well is there any um capability right now or any plan to do any capability where you can tie the actual amount of times the alert fired and back to the Playbook uh interface itself I'm sure you can probably produce that directly within the other interfaces but is there like a you know this play has fired this many times yeah that was that was something early on that I was looking at and it's certainly doable um everything is accessible via apis all that data is accessible via API and so the plumbing is all there we just got to tie it all together it is something we're looking at but it's not something that is currently implemented but yeah I would love I would love to be able to pull a report and say this play fired 50 times over the last three months and out of those 47 were you know false positive three led to incidents you know and something or other I think that's definitely a possibility yeah keeping my manager hat on for a second here now my mind's kind of running wild and like thinking well if you could track how many times the thing fired and you and think about you know like when it was acknowledged versus how long it took to go from whatever state to to closure you could get times and all that sort of stuff and then all of this would become one really really nice like metrics generating package for management so some really Co possibilities yeah one of the other things I was wondering there are a ton of obviously rule sets of All Sorts that are out there whether it's surcot rules or you know anything based on any rule set right there's hundreds of rules and analytics that any environment may have at any given time how do you decide what of those rules become plays in Playbook versus the ones that maybe you just kind of leave as an IDs signature and you don't independently document that's one of the things I always struggle with trying to figure out how to do sure when you think about in particular uh Network intrusion detection system signatures there's they really run the gamut some of them are really looking more for Atomic indicators right like they're looking for particular IP addresses and things like that I think those you could possibly disable inside of the nids engine and do a play Just for Atomic indicators in your environment but there are others that are really specific and maybe looking for C2 communication or some particular malware um that communicates via the network and it really makes more sense just to keep it as a let's say A you know seot or snort rule so I don't think any of those are ever are going away anytime soon I think it's going to be up to the environment and the the team that's looking at it to decide whether or not this should only live in uh something like a a nids rule or it should be pulled into a playbook play I definitely think with tuning you can make your nids rules much more manageable but I was actually looking at one this morning for uh it was curl curl outbound there were a number of different applications that were setting off one of them was from Cisco moroi it was it was doing a port ADH GTP um curl outbound and I mean I don't care about that that's not great in my environment but I don't care about it so I thought about maybe could we create a play that filters out some of that kind of noise but brings and some of the other more interesting kind of curl events um with a little bit more Nuance than I can do with the other tools right now inside uh nids rules I'm not really sure yet but my point is that I don't think there's a one- siiz fits-all when it comes to that I think there's always going to be other alerting engines and the alerts that they produce that we're going to have to figure out how that integrates into something like a a detection Playbook is that making sense yep um yeah I think uh one thing you said kind of reminded me of maybe an approach or you know kind of maybe how I've unconsciously done this myself in the past uh you said you know specific indicators right thinking of the Pyramid of pain right yeah uh it seems like a lot of the sigma rules are oriented more towards those ttps and the things that are miter attack you know Navigator layer style you know sub techniques or parent level techniques and things like that so maybe that's a good way to kind of divide that in between you know this is super specific and these are generic kind of processes and procedures uh you might expect is the new Navigator uh stuff supported in there as well the the sub techniques and all that kind of stuff y yeah specifically for Sigma um there was some recent changes so the sub techniques are all supported and in Playbook itself that's all supported as well okay fantastic were there any other unique challenges or things that you kind of had to think really hard about in terms of process or otherwise while you were developing this what what do you think other teams might run into and and work on this help them from an implementation perspective so specifically with Playbook and uh within security onion we import 500 plus Sigma rules from the community repository one of the things that I think can be a challenge is just like any other rule set is should I go ahead and just enable all 500 right and then how do I deal with the Deluge of alerts that come through and that's where I think when you're looking at doing something like Playbook keep in mind that uh there's really two ways to approach this there's the predefined rule sets there's lots of places that are now producing their own Sigma rules that you could easily import um and turn those on but then there's also more organizational specific Sigma rules or you know detection plays that you'll right those will probably be a little more High Fidelity because they're specific for your organization and so those I wouldn't necessarily have a problem with saying let's just get all those turned on and make them active and and go from there but be cautious about just enabling you know hundreds and hundreds of detection plays without really understanding what you're looking for and why keep in mind that uh Sigma rules really need specific log sources so in the sigma repository you know you have log sources you have uh cismon Windows event logs Powershell CIS log and a bunch of other types of log sources so unlike let's say a nids like sakata where you can just enable everything um I wouldn't recommend it but if you just enable all the rules it's just one log Source it's your network data versus the the log sources for Sigma are much more disperate and so you just have to be more cautious and aware of what you're enabling from that perspective that would be my my thought as you look at moving this into production using something like Playbook yeah I'm glad you touched on that that's actually another conversation I like having with classes is uh you know obviously in the false positive in tuning section of the courses when I'm teaching one of the things I always bring up is you know what's the right or best approach and I'm not sure that there's a solid answer to this but I I think I know how you might fall on this question but I wanted to ask it anyway when you get any sensor whether it's you know uh the list of default Sigma plays or anything else is your approach that in General you start from like turn everything off and turn on the things only that you know you want or do you kind of leave the default set on and prefer to like work down from the false positives that are generated and just turn those things off honestly it really depends on the environment and what I'm looking for um I've been in uh small College environments and I've been in more organizational executive office environments and I think depending on the environment I would probably do it a little bit differently but I would typically start with just enabling the things that I know I want rather than you know turn on all the policy violations I don't want to have to go through all the work of disabling all the rules that alert on um you know instant messaging and things like that in a college environment because it's not a problem in that kind of environment right and so that would be my answer it's going to depend on the environment and you know what you're looking at but I would typically start with nothing and enable what I'm looking for at that point yeah that's usually kind of my Approach as well is I I find at least in my EXP experience when you leave the default rule set on analysts will see an alert for the first time maybe it's never gone off before and they're like well do we care about this or is this just the first time it's gone off and now right this moment we're realizing it's irrelevant and a false positive Y and so I think that's kind of a clever approach to kind of getting around that and cleaning things up right from the start right yep y uh in terms of trying out Playbook what is the easiest way that people can kind of see the application in action is there a way to do it kind of outside security onion do they just drop into the iso and put it in eval mode like what's the the shortest route to see in what you've done here sure sure so Playbook is an application I was working on previously to security onion but and you could certainly still run Playbook uh with a little bit of work outside of security onion but it's really best used inside security onion because that's what it was designed the last year plus of uh integration with security onion and so best thing to do is uh go to our um specifically there's free training out there called security onion Essentials on YouTube and that gives step by step on downloading the iso and installing eval mode um and you can get up and running within an hour and check out Playbook from there so that would be that would be the best way to uh to go at it fantastic final question any plans for new features going into the future where's the road map do you have anything exciting coming up sure so there are there's quite a few things um some of it's already implemented just hasn't been widely uh pushed yet one of them is play unit testing so that when I create a play you also will put a copy of the log that you're trying to alert off of as part of that creation process and then anytime the sigma changes it's going to run and verify that you still get alerts based off of that log and if you don't you'll get errors associated with that because again environments change over time right so or I might make a mistake when I'm editing the you know 6 months from now and so I want to make sure that I've got a valid detection for the log that I was originally working with if the log changes that's fine you can update that so unit testing is one of them uh for detections uh secondly is this one is coming in the next little bit as well when we're bringing in the 500 plus uh Sigma rules from the sigma repository some of those need to be tweaked for your environment and we also keep those updated so if you're trying to modify a rule for your environment and then you know we'll just overwrite those unfortunately every night when we do our updates and so we have some logic in play that we're working with from a contributor that will allow you to we'll either merge the changes or make it so that we won't overwrite that that signal rule to make it a little bit easier to customize it for your environment that's one thing that I don't think I really mentioned much is that with the community repository rules you need to make sure that you customize these as needed like the remote desktop protocol um detection I mentioned earlier if you've got a terminal server publicly accessible please don't never do that but if you do you know you're going to need to specify in your Sigma rule you know this is legit to see RDP traffic to this IP address anything other than that though you know should be a problem so those are two main features there's certainly some other automation things that I won't get into now but you know we're really on the ground floor of Playbook and uh I'm looking forward to continue developing it uh in the near future very cool love the unit testing thing I didn't even think about that one yet and asking so yeah glad you brought that up that's a really important thing right it works when it started versus it works three years later it's two very questions exactly exactly Y where can we uh follow projects you're doing security onion where's the best place to uh keep up to date with this online so my Twitter account at defensive depth also have a Blog defensive dep.com those are really the two best places to keep up with a lot of the work that I'm involved in specifically with Playbook osquery or security onion in general well I think that wraps it up for today thank you Josh for being on the podcast you teams of the world certainly appreciate what you've uh produced here really really nice wellth thought out integration of all these tools metadata and the case templates and Tac Navigator and metrics and unit testing really excited to see where this goes in the future so thanks for read on the podcast thank you much really enjoyed it hey blue teamers I hope you enjoyed today's episode of blueprint if you've got a second and want to help support the podcast Please Subscribe and leave us a review on Apple podcast it would be really really meaningful to us and if you have any ideas or suggestions I would love to hear them your reviews are going to be one of the best ways to help others find this podcast so anything you could do would be a big help as always thank you for listening you can connect to me on social at seub Hubb on Twitter or on LinkedIn so until next time thank you for listening to the blueprint podcast [Music]