A guide on how to operationalize the Databricks AI Security Framework (DASF)

Key Takeaways

morning hello everyone my name is Arun palati I am Senior staff security field engineer at datab bricks with me is Ben and Ben Johns is part of our AI security work group that we ran in 2024 over a period of four months assembly of lot of experts from different parts of AI and security and um with their cont contribution we updated datab a security framework 2.o and did a lot of work around mapping a security framework to a lot of Standards 9 to be specific and also we did uh impact analysis so we are here to present that work and how to use datab security framework 2.o with the accompanying compendium document as well as U other assets that we are about to release as part of databook SEC framework 2. so before we go further I would like uh Ben to introduce himself and then uh we'll um get started with the Deep dive into datab security framework 2.4 thanks forun yes my name is Ben johnes I'm a cyber security risk expert and had the great pleasure of participating in the contribution uh working group um for day of 2.0 so I will be taking you through through the compendium document later on in the presentation back to youon thank you thank you Ben dat a security framework is a holistic approach to a system security to accomplish that what we did was we took an approach of um looking at a system as 12 components starting from simple rata injust all the way to how you integrate that into your AI applications through inference request and inference responses with those 12 components in hand for the AI system components we identified 62 technical security risks within those 12 components and we also identified 64 mitigation controls for those risks in the datab security framework white paper we go at length how you can have uh different uh teams within your organization collaborate to understand this AI system components the risks involved and the responsibilities around how to mitigate those risks with datab security framework 2.o we updated the risks and the controls as part of the work group that Ben and I was involved and then we identified the necessary controls to um controls based on 9 plus standards U that we reviewed and mapped in that effort we released that as a compendium both as a Google worksheet as well as uh M Microsoft Excel document so that you all can actually have a copy of this effort uh which we are going to go uh in in a bit more detail when Ben uh walks us through the overall worksheet we also did this impact analysis and then in the white paper we documented how we approach AI uh security and the red teaming of uh your AI system overall like how you should be thinking about and what tools and uh what process you should be following to do red teing on your a system we also documented how we approach a datab BRI the insurance response to your systems how it is different from your traditional cyber security insurance response as I mentioned the compendium uh in includes the nine different standards both for the risks that uh the 62 risks that we identified as well as for the 64 controls so that if you are approaching your a system security from any of these standards perspective you have a detailed documentation of those standards as I mentioned we took the holistic approach to a system system and we broke that into 12 different components so that it's easier to understand and identify where the risks may appear in a given AI system and we documented for each one of those AI system component the specific risks as part of the white paper and the compendium so that you have a good understanding when we talk about a specific risk you know where that risk is in a in a bigger a system component so that uh you will be able to assign and identify that specific risk on a given a system component and who would be responsible for handling that specific risk here is the comprehensive list of of those 62 risks in those 12 components and we make this available with um detailed description and various types of uh mappings from various standards so that you have comprehensive list as well as the details of each one of those specific risks once you understand those risks and where they may appear your next step would be how do I mitigate these risks so we took the time to study those standards and also pouring in our own uh experience in the field at datab bricks we identified 64 controls and we documented these controls in detail both in the white paper and the compendium so that that you know what the risk control and the description of that control so that if you're are using any generic platform not necess just datab braks you know what the control should be and then if you happen to be on datab bres platform we provide you the capability that would uh help you accomplish that specific control in the white paper as well as in the compendium we leave you with documentation links to all those 64 controls and how to imp them on datab bri platform so with that I I request pen to walk us through the process of how to use datab Bri security framework with the white paper and the compendium that we are going to leave you with so that you can actually practice a security uh in your Enterprise Ben thank you yeah so so what I'm going to work walk you through today is how you would operationalize this framework within your organization so the first section is as as run sort of explained to us we need to sit down and understand our use cases our stakeholders and how you're deploying your models so the the white paper and the compendium document will help you sort of sort of get that understanding understand your deployment model which is primarily How We Do risk applicability within this framework the second second section is you need to sit down and review those AI security components and the risks and understand them so this is your your chance to review and learn and then understand what mitigation controls might be available for those risks um so the way that the compendium document and the white paper is written it's it's written so you don't necessarily have to be a data bricks customer but if you are there's a lot of helpful information on how you would um understand those controls and Implement them if you needed to and the next section is you would need to understand how you would evaluate those um those risks controls and impacts so the the compendium document will help you operationalize that by um doing some basic filtering on deployment models which will give you a refined set of risks controls and impact statements that you could use for your risk assessment and your implementation of controls so whether you're a databas customer or not a datab BRI customer it will give you some very helpful information on how to implement controls to reduce risk within your AI systems so this this is an an attempt again to help you operationalize in a workflow of how you would refine those six two risks down to the applicable risks and applicable controls for your AI models so in step one you'd be understanding your business use case for your AI systems and you do a bit of documentation around you know what are your data sets who your stakeholders what are your compliance requirements and how are you deploying these models into your applications whether they're business fa customer facing or just internal business applications step two this comes back to your deployment models So based on the white paper and your understanding of deployment models within your organization we have these six high level deployment models that have been documented in the white paper and in the compendium document you can use these to filter and come to those refined set of or subset of deive risks so that's yeah coming into this sort of this second column what is your risk applicability based on your use case and your deployment models that's where You' come to understanding of your refined set of risks and into the the third column once you understand those risks you need to then understand what are your actual controls that you need to implement for each of those risks to be able to manage them and Monon properly so that's the the the basis of the workflow that we're introducing as part of desf 2.0 to help you try to oper operationalize um this framework into your um into your systems so on that point I will share the compendium document on screen now okay so the first couple of sheets um we've got like an introduction into the compendium document um and again this is available uh as a Google sheet as well as a Microsoft Excel document and they're exactly the same so the the content and the way the document works is the same on both versions um the second tab is just a table of contents what we put into this um into this um compendium document the the third U worksheet is a full list of the AI risks that have come from the white paper so we've just basically done a like a data dumping and started formatting them into some nice um some column structures so it starts with risk ID and then description and then the uh the mitigation controls that are applicable to those risks and then the description of sort of more of a detailed description of each of those controls and then coming on to the deployment model so each risk has been given assignment to a deployment model depending depending on how we've defined them and how we've um understood them to be applicable or not applicable for every scenario so that that that information is across all of the all of the risks that we have in the white paper for this framework um and then the next piece of information we've got is applicable risk impact so this is something new to version 2.0 that we' put together um So based on every risk in the framework we have some very specific impact statements so initial impact statements and then secondary business impact statements this is like a risk taxonomy or impact taxonomy um that you could use as part of your risk assessment or impact assessment um for your AI systems so it is a um a starting point for you to understand you what the what the potential impacts might be for every every given risk within this framework the other main section to this version 2.0 is how we've maap each of our risks to external standards and Frameworks so we've got nine standards that we've mapped to um each one that we've gone done done multiple rounds of reviews with external um contributors to make sure that we are mapping them to the right framework control um for each risk so where we found um helpful information we put in hyperlinks um to these external uh websites where you can go and look at F for further information and and for your own reading um and education if you need to um but we've done that for all of the um the nine Frameworks that we've chosen to be the most appropriate for this AI security framework so that's for your reading and um and education um if you wish to to look at that so on the the second that's so the next tab the mitigation controls we've also done this this mapping I'll just quickly show you yeah so we've done like a reverse mapping so we've got mappings for both um the risk um State the the the risks as well as the mitigation controls all of those nine Frameworks I'll scroll the way back over so this spreadsheet essentially is basically all of those um dat of controls that we've documented um all the way up to what is it 60 64 64 there you go so all of them so that's got the the sort of all of the detail that's been um put into the white paper we we've extracted that out um into this sort of tabular um format um so you can you can export it out and do whatever you want to do with it if you need to um again if you're a data mix customer you've got links here to how these controls are being implemented or how you can Implement them if you need to and then some other um sort of more helpful information I no R do you want to quickly talk about security analysis tool yeah definitely so if you are an Arab customer you can also validate if you actually uh deployed um these controls in your databas workspaces so you can use security analysis tool to automatically check um if those controls are enabled are implemented within your datab workspace yeah yeah so that we've tried to pull as much helpful information out of the white paper into these into this compendium document so yeah so it's it it sometimes it can be a bit more easier to use um than reading through the white paper and the other thing that we we introduced is this ability to refine down what your applicable risks and applicable controls might be so instead of looking at every single control and every single risk you should just maybe look at the ones that are more appropriate to your um AI system um so that's the the next two what worksheet documents so we've got risk applicability and control applicability so we'll take you through risk applicability first and again this is available on the Excel version as well and it works in exactly the same way so what you'll notice here is there's a bit of informational detail around your AI system so there's an area where you can quickly just note down what your AI use case might be what data sets that you might using to train or or or fine tune on an existing model um some of the stakehold information which is quite important you if they're external stakeholders or internal stakeholders any compliance requirements and that again might be external compliance as well as internal um and then a section where you can you can note down what applications might be using this AI system it might be a public facing customer application or it might be just something internal um and then yep go for now we are bringing all that information from the AI life cycle risks um sheet and AI medication controls um we are bringing all that together now we are making this actionable from this sheet so in this case your AI use case may be something like that or um it could be a medical use case diagnosis patient diagnos it could be anything whatever your AI objective might be um and then then the data set may be personal health data set yeah and then um stakeholders can be the lines of businesses that are um either providing the data set or you know on the receiving and of this AI system where they are integrating and building apps on this and um the legal team that is approving so this would be uh your um orall team and the users of the system you would be annotating them here as part of stakeholders yeah and it could be um external users that might be affected by a decisioning model or something like that um that is it's up to the organization to predefine that understand what that is and this is an area where you could document it if you don't already document it somewhere else yeah that's a big one yeah and the applications are um the applications that are being built on top of this a system that's correct yeah let's show them how this looks in action yeah so at the moment this is uh this list of risks is unfiltered so that's every single risk that's in the framework is currently all the way down um as soon as as you start using this AI deployment model drop down list that's when the risks will be filtered on applicability so whatever we've marked as yes in the main worksheet for the model deployment types will be filtered using that drop down menu so if you're if you've worked out that you're just using Rag llms and then you select that one it goes through and works out what the applicability is and now you've got a refined set of risks it's not every single one it's just the ones that we've selected yes and then if you if you've worked out that you use multiple deployment types you can select multiple from the drop down field drop down menu and again it will just work out which where you where it sees yes on based on those both two deployment model types okay um and it's it's an interactive menu as well so you can go and deselect it and um the the script in the background the filtering will go through and update it as well and you'll get a little little message here saying what it's doing so we've tested this extensively um it is working very accurately and again the Excel spreadsheet Works in exactly the same way so you a drop down menu you can use multi select as well yeah if you scroll down now the subset of 62 risks are now filtered to that specific deployment model so you have a good starting point to look at if I'm using uh in my AA system this type of deployment model U here are all the risks that I should be uh thinking about and here is how now I'm going to look at the initial risk impacts and business impacts and and now you are going to reason these with your business stakeholders uh and the applications that are going to be build on uh with with this AI system that you're building and then you will be looking at the applications that are going to be integrating into so you have a good starting point with this worksheet to start uh implementing the control so let's let's see how the applicable controls looks like from here yeah yeah no problem and before we go on that it's like it's a good point you talk about it it's a good starting point to start the conversation with business um to show them that you know there has been some thought around what the deployment model is what your deployment what data sets you're using who your stakeholders are what are the compliance requirements what are the potential impacts if you do nothing what are the business impacts if you do nothing and then what are the controls you can Implement to to stop this negative stuff from happening um yeah so onto control so what what we found is when you filter on risk you still have multiple controls that you need to implement to reduce that risk and our controls they run across multiple risks so we needed to work out what is the actual set of risks that you need to go and look at so one controll at a time so that's when we we built this control applicability based on the same filtering of the deployment model type so where you don't need to select this deployment model type again you only need to select it once in the main sheet and it comes across into the second sheet so yeah so you won't see that drop- down menu to select a different set different deployment model type if you wanted to change your deployment model type you go back to the first one and change it there so we'll add in a new one let it do fil and the control applicability will be updated based on that yeah then let's fill the in the days of risk applicability let's fill up the use case name and show them how it looks final sheet so let's just make up some air use case as you know um we go here uh customer credit history something like that um customers um something like that let we just go here it's [Music] um application whatever yeah if you enter all of the information into this first sheet it'll come across into the controls applicability sheet you don't need to retype that in um so if you want to do any editing you do editing on the risk applicability using the dropdown menus or manually typing into these free text field and it will come across automatically into the controls applicability sheet did you want to add anything else to that room before we go through and look at the um the all the extra information on the controls appity applicability worksheet this is great at this point there have a comprehensive list of controls that are aggregated or like summarized for the risk set that is applicable to that deployment model we are at a listing of the controls that they should be looking at so please work them through the controls and how this sheet would be helpful uh for them to go go to the next step yeah okay so so at this point we've got a refined list of controls so we've we we pulling the information from the main two sheets um where you can you can see the control reference ID the description um and also a bit more information of where that control sits within the components of each of the um of of the AI system that's been described um in the the main white paper document the next lot of of columns is just again there's more information that we've chosen to bring across out of the the main sheets which the the main home for that is the white paper um so we've got the responsibility model um if you're a data Bri customer you know if it's a control that you need to configure yourself or it's a control that's already done within the platform and and and maintained by datab bricks if you're not a customer of um of datab bricks um obviously these controls might sit in with your um um machine learning platform of your choice um but again the controls have been written in a way where it's very it is very simple English um if you need to build a control on your own it's very good guidance of how to how you build a control or how you would be able to see um if your your own platform um has a similar control uh again the the security analysis tool and those if you're using the security analysis tool on datab bricks you can see if your control is implemented or not and then again there's more helpful information are for. R customers and links to controls where you can have guidance for implementing them for you know your each of your Cloud platforms that you're using dat on the next thing is the refined list of m control mappings so all these external Frameworks that we that we're choosing to map to so that's the the nine Frameworks that we've um that we did as part of D of 2.0 so you have moer moer Atlas two versions of there moer attack OS um llm top 10 this is the latest in 2025 um and again where we found um helpful web links we put that information into into this so that's all prefiltered as well and the new um ISO 421 AI management system um controls like this so all the annexa controls that we found applicable that that sort of had some correlation or intent based on the death of control and also I at 27,000 A1 this it's actually a good thing to bring up here is the Dasa framework is not just an AI security framework it does look at traditional risks and controls you know around encryption you know authentication access control um which some of the the AI the newer AI Frameworks they they just really look at the novel risks and controls not the traditional risks and controls so it was good to put these two together side by side to show show how the breadth of the daive framework is not just AI security it's it's also a lot of you know the traditional stuff that you still need to you still need to be acrossing to have your AI system secure and stable n this is a big framework that we that we map to as well as well as high trust um um AR do you want to talk about the the mapping you did with high trust Yeah we actually worked very closely with high trust with which is a standards body for uh Health Information Systems within United States they actually released their AI system uh security uh standards and we mapped to high trust here and high trust also actually mapped their uh recommendations to datab Bricks a security framework so it's nice to have this bidirectional between uh our standard as well as hyra so um so that like you know if you're coming from one of is you would be in a good place when it comes to uh building your health information systems within us and then be able to accomplish the necessary controls that high trust recommends yeah yeah so that that is the the compendium do document very high level um quick walk through um yeah just so of to sort of recap on of what we tried to do with the compendium document again is is to give you a another more a detailed version of you know trying to understand your model your deployment model types and then a way you where you can you can refer back to to to understand the components and the risks that are applicable um to your deployment model types and then also another place where you can learn about the the risks and the mitigation controls and most importantly is the way you can evaluate and help you operationalize you know what can what risks controls and impacts are applicable and then hopefully help on a way for you to help you to deploy those controls and prioritize them as well as as well thank you very much for that Ben thank you all for joining us hopefully this was helpful for you to put datab security framework to in practice and have a practical approach to implementing a security controls for your AI system components thank you thank you