Best Practices to Mitigate AI Security Risks

Key Takeaways

So the goal of this task this talk is going to be bringing you all together and literally on a picture and hopefully give you enough so that you all can work with each other right at okay we are at time uh so let's uh jump straight in we have a to cover in the next uh 40 minutes. Uh so we I just want to start off uh by recognizing that I don't need to convince you that we are in the middle of an AI revolution, right? Uh we've seen various uh articles uh from various analysts say that almost like 90 plus percent maybe 93% of enterprises are increasing their um investments in AI. Right? AI has moved from being something that was uh a niche project that was primarily for you know uh um science and for some kind of decision uh science efforts to being at the core of uh the enterprise strategy and enterprise architecture. Right? What that means is there is a huge top-down pressure to both develop and deliver uh these AI applications in production. However, as we see in the slide here, what we have observed is that uh it has been very hard to deploy these applications successfully in production at scale. Everybody has been able to deploy a few niche applications in their um organization. A quick show of hands. How many of you have any AI applications that are deployed in production uh within your organization today? Quite a few. How many of you actually have the practices to deploy these at scale especially across from an enterprise perspective? Much smaller number of hands but there are a few that's great. So we are in this increasing maturity curve and I think the goal for us is to talk about how can we help you uh deliver AI at scale because what you cannot secure you cannot scale. Uh now when we look talk to our customers about what what is really causing the friction to getting them to deploying produ uh AI in production at scale we arrived at three highle reasons. The first of course that we just talked about is security right um AI brings novel threats in addition to the traditional cyber threats that we are used to right uh in particular uh models can behave unpredictably and leak data is a huge concern for many of the security officers that we talk to. The second is with our more autonomous agentic applications there's a lack of control and ownership especially around how the system behaves and who has control over various parts of um the uh a multi-step agented workflow for example. The second is uh the quality part of it, right? It's again the unpredictability leads not just to security issues but also to quality concerns about um uh is the uh model performing um with the right kind of behavior and secondly how do we deliver this quality and how can we automate around this versus spending huge number of hours trying to get this quality at scale. The last one is the cost of actually these LLM applications in particular because it's expensive right now. The good news is the costs have been going down 10x year on year. So hopefully over time this will not be a factor but today it still is one. So that is the third reason. During this conference here, you're going to learn a lot about uh how to improve your AI application quality and uh how to improve your productivity so that and your automation so you can deliver AI applications at scales. From that perspective, there are many other sessions. In this session, we're going to focus on the first box here which is on the security aspect. When we think of security there are in this session we're going to divide this into two parts. The first Arun is going to cover a comprehensive framework to uh list all the security risks across the AI life cycle and then provide a key set of controls that helps mitigate those risks. After that, I will help provide a set of best practices um on how to get started with the data brick uh with the data bricks platform and apply some of the controls that Arun would have talked about. And with that, I'll hand over to Arun to help talk about the framework. Yeah. So, Sam, why don't you introduce yourself because not not everybody in the room knows you. Uh thanks. Uh I'm Sam. I lead uh platform uh security at data bricks uh from uh uh across uh both AI serverless and uh uh data protection aspects. Hi, thank you Sam uh with that introduction. Um my name is Arun Pamupari. I'm principal security engineer at data bricks. I am co-author of datab bricks a security framework uh which we I'm going to touch upon if you haven't heard of that. How many of you heard of databris a security framework? Thank you. You made my day. All right. So for rest of you, this is the time to get to know databrisq framework. Um so it's a comprehensive work that I've been work uh doing over last one and a half year and uh it is a collaboration within datab bricks more than 30 people um it like uh security minded people uh contributed to this. Outside of data bricks there were um more than 40 people including NIST and uh MITER and uh high trust were some of the people who actually participated in this work group that I ran. Um so the idea is to provide customers with a comprehensive uh framework and make sure we recognize the risks that are there um without even thinking about data bricks as a platform. Um so the goal is to do more of a community work. what AI system looks like, what are all the different components, what threats may be there in each of those components and mapping to standards like you know MIT atlas was LM top 10 uh NI 853 I have a session today with one of the uh people who actually built on that uh in the evening if you are into NIST 853 are some of the compliance related uh uh organizations you can come to that session uh but the idea is to map those into 12 different uh components of AI system and then um we we found there are um 62 uh risks involve um you know in in the entire system and then we match one of those risks to those standards where uh those standards are called you know recognizing the same risks and then we made sure we don't leave you with um you know with those risks and then say okay you know here are the controls uh that would help you mitigate those risks. So those controls are also not uh data bricks specific. Those are general controls uh that any security or like you know IT people uh would would recognize uh and you as a as a data science and data engineer would would be able to recognize those as we go through the session today and make sure that um we leave you with how to implement those controls, how to deploy those controls if you are using data bricks as the last effort that we did. So we start with a system and then we are going to go through risks and we're going to understand what are the controls that you need to have and then we'll think about how to do that in data bricks. We have all the documentation links left in each against each one of those controls uh in the worksheets that we published uh recently and then Samro will come back and kind of emphasize on some of the key controls that you should be starting with. Okay. So with that uh this is the work that I talked about this paper this this uh once you go to that uh QR code which you're going to get another opportunity to take a picture of at the end. Um and from that document you'll be able to actually go to this worksheet uh which has each and every uh risk and the control description and the links and everything that you need to know if you want to operationalize this. Right. Um so as we move into the AI system at at a very high level you see these pro four broad uh components right four broad sections. So when I ask about like you know data engineers you fall into that very beginning you are the key um you know support team or key team that is bringing the data um so that you can actually make sense out of the data and build data intelligence of the out of that. Um and then the second PE set of people are the operations is going to be data operations which is basically model operations which is going to be building evaluating um you know everything that goes into data science side is the second piece and then once you build that it needs to be deployed at scale that's where you going to be deployment and serving of these models is going to come in or like you know sometimes you may be just connected to an endpoint that is also part of that right like you know if there is a geni model from Azure OpenAI uh someone needs to kind of provision that someone needs to share those credentials, make sure they are safe and all of that. So that goes into this third part and the last bit is of course at scale basically how I I do CI/CD um you know all the automation that goes into and what platform compliance um that the platform that is uh that is running your um you know AI system uh you need to make sure you have those uh in place. So that doesn't help put the picture it just boxes. So let's start with how AI system comes to be right. So this is the uh important piece where um we break the AI system into multiple components. So you start off with like you know a a platform it may be DIY platform some operations but most important thing is you're going to bring the data in uh into your system. So that's the first component and then you are going to do all kinds of transformations extracting the u you know the value out of the data by kind of you know featurizing and uh all the things that you do as a data engineer uh that goes into that second component and then you're going to basically um provide this data and maybe it can be synthetic data um so that that's something that you need to build off of and then you need to catalog that data so the rest of the organization knows about the data existing the existence of the data. You need to set up some kind of controls on that. You need to label the data and everything that needs to be governed for the data. So that is your left side, the first box, the big box. When we go back to this picture, uh that's the left box that kind of composes of uh those four boxes. Okay? And then starts your model operations where you are basically taking the data, you're training the models or you are taking some kind of external model, putting the data in front of that model. remember you your your traditional ML um you know is still there that's where most of the production is actually happening right you know genai is something that you are experimenting and bringing in but your custom model that are ML they are going to be part of your AI system right majority of the time what you see on the Twitter and some kind of PhD blogs is that when they talk about model security this is the box they are talking about that is important for that uh you know for the research important to protect those but talking about Modern security is talking about putting a you know some kind of armor on a king and say we protected the king. There is a lot of things that go around the king to make sure we are also protecting you know uh the perimeter and everything that goes into to protect the king right. So model models are important. They are your digital twins. They need to be there is your intellectual property put into those models. So yes you need to protect them but data is also super important that you are protecting because that's the data that is going to go into your models. That's the data that is going to be exposed to the agent. So you need to make sure that is also taken care of. Right? So once you have the models then you're going to give it to your infrastructure team that that's third box where the infrastructure team is going to operationalize these uh you know build endpoints out of that. This is where your models come alive. These are the models that go into your apps. These are the models that are going to be integrated into your upstream downstream systems and you need to make sure there is sim similar uh guardrails and similar protections controls that you have in your traditional uh applications. As Amrad mentioned AI is your part of your enterprise software now that's why the security is more important than maybe a couple of years ago when AI was a output of AI was kind of a PDF file from a data scientist. So there is less emphasis and less need for uh you know worrying about the PDF file ending up on somebody's wrong inbox. Um but here if something goes wrong um everything upstream downstream and data uh excfiltration is going to happen through these systems right so once you have this system that's when your um uh you know external applications are going to interact with users are going to interact with through prompt engineering through inference requests if it is traditional ML and that goes into your uh live system that is uh productionized so with that now AI system is running and agents are talking to your data Um remember without data being put in front of your AI it is just general intelligence that means you can ask and do your kids homework but you cannot ask and answer your enterprise questions right so to get value from that AI you are going to get get the data which is in the cataloges and files and everywhere that is in the enterprise and you need to put it in front of AI so that you can answer um intelligent u you can answer the questions with intelligence so you need to monitor that AI and And there's going to be feedback loop. So as the apps come in and talk to your uh system, there's going to be up and down like you know yes, this is the answer that I'm looking for or there's some kind of feedback that becomes raw data that goes back into your uh your rest of the system. Right? So now you're going to put all of that on some kind of uh you're going to operationalize that with some CI/CD process. You're going to put it on a platform and the platform needs to have some kind of security controls. Right? So this is a big picture of uh I call AI system. This is what we and the enterprise need to be caring about. Not just the model security. Model security is important but everything else around the model needs to be important. It needs to be taken care of. So when we did the study we took the whole thing because uh the approach for enterprises you all know is that uh defense in depth. You cannot just put one or two controls and hope that everything going to go well right. So to do difference in depth first of all you need to look at threats at each one of these um components. So as part of the cataloging of each of these risks by studying for last almost two years um we we found the blue numbers are 12 different components right in the AI system and then at each one of those numbers you see um a you know an orange color uh alert which is basically the number of threats that we identified in each one of those. So here is the comprehensive list of threats that we recognized as part of that. So each one of those 12 components as you can see at raw data we said 11 risks. So these are the full list of this these list u the each one of those is described in that excel spreadsheet or Google sheet if you if you are on Google we publish both of those both versions and described at length what this risk is why and where they appear and who who else is talking about is highrust talking about it it is you know um NIST 853 talking about this or like you know some other WASP LLM talking about this risk so we kind of catalog those so that you If you're following one of those standards um you know NICF 2.0 DO you can work backwards right so as I said we didn't want to leave you with only risks we want to give you controls that's why we are the platform we are the vendor we are supposed to provide you assist you in deploying those so here are the 64 controls again you don't see data bricks name in them like that was by intention because it's a community work we wanted to provide for anybody who is practicing AI DIY to even our competition you we will have armed with this you can ask the right question saying that Hey, we want to know how to deploy this control um on your platform. If you're using data bricks, you ask our people, you ask us or you can go to that excel spreadsheet and there will be documentation link for each one of the datab bricks clouds like you know if you're on AWS there is a link uh you will find how to configure datab bricks to deploy these controls right so with that I'm going to hand it over to um hand it over to Samrat and to kind of go through some of the best practices or some of the controls that you should be thinking about u because we don't want to just leave you with some Excel spreadsheets and some uh links and everything. So Sam Sra, take it away. Yeah, thanks Arun. Well, as the slide says, that's a lot. Uh yeah, so uh AI security is not optional. But then uh wrangling 64 controls across your entire life cycle, software development life cycle is hard. So how can we help you get started? So here we're not going to go through all 64 controls obviously, but we want to help you get started by focusing on three broad focus areas and key controls that are enabled through the data bricks platform uh that help you get the most uh proverbial bang for the buck. Right? So the first one is a broad area where as um Aron talked about is building and deploying on a secure platform. So this and the second uh dimension is to make sure that you have the policies and the um controls in place to have an end-to-end governance of platform data and models all three together these two boxes together form what I would call as the security scaffolding right so effectively it's the sort of set of things that are the preventative measures as well as some of the more deterministic detective measures that you have that help create the housing in which your intelligent applications can live in effectively right in a way that is secure. However, even after you achieve this, all right, one of the key things about these new uh LLM based applications, the genai applications, whether they're agents or some you just applications that are using these genai models is that they are unpredictable. Right? All of you would have done some level of perhaps system prompts etc to constrain what the models are doing right you might have even implemented LLM as a judge by putting another L model in front of it but despite all those steps uh from when I talk to the people who know who are much smarter than me in this domain um all of them tell me that it is there is no 100% guarantee that this thing is going to work as you thought it would right so one of the key dimensions of Security here is for you to have a practice, operational practice to monitor and audit your model performance from a security perspective which is from an adversarial attack perspective, from a data leakage perspective, all of those aspects. You must have this control in place through the life cycle from the start because you you have to reduce the risk before you take it to production. But even post-production you need to be continuously be monitoring for this because things can change over time right uh based upon both uh the change in your model but also the change like you know development of new threats for example in how prompts are um uh can be used to game these models. So let's start with the first one building and deploying on a secure platform. uh platform security for those who have been in that area obviously is a vast area on its own but from an AI perspective I would say there are three core things that you need to uh think about the first is you want to run on a hardened runtime on data bricks the good news with data bricks is we do the hardening for you especially if you're on serverless we do most of the hardening for you from making sure that you know our all the data is encrypted at rest you got right kind of isolation of your containers you know that you've got um you know proper proper privilege management of each of the containers so on and so forth. The area where you need to focus on and I'll go through the next few steps is making sure you've got a secured parameter. Why is this important is because um you may have not thought of like a secure parameter being a mustave before because you're building an application, you know what your application is doing here because the behavior is somewhat unpredictable. Again, the scaffolding putting the parameter together is very important. And lastly, you have to encrypt your data. Like one of the things we have noticed with these applications is in order to uh build these models right correctly you can no longer just bring uh uh mask data to the cloud you can no longer have redacted data to the cloud you have to bring in the full fidelity data. So it becomes even more important to make sure that you don't have side channel access like you don't want your infraadmin gaining access to your employee records for example or to your quad financial uh result uh kind of data sets right um so securing the parameter what does data bricks provide for you and I'm not going to dwell into a lot of depth into each of these uh because of time limitations but I just want to talk through a few key things we are providing a serverless platform for most of your AI functionality Right? Like and that's a great thing because it's a hardened platform to begin with. You have to have less things that you have to worry about in terms of security. But one of the key things about moving out of your network is what happens to the firewalls that your IT maintain for you. We now restore that functionality through a feature called serverless egress controls. It's built in and optimized for data bricks and helps you control all your outbound connections across three surface areas. the internet which is the obvious one but also to cloud storage making sure that your applications cannot access like public buckets out there which have not been sanctioned and lastly even on the data bricks API surface so for example you can't have your model call the files API of another workspace in a different account and excfiltrate data that way right so these this is very important for two reasons the first is of course to mitigate data excfiltration risks by somebody gaming your um agents to do things that they were not supposed to do but secondly it's also very important for you to secure your software supply chain because now with this control for example you can uh ensure that your systems when you're building them when you're deploying them do not have access to untrusted repos because you don't know what's getting downloaded in many of these um uh sort of uh development life cycle. So you can cut off connections to all public repos and make sure only scanned and approved binaries are accessible through your private repos. And you can access these private repos with a new feature that we have now which is the ability to configure private link from your serverless runtime into your VPC. So if you have an artifactory there then you can just configure the private link to connect to that. Yeah. So this also helps with like um if you if you have a model like deepseek and if you are another third party model um from hugging face and if you want to bring it into your infrastructure you need to make sure the model is um isolated in three layers of isolation cannot reach into your data assets without your you proactively configuring them and it doesn't exfiltrate the data u without you without you knowing uh back to you know the attackers. uh you know system right like you know that's why sometimes you know on hugging face they may plant a model and it may be attractive for your business and you may want to bring it but you need to have those controls in place in your system before you put them into your system. Absolutely. Uh thanks Aron. The second thing to consider is DOSS attacks especially if your models are being exposed close to the internet like there hopefully you're not nobody's putting the model APIs or the model interfaces directly on the internet with a thin UI. Hopefully, you've got some app in front of it, but even with that, it can still be subject to DOSs attacks. And therefore, uh we have uh uh a component called AI gateway as part of our model serving infrastructure. And that AI gateway does give you very basic per user and per endpoint uh ability to define throttles. And that helps you mitigate risk from um DOSS attacks, especially at the AI layer. And lastly, here's a new feature which is in private preview. Uh at this point is a contextbased access. So I'll give you an example of why this is important in the AI world. Back in the day when you have you were using data bricks just for doing your notebooks. You could just say oh this is my back office. I'm just going to make it private and just keep it there. But with AI you suddenly need to connect it with other systems. So for example one of our customers they want to connect some of their models to their Salesforce application in in the cloud in SAS. Now Salesforce is a multi-tenant app. So when even if you provide some credentials to connect to your databris, you're not 100% sure which tenant in Salesforce this is coming from because they all share the same IP. So they want to reduce the exposure and say I only want these two models to be accessible from Salesforce. I don't want to open up my entire workspace to Salesforce, right? And now we are building this capability for you to do just that. So you can now open up very specific endpoints in your workspace uh based upon the identity of the client as well as the network source of the client. Right? This feature is right now in private preview. What we have in GA is the ability to keep things private or define um IP access list at a coin level for your entire workspace. So I talked about platform security and you were like this is not AI security right like except for maybe perhaps a little bit of the AI gateway but it's actually very critical to AI and for anybody who's been um uh working with regulated uh data. Um so here's a customer which is National Australia Bank. you might have heard of. They're one of the largest banks in Australia and as you can imagine as a bank they're highly regulated and they were able to actually take um deploy a jire capabilities within their organization by leveraging these very features and that's what they called out when we said okay what enabled you to get to production they've called out things like serverless egress control private link connectivity as well as like AI gateway in there right as key capabilities that enable them to get there so let's talk a bit about governance and uh the governance has uh three components again uh you must make sure that everybody's properly authenticating to your AI endpoints and this is pretty standard I'm not going to go into details there you have to do MFA oath is a better way to authenticate where I'm going to focus a bit more on is on fine grain access control through the AI life cycle to make sure that everything is properly governed there and lastly this is a new feature that we have just launched very recently um it's the that your AI agent is running with the right author authority. What do we mean by that? Let's take an example where I've got an HR uh bot in my enterprise to help employees, right? And it helps you understand your compensation and what's going on. If it runs with a service principle and it has access to everybody's compensation, right? I could through some appropriate prompting create a confused deputy attack effectively in security terms which means that I can get Arun's uh compensation uh through that uh HR bot and Arun is obviously not happy about that right so we need to prevent that from happening we need to prevent this confused deputy kind of issues and we need to mitigate those risks we need to push the access control away from uh the model trying to do the right thing to your back end being able to tackle correctly on Right. Um, so that's very important. So agent is your uh representing you. So agent should not have more permissions than you just because it's an agent. Right? So when you are asking an agent a question, agent should say this is Jane and then agent should represent Jane when it is asking for data and the system that is providing the data should only return the data that is that Jane would know uh if they run if they ran a SQL query. Yeah. Right. So that's the purpose of that OBO on behalf of Yeah. We'll talk about that a bit more. But of course we'll start with the heart of our governance story which is Unity catalog. I don't need to introduce Unity catalog to any of you. So I'm not going to spend time on that. I just want to uh talk about a few new capabilities that we think are particularly relevant from from an AI landscape perspective. The first one is attribute-based access controls. This feature is now in what we call as beta which is like a very early uh preview state that you can start uh playing around with. uh and what it actually allows you to do is you can define tags in your uh data sets like for example PII tags and then you can define dynamic policies that is given a tag who should have access to it should it be masked and so on and so forth right this is very critical because when you train on this data you want to make sure that only the models that are meant to be trained on PII have access to PI nothing else has access to PI right so that is very critical and this allows you to do this at scale effectively it was a very hard thing to do if you had to do per model you have to define an alkalle that was very static right so these dynamic policies are very important right and to help you with the this tagging effort because it could be a major effort for you to tag we now have a new feature again in beta called data classification that is basically an AI um model that helps classify your data according to various types including identifying pi now just as with any model it may not be 100% accurate so therefore a human in the loop is a very important thing to to make sure that your tagging is accurate eventually, right? And lastly, Unity catalog offers integrated lineage between model and data. So for example, from a model to its um um you know like to its parameters um you know a feature store it and how that is derived. So you get an integrated lineage which is very important for you to determine model behavior and understand what the model has access to effectively. Right? And um as I mentioned this, the key reasons why you want this is you want to minimize access to sensitive data from less predictable um applications and you want to make sure that you have a good way to do root cause analysis when the model is misbehaving because you can identify drift over time in terms of like what data um was being used to train the model. Second thing um I'll bring back AI gateway. you'll see AI gateway show up in multiple places because it solves many problems. uh uh is you want to of of course Unity catalog helps you govern everything across your life cycle from data access to model access at uh but once you've deployed the model in production then you still need to be able to determine you configure access control to the serving endpoint and that is done through an endpoint alle feature that we have and on top of that uh the model itself or the agent would need access to other systems like a model behind the scene and so we provide you with the ability to uh securely configure cred credentials and not have the models have access to the credential. Actually, we hide the credential away from your agents and the models through the AI gateway. So, the modeler credential can just call the AI gateway to get down access to downstream systems and this is the on behalf of capability that um that we just referred to earlier, right? And uh over there uh we're talking about the agent itself can assume the identity. Think of it like those who are familiar with this assump identity assumption or I am a role assumption from the AWS world. Uh it's very similar in concept what are you running as and effectively the agent can run as a service principle when it's talking to shared resources but it can run as you when it is having to talk to data that belongs to you as a user. So therefore if your backend systems for those which are like sensitive user oriented information should only grant access to the users and not to the service principle. Therefore, the agent has to use this particular downscope user token that is issued that gives access to very specific entities on behalf of the user uh in order to gain access. And this again provides a guardrail effectively at a not in the context of like the classic AI guardrail but in terms of security guardrail to make sure that the agent behaves correctly. So the last part of it um talking about monitoring and auditing which as I mentioned is very important. And the last but definitely not the least part of the security here. Of course, you're going to enable and monitor Unity catalog audit logs end to end because you want to understand what has been accessed particularly at runtime. You must have a sense of what data is being accessed uh particularly at after things have been deployed over there. But it's important that you understand how your model is behaving and this the part that I was talking about which is nondeterministic 100% because it's the intelligent part of your application. How is that behaving? You must understand that end to end. So you must have established a baseline of what the model behavior is at the time of development and this can be done by creating the right kind of metrics and um measures that you can use to establish that baseline right. Uh you can also configure signatures in MLflow for example. So all of that helps you establish a baseline and then you have to compare that baseline through the life cycle and then even post deployment in production you have to continue monitoring this because things may change after you have deployed and things and you have to see whether after deployment is the model behaving in a way that is u insecure. So let me talk about a few um capabilities within the mlflow framework that and as you know data bricks provides a managed mlflow service. So with the MLflow framework um there's auto logging which has always been there and that allows you to capture lineage hyperparameters metrics files to establish the baseline for a model. But now with MLflow 3.0 which just launched very recently we now have uh MLflow tracing and this is an end-to-end observability system for your generative AI application. So for example, if you look at the um uh screenshot on the right, uh it gives you the full trace of a particular rag agent um and you're able to trace every step of the rag agent. You're able to see the intermediate results um uh sorry the outputs and also which documents it's retrieving, what data is it accessing. All of this now certainly becomes available to you to understand and then capture as a metric and as a trace. So then if the system starts behaving differently, you can compare against your baseline trace to understand the risk and and harden against um potential adversarial attacks as well as if you need to harden your like harden your system prompt. So to make it better, you can also use this as a way to do that. Uh lastly, uh what happens when you deploy to production? So MLflow helps you all the way till deployment in production. So but you want to as I said continue monitoring it. So again the AI gateway comes to the rescue here. One more time uh you are able to now um it by default it enables the usage auditing. So when you deploy in production it comes with AI gateway and you get various uh uses like classic audit log kind of metrics out of it. Who accessed when, what was called, so on and so forth. However, one of the key features that it has is the concept of inference tables which you can enable. And when you enable this inference tables feature, it actually logs both the user prompt as well as the response payload from the uh from your application or from the model. Right? Uh this information can obviously contain sensitive data in itself both the prompt as well as the payload. So again this data is not stored in some random place. that is stored in a unity catalog u governed uh location right and uh subject to your unity catalog access policies right u so so to summarize you have to monitor for all your sensitive data leaks through the life cycle of the product you need to monitor for what the system behaves against adversarial attacks so you need to do the red teaming up front and make sure that this output from those adversarial attacks are captured in your metrics and you have established a baseline against it and you have to continue to look at that and make sure that you're using all of this to harden and to enhance your system prompting. One thing which I did not talk about but I just will touch upon to end it is that your system prompts should be hidden from the vast majority of your users because that's the core of how you are controlling for some of this behavior. So you can do that by making sure your models are not directly exposed to the wide users but that you have an application layer in between that only allows user prompts to be set and not system prompts. Uh with that I'll hand it back to Aron to help wrap this up. So thank you Samraat. So we only covered um maybe about nine or 10 controls of 64 um controls that are out there. Um so this is taking that picture AI system. This is the first time I'm actually now bringing data bricks capabilities at each one of these components. There is a database capability that you are all familiar with and they come with security and uh monitoring and governance that is needed for AI system and if you observe I've been saying AI system it is by intention we want to think of AI as a system not as a model is just a simple part of entire system and we need to make sure we address the entire AI system security right um so if you want to download the paper um I know we went really fast. Uh there is a lot of good work that went in. Uh thanks to all the contributors. Um you will be able to get aund I think eight page PDF file. You can put it with notebook LLM and listen to it on your commute. Uh I wouldn't recommend you reading the whole paper even though I am the co-author of it. Uh there is execute ebook that I would recommend you to eat it. Uh read yeah you can eat it read read it. And then uh the PDF the the uh u excel and Google docs sheets are available the links from that that is the most comprehensive one that's the one that you want to share with your colleagues think about hey you know here are the risk did we think about all these risks and it's okay if you you know if you ignore or if you say this doesn't apply to us but knowing that these exists and then knowingly making the decision is more important uh And to give you and help you with operationalizing this framework uh you know go walks you through the worksheets walk you through a process of taking a use case looking at how you're deploying and using AI and then you can filter down based on the way you are deploying the which risks apply to you because let's take a simple example model theft is a risk but if you're using Azure OpenAI endpoint the theft part of the model is responsibility of Azure OpenAI It's not your responsibility, right? But the security credentials that you got are responsibility for you to protect those credentials and safely store them in AI gateway and make sure you are providing that access control to your employee so that all of that shared responsibility is also discussed in that worksheet and in the paper so that you know which one you own, which one the vendor owns, which one the endpoint uh provider owns and then you can put this to work. With that um I'll leave you with u you know the resources that are out there. Thank you very much and uh if you have any questions we'll stand outside. I think we are at time uh but we can answer those questions outside. This was of value to you. Yeah. And please uh provide your feedback uh through the app. Thank you.