How to Fix Years of Database schema drift using AI

Key Takeaways

Hello everyone. I hope I am audible to everyone. So, we will start soon. Feel free to ask your question either in the chat or you can also talk. I will give you the permission to talk in the Zoom. Feel free to introduce yourself in the chat like you can tell us where do you work at and what work you do. Yes, the recording will be available later. It is anyway streamed on the YouTube, so you can get the recording there itself. Okay. So, I will start now. Yeah. Okay. So, today in this webinar we will see how you can connect your database to our AI or AI agent and you can securely access it and accurately get the answers from your database. Okay. So, we will also see how Turno AI does it and how Turno AI is able to securely and accurately give answers on your database. So, there is one component in Turno AI that is called Turno DBA that we can see here. Right? So, this is the essential part of Turno AI and which provides the security and other things for for the database and we will see how that works in this webinar. Right? So, the agenda agenda for this webinar is this. Right? Whatever I just told. So, So, we have three speakers here. So, myself Sandeep Akode. So, I handle the technical aspects of Turno AI and look after various things here. And we also have Naveen and Akansha here. They will introduce themselves. So, let me start the slideshow. Okay. So, what's the problem with AI AI or AI agents? So, the main problem is they can often hallucinate the data when we provide them a lot of data when we directly connect to our database. So, in that case because of the huge context, it can hallucinate the data. Or when the database is connected, the table actual table name and column name might might be might have some vague names like suppose in our company we have some database. So, we we can name it anything and that term might be known only internally. Like table we do final or something else. It can be anything. So, that table name or columns names are not very intuitive to the AI AI or AI system. Right? Or there can be missing foreign key relationships. Like in case of database they have foreign key relationships, but in case of data warehouses they store data in different different tables instead of creating a foreign key relationship to to save the computer time. Right? So, in that case they have missing foreign key relationships, so they are not so the AI agent is not able to answer questions very well. Then the raw database schemas can confuse the AI. Right? And the business logic can be it may not be documented anywhere, so it is quite hard to answer questions on your database. Right? So, first we will see what is Turno AI. So, in single line Turno AI is your AI data scientist. So, as a data scientist what task you will do, Turno AI will be able to assist you and help you in doing that task faster and like in a very accurate way so that you can interpret multiple things and answer get correct and accurate answers on your on your database. So, it can learn and understand your data sources automatically and also its interaction with you. So, what is Turno AI? It can instantly analyze the enterprise data and we we can also get all the insights from the questions that we ask in plain English. So, even if you are from a tech team or a non-tech team, both of them can get answers and insights using Turno AI. So, you can simply ask a question in plain English and it will be able to answer you. Uh this diagram will be uh clear to you. Yeah. So, you can connect any type of data source, MCPs, ERPs, warehouses, and data lakes, etc. You can also insert files to it like CSV, JSON, anything. Okay. We will see how everything works in a demo soon. So, Naveen and Akansha will give you the demo. So, Turno DBA as a middle layer. So, Turno AI is our main agent. But there is one layer that comes between Turno AI agent and your database that is called Turno DBA that is database interface. Okay. So, this Turno DBA ensures that the all the things whether whatever you are executing on your database, those are secure and those won't harm your database in any way. Okay. So, it is an intelligent layer between agent and database. Here agent is Turno AI. And database is your database. Turno DBA sits between agent and database. So, it is already public and open source, so you can access it here. I will paste this link in a chat. Just give me a second. So, Yeah. So, if you have any questions regarding anything in this presentation, you can just ask in the chat or just DM me. Okay, we will go ahead. What is Turno DBA? So, we have seen what is Turno AI and now we will see what is Turno DBA that sits between agent and the database. Once again, give me a second. I think Naveen will handle from this. So, it's now to use. Naveen, please go ahead. Yeah. Uh thank you, Akode. So, uh hello everyone. My name is Naveen and I am a software engineer at Turno AI. So, I am one of the engineers who developed Turno DBA. So, I will uh walk you through walk you through the next part of the presentation. So, uh yeah, let's start. Uh is my screen visible? Yes. Okay. So, as Akode already uh discussed what is Turno DBA in a deep way. So, I will discuss uh what exactly it is. So, it is a secured intelligence layer which sits between any AI agent and data warehouses and databases. So, you can connect Turno DBA through any of your AI agent and you can interact with your data warehouses and databases securely and correctly. So, what it will do is when you connect your database through Turno DBA, then it will just create a virtual schema out of your database and it will also enrich your database with the descriptions like it it it will create table and column descriptions so that AI can get context aware uh and so so that AI can get context about your database and it will generate the correct SQL. It also controls and governs data access for AI agent. And it also translate LLM generated queries into the actual warehouse schemas. Uh I will discuss these things in the uh in the demo like how it exactly will do these things. And it also validate and safely execute queries on your data warehouse. So, let's see how Turno DBA exactly works. So, uh like firstly, we have to install Turno DBA from PyPI. So, let's go here on the uh on the GitHub page of Turno DBA. So, we will just follow the quick start and we will install Turno DBA in our system and then we will start Turno DBA and we will connect Turno DBA. Uh we will connect our database uh in Turno DBA and then we will uh we will uh interact with our database through any AI agent uh such as Cloud or Turno. So, firstly, we have to install Turno DBA. For that, we use this command pip install Turno DBA. So, I am just yeah, I'm just uh open uh my terminal and I pasted the command pip install Turno DBA. Since it is already installed, so it get installed uh in very few seconds, but it can take uh uh 20 30 seconds in your case if it is not already installed. So, after that, uh I have to start the server. For that, I will use this command Turno DBA start. So, I am just copying it and I'm pasting it here. And yeah, it just started my server. And uh there are some text which is written here. So, firstly, it is telling me that Turno DBA started. Then, uh it is telling me that our server is at this link. So, I am opening up and here we can see uh uh Turno DBA is started. So, I am going into admin and yeah, again it is uh telling me that admin panel is here at the this link {slash} admin. So, yeah, I have opened that. Now, it is telling me that add your data source and then connect with any AI agent via MCP. So, I will do this two step. So, for that, I have to click on add button in the data source. So, I'm clicking on here. Then, uh I have to give the display name, then description about database. Then, I have to select my database type. Uh then, I have to give my connection string of the database and then I have to just click on save button. After clicking on save button, uh it will just create a virtual schema of our database and uh it just got attached. So, I have already created one one of the uh I have already attached one of my database, which is Neon DB database. So, after creating it, you can see the tables and the columns of it. So, after that, uh I have to like uh I have already configured my uh database. I have already launched my server. Now, what it is telling me that after after uh uh after configuring your database, you have to just configure the MCP server into AI into any AI agent so that you can interact with your database. So, for that, uh I will use Cloud Desktop. So, you have to install the Cloud Desktop or any AI agent which you want. You can also use Turno. So, yeah. Uh I I have just uh installed it. Uh I I have already installed it and uh then I have to go on account setting developer and then I have to uh copy paste this I have to copy paste this JSON configuration into uh into this. So, yeah, I will just walk you through how we can do. So, we have to click here. Uh then we have to click on settings, then we have to click on developer, then we have to click on edit configuration. When we will click on it, then this will come and yeah. This file just open. This JSON file. Here, we can configure our MCP server. So, for that, I have already configured it and attached the API keys for it. So, we can see here that we need API key also, which is service token for our database to secure it. So, for generating the API key, you have to just click on service token inside. Yeah. Uh you have to click on service token and you have to add your service token to add it in the to add it in the configuration configuration file for any AI agent. So, for generating it, you have to name it. You have to uh you have to select the type and you have to give the access of your database to this token. And if you want, you can also uh set the expiry time. And just you have to click on save. After clicking on save, uh after clicking on save, you will get a token authentication key like this and you have to just copy paste it here at this place. So, after that, uh you have to uh select the query type as admin service and you have to copy paste that token and after that, you will uh click on save and then it will also create a token for uh admin service and you have to copy paste that token here. So, after that, just save this file and go on to the Turno Desktop like uh Cloud Desktop. And just restart it. Then, you can see here that uh your your two of the uh MCP has been configured here. The Turno DBA admin and Turno DBA query. And if you want to see its tools, then you can also see here. These are the tools which is available in Turno DBA admin and these are the tools which is available in Turno DBA uh query. So, after that, uh you you just want to query your database. So, I have already queried my database. I'm just showing you what I have done. So, I asked list my data sources. So, it used it it just load all the tools which is available, then it used the tool which we provide in the uh query service list data source and it just give me the data source using this tool, which is list data source. Then, I told him to generate the descriptions of its table and column and attached here in the database. So, it successfully generated all the uh all the descriptions of table and column and it just it it it attached into the uh database. So, we can see here. So, it is generated by uh LLM. And we just we have to just say to uh any AI agent and it it it will just generate that uh description for the table and column. Uh this way we can enrich our uh database. So, after that, I just asked him to give me analytical question which can ask which which I can ask to this database and yeah. Uh it it given me this question and I have asked this. And yeah, it just uh written out this uh query to this data source and yeah, it it just give me the uh final answer, which is the total revenue generated. And yeah, so this is the whole configuration how it works. I will come to security part uh later. Uh just uh like uh I have to uh first go through the presentation, then I will come to security part like uh what how we can secure our database, how we can secure our uh uh like how we can secure the AI agent query on our database such that it will it can't delete the things, it can't delete our uh table, column. Uh also, uh I will show you that uh how we can hide our table column, how we can make table column private through this through Turno DBA. So, we will discuss that later. Firstly, uh We have one question in the chat. So, Pavitra is asking, is this a like library, Turno DBA, or um like if we can connect it to different data sources, and also it seems like you're using VS code something. So, Pavitra, it's a Python library, the Turno DBA, and you can connect any kind of databases through it. And uh for like uh he used VS code to configure the MCP JSON file, which uh he showed in the uh presentation uh to connect to cloud. So, uh that is Yeah, uh I just opened that configuration file into my VS code. Like, you can open it from anywhere. Uh so, this thing, like edit configuration, you just open with any of the thing. You can open it with notes also, text editor. Yeah, this file. And you just have to edit this. So, you have to just copy paste this JSON file which I have provided in the GitHub repo. Here. Uh Wait. Yeah, here. And you have to generate the API key, which I have shown. And you just have to uh in this way you can just configure your database. You uh the Turno DBA. And to connect the database, uh where do we need to go? Yeah. So, to connect the database, we have to go on the server, which is this one, which we started using Turno DBA start. And we just have to go in the data source field, and we just have to click on add. And here we can easily connect our data source to connection string. Yeah. So, we have discussed this thing, like the installation part, how we can launch our server, and how we can connect our data source, how we can connect uh Turno DBA as MCP through any AI agent, and how we can generate the token. Now, what is the tools available inside Turno DBA, the admin tools and query tools. And how we can interact with AI agent, we have discussed all of those things. So, let's just start how it exactly works. So, uh firstly, I will discuss this, the entire flow, how it exactly working. So, after launching the server, we configure our database, and we connect Turno DBA to any AI agent as a MCP. So, till this, we have done things. Now, we wanted to know how it exactly work, how it exactly answer the answer our query, how it exactly providing the security. So, let's say I'm a user, and I have a uh normal question, like what is the tables inside my data source? So, I just type that as a prompt to AI agent. Now, AI agent have LLM. It will decide it will decide which tool which tool I have to use to answer user's query. So, after deciding that, like I have asked to give me the tables of my database. So, it will it will just see the tools which is available to list the tables of the data source. So, it will just use list table tool. And after doing that, it just authenticate that user, that the user that user have the proper authentication to execute that tool to see the data source tables of that data source. So, it will just authenticate the user request, and after that, we have the security engine, which is SQL shield. So, let's say here uh the query is uh pretty normal, it just used that tool. But what if it has a SQL query? So, there security engine, that is SQL shield, come into the picture. It will just look at your uh SQL, and then it will see if it has any if it has anything in the query which can disrupt my database, which can delete the things, uh and all. So, it will just say it will just give error that you can't perform this operation. I will I will also show you this thing in the uh upcoming slides that how it exactly do this thing. So, after authenticate after authenticating and uh making the query secured, it will go to the metadata layer, and yeah. J- and after that, it will go and on our database, uh and just it execute that SQL query, which is generated by agent, which is now secured onto our database, which here uh where we have the roles of it. So, it just execute that database, and it get the result. After getting the result, it will pass to your AI agent to the LLM, and then it will give you the answer. So, this is the exact flow how it works. So, uh like this Turno DBA has two MCPs. One is uh the one is the query service MCP, and another one is the admin service MCP. So, query service MCP give you the tools which is related to query, just like list data sources, uh list tables, list columns. It it it can't uh it can't modify your table columns or data source. And in in uh like uh we have also admin service tools, uh which is like rename the table name, rename the column name. Uh don't worry, it is not the actual table name and column name. It is just the metadata table name and column name, which uh like the virtual schema uh table name and column name. So, uh and also generate the uh like uh update the uh descriptions of my table, update the descriptions of my column. So, these all tools are available inside the admin service. So, uh like uh we discussed that how it exactly uh connect your database, and how it uh create the metadata And uh yeah, we have like uh in your database, you have some name of the table and column. So, it will create a public name of that table and column, and you can change change the public name of the table and column of your uh database into Turno DBA. The actual name is uh just uh it just what it is, and you can change the public name to a better name, we so that AI agent can understand that easily. Now, uh we discussed that how we can uh connect the Turno DBA to the AI agent, and how we can generate the descriptions and all. And yeah, we have also discussed that how we have uh like like uh we have provided Turno DBA as a MCP, and you can also connect to any of your AI agent. So, yeah, if we have discussed this part, this split MCP architecture part. Now, we have the security and access control part. So, let me uh give you the demo first. So, yeah. I've asked it, like uh we can see here that in this data source, Neon DB, we have three tables, products, user, and orders. Now, I asked AI agent to delete one of the table, which is product. Now, it it it tried to delete it. And it used a query called drop table products. Now, it it got an error. And in error we can see it is saying that as of now only select statement are supported. So, this means the So, here our SQL shield, which we provided it as uh security layer, which came into in picture. And it is not allowing this query, which is drop table. So, we can't drop tables of our database. So, this is one of the security part. Now, another security part is uh we have something called private private table selector and private column selector. So, through this what you can do is, let's say I'm clicking on private table selector and clicking on the data source. So, let's say this this three are my tables, which is present. Now, I want that uh my like I want to make this user table as private table, so that nobody can see it. So, that AI agent can't see it because it is private. Because it may contain some sensitive information. So, we want we want uh want to see uh we want we we we don't want that AI agent can see this table. So, we just make it private. Now, after that we just save it. And now, we just ask to AI agent that what is the tables that is present inside NeonDB data source. Yeah. So, it it will load all the tools and it it load the list table tool. It is firstly seeing the Yeah. It it firstly see the data source, like what is the exact name of data source. So, it says me that uh yeah. It after seeing the exact name of data source, it it see the table name and here it executed that query. And here it gave me the exact table name that NeonDB data source contain. And we can see here we it only give me products and order table because we have already hidden the table called users table. If we remove it from here and save it and then ask it again, then it will give me all those three tables. So, if you want to try, just ask it. I'm just asking it again, what is the tables which is inside NeonDB data source. So, it listing down all the tools and it gave me the table names. Like it has product, user and orders table. So, now I have removed that table from private table selector. Now, uh it give it given me all the tables. Here we have zero tables in the private table selector. So, this is the second thing which we provide. And also, let's say you have some tables which is private, but you want but you want that uh some of the people can see that table. So, for that we have group table selector. So, you can include that table into group table selector and people can see into their query that this table is present inside the database. But this is universally hide it hidden from all of the other people which is not present in the group which you created for for this uh hidden table. So, this is the second thing which we provide. First one is the uh uh we can't perform any destructive SQL uh on our database, so that it can delete, drop or change our actual database. And uh second thing is uh Second thing is we are providing uh the private table selector and private column selector. Third thing which is present is table row filter. So, how it exactly work is uh you have to just select your data source. You have to just keep the name of the table and you just have to apply the filter on the row. So, how it exactly work is uh I I have given it here like row is greater than 1,000. So, amount is greater than 1,000. So, uh if I will show you in the presentation. So, how it exactly works is Yeah, here we can see uh like we have asked this query to what is the revenue across all order. What is the revenue generated across all orders. So, it given me the revenue of uh around 3K dollar. Right? Now, I have applied a table row filter, which is this that amount which is greater than 1,000, just show that rows only into uh whenever uh AI agent will query the database, then apply this filter into the SQL automatically. So, now again when we ask this same question, so we can see this query and this query, both of are same and we have just applied this uh this filter on our rows and we can see that uh the revenue generated is decreased. Means it is uh just filtering out the amount which is greater than 1,000. So, here we can see the total revenue generated across all orders is 1,200 dollar. So, yeah. This is the third thing we provide in the security. And uh Yeah. Yeah, there is one more question from Pavitra. Okay, like can they use TurnoDBI for data and migration related things? So, I guess TurnoDBI cannot do data modifications. It can do like select queries. And uh any more information you want to put into this? Yeah, like TurnoDBI will create uh layer upon on your actual database. It will not modify and do anything on your database. It will just create a virtual schema out of it. And uh Yeah, it will just give those table and column to the AI agent and that AI agent will write the query and that query will executed on your actual database. Hope that answer your question. So, next part is advanced integration. Like uh uh as we discussed that we will provide TurnoDBI as a MCP and you can use with your any AI agent. We also provide TurnoDBI as a Django plugin mode. So, let's say you were building any AI agent or any AI system in which you want that uh I have I you want that a you want a ready-made system which can attach your database, which will generate the descriptions, which will rename the column and table name, then you can directly use this TurnoDBI inside your project. And uh for doing that, you have to install using this command pip install TurnoDBI. Since TurnoDBI is uploaded on PyPI. So, and you have to just give the path uh all the URLs of this TurnoDBI like this. And you just have to migrate the uh tables of it into your database. And you can simply import its model and services into your code base. And you can now uh use this uh TurnoDBI inside your existing Django app. So, uh This is the overall presentation. And we have uploaded TurnoDBI at PyPI. Yeah. You can just install and uh this is the GitHub page of it. This is open source. Uh so, if you want, you can also uh contribute to it. You can use it and yeah. So, if you have any question, you can ask. Naveen, can you share the GitHub link in the chat? Yeah, yes. So, if you like it, give it a start. Uh hi Patrick, we have allowed uh you to talk. You can unmute yourself and ask uh your question. Uh yeah, Naveen, there are other questions uh in the chat. Uh there's one from Manish. When you were creating the data source, I did not see the MS uh SQL database server. Was it there? Yeah, it is there. You can connect your MySQL database also. Okay, I guess there are no more questions, right? So, uh do you have anything else, team, that you'd like to share? Yeah. Thank you, everyone. Have a good day. Yeah. Thank you for wonderful set of questions and yeah. Bye-bye. Have a good day.