Build AI-Powered Applications Natively on Databricks

Key Takeaways

Hello Bricksters. Good afternoon. Thank you so much for joining us. My name is Allison Baker. I'm a senior solutions architect with Data Bricks and I'm here to introduce you to our next speaker. Couple of speakers we've got. Before we do that, I want to take a moment and share a few things that we have ahead in the afternoon. So, we've got a few more sessions coming up after this. So, we're going to go from uh from this quick introduction to building AI powered applications. We're going to hear from a few folks on that. Followed up from there, we'll look at building a roadmap for impact in just 30 minutes with Lexi. And then Jason is going to close us out today and how we can elevate our SQL productivity working within notebooks and within the SQL editor. And with that, I am super excited to present our next two gentlemen up here on stage. So, we've got Andre and he is a staff software engineer and he's going to be he's going to be presenting with a very special guest, Coy Mcnu, who's with Bridgestone. If you were with us this morning in our keynote sessions, you saw that that was highlighted as we talked about our data bricks apps. And they're going to share how data bricks apps is super easy to deploy, serverless, makes it easy to scale to all of your customers and build applications so that you don't have those headaches of infrastructure and having to manage all that. So with that, I would like to introduce the gentlemen to the stage. Andre and Koy, take it away. Thank you. Hello everyone. I'm super excited to be here. It's the second year that I presented. I am actually the founder engineer of data bricks apps. I built this thing. So if you have any complaints about the product, it's probably my fault. Um but um lot has happened since last year. Um and this time is all about AI. But where is the clicker? and we're going to talk about AI powered apps and how AI actually changed a little bit how we develop applications. So this is not a like 101 session for apps. I expect everyone to like know a little bit about apps. Is it like just who has not created an app yet in data bricks? Okay, that's actually a lot of people. That's interesting. Um so I'm going to so maybe I step back a little bit. uh data bricks now you can host applications we saw in the keynote you saw a demo how it is uh but I'm going to walk through like and I think there's like hands-on sessions all is like this data summit is pretty appheavy from what was like my only session last year um but in this session I want to show like what do I know like what do you guys what I want to show share with you from the the person that's actually building the the the platform and um and how AI is actually changing how we actually architecture and pl and deploy applications. So, uh let me let me start with a few numbers of how this is just to say that you guys shouldn't trust me. Um, so we talk about 20,000 apps created, but what's interesting is that there's actually more than 8,000 apps right now that are daily used in data bricks, which means that I have a lot of Kubernetes clusters that I need to maintain. And uh, the other thing is that it's more than 1500 daily active customers. So there's all like 1500 customers, not workspaces, accounts that are every day touching an application, which is pretty crazy. And that just show that apps are really democratizing data. And um like the way to share and build your own experience with your end users is through apps. So how do I do this? uh to but instead of you hearing me about like how do we I build apps I don't build apps I build a platform uh I want to bring to the stage Koi which is one of our first uh users of data bricks apps we started long ago and he built some really cool stuff on Bridgestone so hand applause to welcome thank you thank you uh Yeah, my name is Koi. I work on AI at Bridgestone. Um, and like Andre said, right, we we got started with data bricks apps quite a while ago, I guess exactly a year ago, um, when you first saw you present on it. And, um, we've been through a a bit of a journey on our app development. And I'll tell you a little bit about that now. Um, I guess before we get into that, a little bit about Bridgestone. Um we have about 180 uh manufacturing and R&D plants across the globe. Like we're a pretty global company. Um you can see some of our foot our footprint there on the map. Um I think you know what probably people don't realize is that we're involved uh you know fully end to end. So, I mean, we're talking growing and sourcing raw materials to um you know, doing R&D on designs to producing tires to selling tires. Um not just to automakers and retailers, but but you know, we also have um I I can't remember how many thousands of um you direct to consumer retail stores around the uh the country and world as well. You've probably seen the Firestone Complete Auto Care stores. Uh but we don't just make tires. That's what we're best known for. Uh we also make uh you've probably seen our golf balls. I think Tiger Woods plays a Bridgestone Ball now, which is pretty cool. Um we do some other fun stuff. We make shocks. We make uh the big seismic uh isolation things for for buildings, which is pretty cool. Uh and then we do some other fun stuff like um we're involved in the design of the tires for the Lunar Rover, for example. It's always a fun one to bring up. U but yeah, B, if you want some more information, visit our website. You know, there's open postings, all that kind of stuff. Uh I'll take you through now just what our journey has looked like. Um you know going from not building any apps at all to building um data bricks native apps, AI powered apps, whatever you want to call them. Um I'm going to describe this from the perspective of our data science team. That's where I've spent most of my time at Bridgestone. Uh and we started this journey maybe a year and a half ago, let's say. Um you know all the way on the left side of this figure, our data science team was making models. were making good models. We knew they were good because we could see the metrics. We knew they were doing a good job of predicting things. Uh what we didn't have was good stakeholder uh engagement, right? So we we were running our models. We had some predictions out there. We had some outputs out there and we didn't see a lot of people actually engaging with those and using those numbers. So it's not really a data science problem, right? It's like an engagement problem. So our solution or at least our potential solution was let's just put some simple you know user applications up there to to let the business stakeholders interact with those models and maybe we could drive engagement a bit. So we started really simple. I mean we just built some simple thin streamllet type interfaces. um you know maybe we just display a a figure of that model instead of instead of just outputting a table of of numbers and we let the user make some very simple selections right change some drop downs about about um um length of time or other parameters and regenerate the output and you know even with a really simple front end like that we started seeing tons more engagement with our models and it really seemed like it was doing uh what we wanted it to do um when we started we started like I said quite simple. We didn't really know what we were doing. Uh we originally started by just simply going to AWS, spinning up an EC2 instance, saying this is the instance for this app, deploying it on there, you know, sharing uh within network via via URL and and move on, right? Move on to the next one. Um that worked fine when we had one app, but you know, you can see we had 10 apps, 20 apps, 30 apps. Now we have all these instances. It gets really complicated. Who's managing all this compute? uh who's checking to make sure things are up uh you know who's doing all this orchestration. Um and our next solution from there was was actually a a talented data scientist on our team. Uh I saw him come in. Is Greg out here? Oh, there he is right there. Okay, I'll introduce you guys after the after the talk if you're curious. Uh a a talented data scientist on our team decided he's going to take it upon himself to to orchestrate a EKS cluster for us. Right. So now we have an EKS setup. um we are, you know, containerizing apps. We're putting them in some kind of um container registry. Uh he's shipping them out. He's managing nodes. He's managing all this. And so you might think, okay, well, there's the compute management challenge solved. But really, it just exported it, right? And it quickly became his full-time job, right? We have 30 apps or whatever. This is how he's spending every day instead of doing data science. Um so and then then we reached the point where okay now we have some some apps served. Um we have to get these out to business users. Uh we have to have some kind of authentication workflow. So what we did is actually just you know write our own sort of Python custom logic to go out to our organization SSO uh you know get the user to sign in get some success uh message back and then open up the app for the user. It really made the code a lot more complicated than it needed to be. And so then, you know, again, like I said, this is someone right there, his full-time job for a while. Um, so this was maybe we were at the state maybe a year ago. We come to uh this conference last year uh and we see Andre and Justin um present this thing called data bricks apps. And you know, from basically as soon as we uh saw what was going on, it seems too good to be true. This is the exact solution we need. Um so we quickly adopted it as you know um um as soon as we got back we got on some private previews we started building things and you know it it was pretty much right what it said on the right right what it said on the 10 it got it immediately alleviated our two main challenges we could serve an app compute's managed by data bicks uh authentication just piggybacks right off of the SSO that's already set up for data bricks so business users can come right in um and we can you know manage authentication that way. This person can access, this person can't. Um and it was great. So we started spinning them up. Uh like I said, it immediately alleviated our two biggest challenges, compute authentication. Um and it just led to a much faster time um iteration cycle on prototyping, right? So we were putting stuff out there. We were making some changes, seeing what stuck and what didn't. Um it really made that cycle super tight because hey spin up a new compute try an app it worked great keep it uh it didn't work doesn't really matter just destroy that thing and move on right uh and as I mentioned right it decreased maintenance so Greg could go back to being a real data scientist instead of a you know compute manager um and then from there we just started you know building in complexity right we had some of these that stuck uh some of them worked really well and some we got a lot of engagement on Um, this is also about the same time that everyone within our organization was clamoring for for Genai applications. So, Genai built into whatever business app they were using. Um, and so we started building on top of that. The if you saw the keynote this morning, you saw um Justin bring up one of our apps. Uh, that is a a sort of a general use Gen AI application that's used throughout the company. um we're talking you know I think 700 or so users at at at the moment 200 or so a day um and you know that started around this time right and started very simple now that data bricks has all the other inte all these other integrations we just kind of built those in one at a time as they made sense so you know now we look at our genai workflow like uh we we have some new idea we develop some sort of agentic backend in python we deploy it through uh um you know the agentic um uh framework then serve it as an endpoint within data bicks we you know connect that to some data source in data bricks say we build a vector store um we serve LLMs through the model serving in data bricks and then now we serve the front end in data bricks as well so then that way it's entirely native uh all those things are modular we can work on all those independently and our our iteration and improvement cycle is is is even tighter and more efficient Um, so I think about, you know, if I'm there's a lot of hands about people who haven't built apps. So if I'm sitting in the audience and I'm thinking about trying data bricks apps, um, what what would my advice be be to you? Uh, first and foremost, I would say uh, you know, similar to what we did, start really simple. So just make some apps. Um, surely you have some kind of um, model already served with some stakeholders you want to connect it to. Surely you have um you know some kind of data set you could connect and do EDA or analytics anything um just build some it's not that hard uh and you know some are going to stick some aren't get some feedback from from your stakeholders about what they like and what they don't uh and then the second point is what I just brought up which is iterate quickly so you know the framework they've set up makes it super easy to um to try stuff some works some doesn't keep the ones that are successes uh the ones that are failures discard them and move on. It's it's really great in that way. Uh and then finally, similar to how we built in, you know, Complexity's intro and workflow uh as it makes sense, you know, add on all these other integrations that data bricks now has so that you can, you know, harness um their data governance and their, you know, the the entirety uh the entirety of the platform and then your your own iteration development cycle will will um you know, copy ours and be a bit a bit quicker and a bit more um efficient. So yeah, with that I I didn't come to hear me speak. So with that, I'm going to hand him back over to the man himself here. Thanks Roy. It's amazing, right? Like and I I was in all hands for AI, mosaic AI two weeks ago and I told them it's like AI without application is just a math exercise because you don't have users. AI is very applied. So you have to apply what you do in your data science work and give them in front of the users and that's what apps does and without going through the checklist in infosac and it and everyone like you just create your app by yourself and that's what K did and it's very successful 200 users a day using his app. It's pretty cool. Uh so AI changed everything. Every app will be AI powered. So talking about AI power is going to become something like an oxymoron. Every app's going to be like that. Last year it changed everything but also for app development. Last year I handcoded an app live here. Guess what? That skill is useless right now. No one's going to handcode an app live. Uh because AI is writing the code for you. It's is actually very different right now. Um so let's talk about what does it mean? What does it mean to have AI in your application? What does it mean for the app development cycle? uh AI applications, data and AI applications are very special kind of app. If you think about like what app development any web developer if there's one here would see is like oh let's build a simple CRUD application. You go update create read update delete kind of thing. uh you have a client application, you have a client, you have some API servers, those API servers run your code. It's a logic and you have a database that's kind of like the basic application for you know a crowd app and that's how most of the frameworks are now if you just look outside that's what they do. uh what are the characteristics on this like request response is in milliseconds you saw about the LTP having a response in 100 milliseconds is not acceptable uh that makes it super simple because you can just go straight to the database return every everyone's happy uh and API servers are also deterministic you write the code once you test you deploy it you will work the same way unless you know you put some LM in it But that's simple. That's that's the way application development has been. But if you talk about data and AI and in the end you shove you put your telemetry in your application performance monitoring some data dog of the world and you're happy right? You monitor your application in production is everything good? Um data and AI apps is a little different. Uh and right now we're in that phase. So we is we just went G. So the journey for apps is just starting and uh so I'm going to kind of walk through what I'm learning through this journey and how I think uh the architecture of an AI apps is evolving. Clients is still there. Uh right now everyone's creating API servers. They're trying to follow the same design as the CRUD and go into a SQL warehouse for example. Well, I'm going to create my dashboard custom dashboard in streamllet. You write, use a SQL connector, go to the to the SQL warehouse, get some data, return. That works great for a little bit and then you realize that your SQL warehouse can take up to minutes to return on a query and then your proxy time out because there is a one minute limit of how much your request can stay there and then you have a hundred users doing the same thing. When you have a 100 users with a lot of long-term long requests all of a sudden your server start to get full because now you're keeping all that stuff while every user is there. So it becomes complicated but that's the reality right now in works and we have thousands of apps doing this uh and we haven't even talked about LM. And then you connect an LLM. Let's build a chatbot. Connect to LLM. You put a chatbot out there. Like Ali said today, everything works fine until you put in front of your customers and the chat the LM starts to spin up things that you should they shouldn't they shouldn't. Uh but so you you you think about like what else we have? We have guard rails for LLM. So we have AI gateway. We can we have all these tools in data bricks. We have authorization and authentication. So we are here. So right now uh this is kind of like the majority of data bricks apps are like this architecture is what they're following uh so but again let's remember some of the requests will turn will be minutes and uh and then once let's start to evolve so if you start to evolve this application what I'm seeing the most advanced ones they're starting to instead of like having request response they're starting to stream the response back. So it becomes like web sockets or server side events and all that. So you're moving your data, you you send the request to the database, the client forgets about it and the app API server just returns when it's ready. That solves some of the problems. Uh so that's stream or a sync. Uh and then we're starting to see a lot of agents coming up. So if you're connecting your app to an agent, an agent has its own the idea about agents is that autonomous actually make decisions for you and do stuff. Uh so you have to think about like how to connect the lamps that you have in a chat or something with an agent and see warehouse and connect all of this together and things that can take minutes each one of them, right? And like I don't know if you guys know but like there's a thing called grace for shutdown servers. Servers have to be updated every so often which means that if you have something that's running for minutes it will likely be turned off because the server is going to have to be updated and it does because compliance requires the service to be updated every two weeks at least. So now we have instead of API servers I'm seeing workflow servers coming coming up. So you you actually design your back end in a way that is workflowful activity like um but now what happened is still kind of a very stateful server. You're keeping stuff in memory uh comes a database that's what lake bay is going to bring us uh and that's the key now we have the all the tools to actually explore explore the potential of the of apps in data bricks before that didn't exist. So now you can save the workflow state and recover if your server restarts. So now you have a much much better architecture. Uh and of course you can save user state. You can actually come back and you can restore that. All good. Um but that is still kind of generic. Um, and I I what I'm excited about building an application platform inside data bricks is because when it comes to stochastic nondeterministic behavior which agents and LLMs have uh you need to have your the data that the app generates very close to the rest of the data that it uses and you need to have the web developers, the data scientists, the data engineers together in the same framework. If they're all working together, we've seen this happen in some customers that are more advanced. It becomes like this flow of improvement of the app becomes very clear. So what happens what I want? So this is like what I'm building for right uh I want telemetry of the apps to be your data where in the lakehouse that way you can you can actually join your app telemetry all the what the agents are doing what the LM doing what the API servers are doing the workflows with the rest of your data in in data bricks in SQ warehouse uh and of course lake base has a sync from the lake base to the lakehouse as well. So now you have you can connect all the data in your database with the telemetry data and you can actually do some pretty cool analysis of how things are running. So that's kind of the beginning of actually having a critical mission critical uh AI powered application and that's that's without being in data bricks. If you try to build this in AWS on Azure man it's going to take a while. I've done that before and it's not fun. Uh because this is like 20 services in AWS right here. Um and then what is the favorite of everyone in data bricks is that uh you have to process this data to feed it back into the development of the app your prompt evolution what you have with the LMS and all this. So you have to have data pipelines to process this all together. And now you see I'm looking at the an app architecture that has data engineer in it. And that's the difference between just a normal crowd application and AI powered applications. So if you if you look side by side is a bit more complicated to run proper production ready data applications. Um, but data bricks now has all the tools that you need. What what I need to do now is make it so much easier for you to follow the best practices and do this easier. So that's my job for now working with you to improve this. Um, so now talking about database, no there is apps and like base is literally just one click. You say create new app add resource database you get a database for you you get the Postgress connection string and you can just use any Postgress client that you have just normal so that now you can save app metadata you can say AI Asian state you can save some operational data you can go to town on this uh it's actually quite amazing like how powerful this becomes um another thing that no one talked about but I want to bring it is that I was just playing around with is um Postgress has notifications and has specific locks which means that you can actually have a queue in Postgress and once you have cues then you have a lot of kind of pretty much cover everything that you need in an application. Uh so I I want to put some demos on that later on. I was just thinking about this. Uh the other thing is that um we really like Justin said this morning we really embrace an open ecosystem. The first time I got a I talked to a PM in data bricks like oh let's build this and I was proposing some ideas about apps and uh the PM say oh we need to support streaml and what not and it's like dude we're going to support anything why do I care it's your code you run whatever you want uh and it turned out to be true because damn like every day there's a new framework AI is really changing things and uh who is going to win I don't know but you guys should be able to try it out um and uh you know I just saw some of this like I collect these ones that I know and use but um they all good and then what the other thing that's happening is that all the AI generation of code are focusing first on front-end code you go to Lavable creates a pretty website for you which is great because static ass generating static assets with AI is way safer than generating your back with AI. So let's start with static assets. Uh but they all generated in Typescript. So now we're supporting NodeJS as well, but it's not Python or NodeJS. The runtime of data bricks apps is actually both. So you can create a app that have the front end in OJS and the back end in Python. It's up to you. You can use a back end in OJS too. You can go crazy and create a front end in Python and the back end in NodeJS. But um but it's it's pretty cool because now you have your application together. we build the front end, serve it in your node in your with in your um fast API or something like that. And now you start to have um a more know production ready application um because you know testing load test doing load tests and API tests and all that you know this is kind of like what everyone is doing every web developer is this and also you can bring the web developers in your organization to databicks apps they're not going to complain anymore oh do I need to write streamlit app um no they can write react and vit and spelt whatever they want to do angular anything. So this is this is pretty cool. Um that's this is what I'm say. And now uh what that's this means too is that we're working super hard to get um a an ecosystem built. So imagine you can go to you know a posit workbench create your shiny application and say deploying data bricks apps. So the key of apps is not necessarily how easy it is for you to develop and write code. It is the security and governance of bringing your app together with your data. So your app closer to the data, not moving your data out of data bricks is the key of this whole thing. So we don't care necessarily where you write code. Especially when you create an application, we actually instruct you to use your own IDE because we're focused on developers and you should not like have to write a notebook to create an app. You do what you want. Um so that's the idea. So on before data bricks I was as a startup and um we were building some uh revenue intelligence thing and uh like the least amount of time was actually developing the app. The majority of the time was actually working with the data that was powering the app and AI is only going to make this this more difficult. Uh but I think what's interesting and why I truly believe the data bricks being an application platform is going to change the game. It's because the hardest part is the data we're bring like there was just a we're bringing the easiest part inside but keeping all the tools and everything together is going to change is going to be really transformative I believe. Um so now all you users can use data bricks and you can share what you did the cool stuff that you did with your users through apps. Um, so on that I have three little things I want to show to you. Uh, these guys create some pretty cool demos. Um, and I'm going to see if I can make this work. So Mike deployed a a full-blown workflow that does Slack agent whatever in 88N as a data bricks app. No need to change anything. He's just running this whole thing as a data bricks app. Uh you can see there's quite a few of the workflows. It's the whole thing. It's not just a fake. Uh and that's what it means to be open. You have a you have your framework of choice. You can deploy that in data bricks and use it. Um most of these frameworks use postgress under the hood for state. So now we have like this. So it just works. Uh the second thing second one uh n oh forgot got the wrong name here. Uh n uh create a healthcare genie. So I was talking to many um many customers they love Genie they really want to use it but it's very hard to put the genie the way it is in the workspace in front of all users in the in the organization so what they're doing is in creating you know create your own interface on top of Genie Genie has a conversation API so now you can create this that you can actually enhance what you do with Genie so you can have things like you can have a healthare care analytics platform, deploy this to your users and have things that are specific to your use case and how your users should interact with that data. So you can have a different UI on top of Genie. This is like literally just what it did. Uh I don't think this had anything to do with healthcare, but that's there. Uh and then Ne uh created like not just he went and then we went one step above and this is like the art of the possible. Um we're this is an application Casal and it's going to be in the we're one thing I forgot to mention is that we're going to put we're going to have apps in the marketplace. So a lot of the apps that we create that we're showing is going to be installable by all of you. just go to the marketplace, install it, and test it out. All open source. Uh, but this thing here, what it does is uh is actually an agent creation app. You use crew.ai and you have agents. You create your tasks, your agents. You see how they're identified. You define your tools that you want to use. There is like hundreds of at least dozens of tools that you can have. Um and you can do like you can run and it gives you the results. So for example, this one genie inquir inquire uh what is what is it? I created an agent that checks what questions I can ask for genie. Simple stuff and returns the list of questions. That's easy. Um but then I created an ad designer too that no it's just like I just ask like create a a designer that does this this and that. And in the end I finalized and added a tool. And this tool uses Dolly to generate an image. And within here it generated this this rock festival u flyer the same way that I expected. Um so I think what's what's cool about this is that is like an application that you have. You can see your law your traces from within the application. You can check your your result. You can chat with it. You can have full-blown configuration. You can def you can integrate with MCP servers. You can add IP API keys for any any tool that you have. You can add custom tools or pre-built tools based on crew AI. You can have prompt management. You can dude like this is this application is really really something. Um but now you have a way to define your own way to create agents and expose those agents not through this to UI but through another application using the data bricks apps. So the bottom line here is that um there is what I say is like if you have a problem there's likely an app for it. you can develop like you're free to go and fix your own problem instead of calling data bricks and say can you add this to the UI uh or can you and like it goes further because now uh admins can also love creating apps to like govern stuff in data bricks but uh but this is like this is this is getting very very interesting um and what else with at um we only have four minutes. I wanted to see if you guys have any questions or if anything that you want to know. Yes. Yeah. Well, thank you. I will be outside if you guys want to chat.