Postgres is Now a Vector Database, Too

Key Takeaways

[Music] you're watching the new stack makers a podcast for people who develop deploy and manage at scale software for more information and articles about at scale Technologies please visit the new stack. now enjoy the show since its Inception Amazon web services AWS has been the best place for customers to build and run open source software in the cloud AWS is proud to support open source projects foundations and partners hey everyone I am here today with s Shandra shikon and shes is a general manager at Amazon web services and you're general manager for the Amazon relational datab database Service Group mhm and today we're going to talk a little bit about open source databases generative AI but first I'd like to yes yes understand why you decide to get a degree in philosophy it's actually not a degree in philosophy it's a PhD so I'm a doctorate but my degrees in computer science and in databases as it turns out so your PhD yes is is in computer science and specifically in databases and your and your Philosophy degree is I don't have a Philosophy degree it's just PhD doctor of philosophy but it's it's a PhD PhD doctor philosophy oh well I've been I've been thinking of you as a philosopher all this time but you I'm I'm sure you have philosophies about relational databases so why don't we talk about your philosophies about relational databases what are your what's your current philosophy now how do you think that's changed in the past five or 10 years the way I think about databases and I've been at Amazon you know little north of 7 years I have a much deeper appreciation for databases than I even did during my PhD days and it's a little strange saying that but I'm truly humbled by the number of customers who essentially bet their businesses bet their organizations on relational databases and how big databases are like it's it's in some ways at the lower end of the stack they're a huge part of what drives the world and so the value that relational database is bring and it's you know 50 60 years since Cod and others at IBM first came up with this concept it's truly what covers the world what was your thinking about relational databases when you were a student and so and and tell me kind of like your reflection on it now what what are some of the observations you you have when I was a student at relational databases but 25 30 years in and a lot of the Innovation at that time was more I'd say towards the analytic site and that that was a little bit of what I did my my PhD is around streaming databases and what's really been sort of astounding to me and you know in a good way is that 50 60 years into databases now it Still Remains a heartb of innovation there's a ton going on Amazon era for example which is a cloud native database that Amazon offers really rethought the storage layer and how some of the uh technology works and it's transformed how people think about open source databases um a lot of things that previously were only done through closed Source we now have customers saying hey open source Technologies are really caught up and we can bet our businesses on open source relation databases when you're thinking about databases now I'm thinking about your work on the Maria Maria DB Foundation but before we do that maybe you can tell us a little bit about your kind of your overall responsibilities and as general manager like what are what are are what are the relational databases that Amazon offers now I mean it's it's grown yeah it has so one way to think about Amazon you know a framing way to think about Amazon's uh Services is that we want to offer customers choice and so aot of what we offer you know I mentioned in myo in the context of influx when customers want a managed service on top of a database they come to us we hear customers and we work back from customers so in databases itself I'm responsible for relational but our portfolio is much larger we also offer a bunch of purposeful databases time series graph you know and so on within relational databases Amazon relational databases and Amazon ER those are umbrella Brands we offer postest MySQL Maria DB Oracle SQL server and our newest engine is IBM db2 and Aura which is our Cloud native database that I mentioned has two flavors one is a postest flavor it's post compatible there's also a MySQL version of it tell us a little bit about what you discussed in the kyot today and you were showing kind of the you know you have some interesting slides there maybe you could just kind of encapsulate them yeah so look the key messages from my keynote were we are increasingly increasing our in m in open source and we've always invested in open source based on what customers have asked us early on in aw's Journey what customers were asking us was we like these open source databases but we find them hard to operate in the cloud so big part of our investment was towards manageability and that's where IDs came to be and some of these other manage services more recently customers are very comfortable running in the cloud they love IDs and what they' said is we want to improve the core engine can you help us with this it started with something very simple like bug fixes security patches but increasingly they're also asking us for features because the workloads need to run in some ways and so in the last 2 3 four years we have significantly increased our investment into the core open source Community where we're working directly with open source in terms of improving databases and also helping the community Thrive tell me then about you know where relational databases are in your viw right now because we're hearing so much about generative AI right and relation database that you said have been around since you were a computer science student and before right and so what's your view now and how does it fit with the story that we're hearing so much about generative AI you know in Amazon we have saying it's always day one and I still believe it's always day one even for relational databases as I mentioned a few minutes ago you know I think of relational databases as being at the heart of a company an Enterprise's data and when you think about generative AI for a company truly to have unique differentiating value proposition the generative AI has to be based on the data they've already accumulated from their customers from their business and so you may have heard of this concept called retrieval augmented generation and what it really does is it allows you to take in a generic llm yeah and customize the results to your business based on the data you have right so what we are hear hearing from customers is that rather than have a net new Vector database they want us to bring generative Ai and Vector capabilities into existing dat bases so they want to bring it into like a relational datab because they have already invested in these open source databases they already have applications so not put the database in the Genera of AI but the opposite a little bit right and so the way we're working with the open source Community but also within Amazon is uh I mentioned PG Vector in my keynote PG Vector brings Vector capabilities into postest and right now the Maria DB Foundation is also working on bringing the capabilities into Maria and then another big part of this is you know it doesn't start stop at databases you have to get from databases through to analytics to your tools U on the ml site and so we're working on zero ETL strategies which allow Enterprises to bring all the data together so that they don't live in silos so as a customer you don't have to look at new technology you get a new existing technology and you we break down the silos for you so tell me about the role of PG Vector then can you tell us a little bit about what is PG vector yeah how long has it been around and how does it serve as being relevant yeah so PG Vector is an extension in post dress so think of extensions as plugins you can bring your own extension and you can significantly alter the capabilities of poststress that's one of the strengths of postgress so what PG Vector does is it allows you to store Vector types in postgress and it also does something called similarity search so one of the things about generative AI is you're not looking for exact results right that's not what an llm does but you're looking for knowledge that is close and the concept of similarity search approximate nearest neighbor is what PG Vector does really well it's been developed by this fantastic developer called Andrew Kane he's been added a few years okay and we were the first Cloud vender to partner with him and offer it it's it's always been open source we offered it in Aurora and rs early last year and one of the things that we've been partnering with Andrew is customers love this because like I said they want generative AI in postgress we're getting a lot of feedback on ads Improvement and that's really been the journey for us is working with Andrew on new indexing types improving performance directly contributing code uh it's been a fantastic Journey you mentioned a few things in PG Vector such as indexing what does PG Vector have and what are the some of the some of the different features you would like to see in PG Vector itself so when we started working with Andrew and when we first implemented PG Vector in rur AAS it had an indexing method called IVF flat which is used link lists and was fantastic for building indexes quickly what it wasn't as good at which our customers were asking for was quy throughput especially when you're looking for higher rates of recall and also iterative development where you're able to insert new rows on the fight so we work with Andrew and we implemented something called hnsw which uses a different data structure it's a graph based data structure which customers love and a big part of the work has been to make hns stute as performant as IVF flat on some of the areas of deficit and so the work going forward is going to be continuing you know it's a fast evolving space so there's a lot we don't know and so we're just keeping a close sort of ear to the ground on what customers are asking for but really it's about scale performance some of the usual things you'd ask look for in a database but in the context of vectors now the graph the graph based approach is what you see customers liking yeah you get better query throughput when you're looking for high rates of recall and you can also more iteratively add add to the database which you'd expect to see in an operational database cuz you know you're constantly running transactions and changing the underlying data because it's a graph um the graph has to do a little bit more with making it simpler to look for nearest neighbors that's why it works better that's why the nearest neighbors thing is a big part of generative AI exactly and Vector search and semar search so it allows you then to build that capability the throughput issue tell me what are some of the throughput issues that customers face it's like any other database you have an online application you have customers coming to your store coming to your website and you don't want to experience latency now you want relevant answers based on your own like Rag and so a lot of it is making sure that the database doesn't take a hit or any significant head when you're running a vector based cuz all the things that you'd want in your application you still want right you know all the asset properties all the performance properties and you don't want to lose any of that generative AI is a different Beast entirely what is it make it reflective of what you said when you say that there's a lot that we don't know you know we learn a lot from our customers right that's been one of our finding qualities and the rate of innovation that we're seeing in the generative AI space frankly like from my experience it reminds me of the doom boom like the the late '90s like the internet's new the world possibilities are endless and so I expect you know the next 2 3 5 years folks are going to come up with new use cases new ways of using this technology which is frankly amazing that we can't imagine right now right and so our goal is to stay humble stay Nimble listen to our customers and keep innovating as as they ask us to do more keep innovating okay so what are some of the things that you are finding that should be in PG Vector for example or that customers are asking for that you'd like to build in you know as I said hnsw is a big lift yeah and going forward it is going to be around continuing to improve performance continuing to improve scale you've had customers you know with hundreds of billions of objects that they want to index hundreds of billions of data points that they want to index with vectors and some of these s of stress the limits of postest itself and so a lot of the work will be both PG Vector but also in core postest and so we'll continue to innovate but PG vector and post are just one part of it you know we have other databases people want Vector capabilities is there as well so we're working with the Marb Foundation um and the other part of it really is Simplicity customers just like with RS uh when we started they want to focus on their applications they don't want to worry about the tool chain they don't want to worry about generative AI as a tool for them to achieve what they want for their business so integrating with bedrock knowledge bases a lot of the other things that Amazon is doing around generative AI offering customers an end solution so dimensions in database are defined by the llm and the LM Ms continue to become more complex and larger in parameters and everything else so that has an impact on databases doesn't it it does so the way to think about it is llms are really encoding knowledge in a higher Dimension space and if you have more Dimensions you have to store more data and you can in a very tactical way breach the boundaries of a single page so one of the things that postris has to do also is learn how to deal with higher Dimensions um and have indexing performance a high level that's interesting point then let's then turn to Maria DB because you're on the foundation there and like it's a good example of a database that has to transform with the time too what are some of the Transformations that we're seeing with Maria DB you know with Maria DB adoption and continuity is the main focus on the board because we the corporation has historically driven most of the contributions to MB we're now the number three contributor behind the corporation and the foundation and a big part of our focus is to ensure that the Marb ecosystem is very healthy so our core Focus Still Remains contribute code make sure that it's a vibrant ecosystem the other part of it is things like gen Marb currently does not have Vector support so it's something that many of the foundation players Corporation are coming together to make sure that we can add something like that intermediate DB so it stays relevant in the gener generative AI space do you think vectors will eventually just become a feature that we'll see in databases I I think that's already happening yeah a a lot of customers as I mentioned are looking for vectors to be a core part they don't want to change the application rewrite it but they want to be able to augment the application to to use vectors think about it this way if if if you had to go use a completely new technology to get advantage of generative AI there's a learning curve you have to move your data and now you may have data in two different places customers don't want to do that they want the data where they already have it they want the silos to break down and they want generative AI capabilities in the infrastructure that they already understand so then maybe you could provide your context about Vector databases from AWS I'm trying to remember now what that is from you all so we offer Vector capabilities in a number for databases postest is one example open search is another and a lot of what you'll see from us is what customers asking give vectors in the databases that you have a second push for us is what we call zero ETL to make it easy for data to flow from transactional systems into the data Lake into analytical systems so that customers get generative AI capabilities wherever they want it and one thing that I find interesting concerns the capability for data scientists to work more closely with developers and infrastructure teams which comes down to in some respects what you're saying about factors doesn't it it does and it's really also about breaking down these silos historically you know relational databases were dba's concerns there were data scientists that worked more on the analytic or spark side and you're going to see a convergence of this I I think you will and so a big part of this is making sure that you take away the infrastructure friction so again customers can do what they do best so relational databases are they still here to stay I think so how come like I said you know in the end what relational databases do best going back to the roots of relational is assd atomicity consistency isolation durability those requirements are never going back if you have a banking application or anything else you know you run a store you don't want to lose transactions relational databases have been the foundation a very successful Foundation of our economy and I expect that to go forward but couldn't an LM start to offer those same features you know I the way I think of llms is I I do think llms have an important role the way we interact with relational databases can change you know increasingly you're seeing text to SQL you know where you ask a question and generate SQL but the foundational technology underneath the covers the thing that makes sure that your data is never lost right think of a banking application you withdraw from an ATM or you deposit you never want that deposit to be lost those in the end are still at the core of what relationship databases do best right Ser thank you so much for your time thank you as well if you like this video please give us a thumbs up and if you'd like to see more videos like this you can always subscribe to our YouTube channel we're on all the major social media platforms forms you can always find us at the new stack. we hope to see you soon [Music]