Digging Into Graph Theory in Python With David Amos | Real Python Podcast #212

Key Takeaways

welcome to the real python podcast this is episode 212 have you wondered about graph Theory and how to start exploring it in Python what resources and python libraries can you use to experiment and learn more this week on the show former co-host David Amos returns to talk about what he's been up to and share his knowledge about graph theory in Python David started a PhD program studying mathematics with a focus on graph Theory though life Interrupted his Pursuit after three years he is still passionate about the subject he's been using these skills to create documentation and teach users as part of relational AI Education team David has also been exploring the Julia programming language he wrote about it on his blog created videos and started a podcast on the topic he shares his excitement about learning new techniques from different languages and how these ideas can enhance your coding in Python this episode is sponsored by Sentry Sentry provides endtoend distributed tracing enabling developers to identify and debug performance issues and errors across their systems and services all right let's get started [Music] the real python podcast is a weekly conversation about using python in the real world my name is Christopher Bailey your host each week we feature interviews with experts in the community and discussions about the topics articles and courses found at real python. after the podcast join us and learn real world python skills with the community of experts at real python. hey David and welcome back yeah thanks it's been a little while yeah yeah it's fantastic to have you on the show again well I just want to start right here like what have you been up to what you what have you been doing yeah I've been working the last couple years at a company called relational AI gotten all back into the graph Theory stuff I know maybe some long-term listeners out there might remember I have an interest in in graph Theory yeah yeah that's been yeah just a blast to be able to get back into that world and be thinking about those kinds of things yeah been doing that I've also my my kids have been getting older so I've been doing a lot more stuff with with the kiddos and they're involved in a lot more activities and everything my daughter's competition swimmer and we've been doing that whole deal we were in Mexico last week actually yeah how did that trip go we talked offline before this yeah so she was there for What's called the CC can CCC an I forget exactly what it stands for but it's Central American and Caribbean Nations come together and have a big swimming swim meet okay and compete against each other and she was there representing El Salvador which is so her mom is from El Salvador and she's got dual citizenship and it was it was great uh except that it got interrupted by tropical storm or I don't know did it become a hurricane Alberto oh okay and so they had to kind of pause the the meat and and kind of do some rearranging of stuff and everything but but yeah overall is good we had a great time hanging out in Monterey and yeah yeah cool that sounds fun so I wanted to talk a little bit about you you kind of did a bit of a programming language journey in between here and I remember that as you were kind of exiting real python there you were interested in checking out Julia and it seems like you did quite a deep dive there so maybe you want to talk about maybe you know you what is Julia I know that some people might know that it's part of the name of the Jupiter notebook there the the ju is uh from that and then I guess maybe we start there yeah so maybe talk about your journey into Julia and what you've been doing with that yeah so that's yeah that's a that's a great kind of story it's it's interesting I've been kind of curious about Julia for a while you know I think even while was working at real python i' you know i' heard about Julia and I had a friend that was using Julia quite a bit and he kept bugging me like you got to check this language out you got to check this language out and I just didn't have time really to yeah yeah to try to learn a new language at that point and so finally he he convinced me like you know just just take a look at this and you know let me know what you think and started digging into it and it it's a really cool language first of all it's it's fast it's really fast but I found it so easy to pick up and I think you know part of is they try to I think their tagline is something like Walks Like python runs like C or something along those lines okay although I would I will say you know the syntax I found I was drawing more on like my mat lab experience from way back in the day oh really yeah I was Finding like trying to like anticipate because I you know I was learning and I wasn't probably using all the resources like I should I just kind of like exploring the language and trying to like I wonder if if this works whatever oh okay no that's a syntax eror like what would it be you know trying to kind of anticipate things from a python mindset didn't work as well as trying to anticipate things from like a mat lab mindset but there were a lot of you know similarities it's a really cool language the the thing that I think got me excited about it the most was not really the speed although that is cool and you know python has plenty of ways to sort of get more speed out of it and compile to see you like the whole number stuff and everything we talk about the rustifer right of of python yeah yeah and so what I found really cool about Julia was it's literally Julia all the way down until you get to the the compilation layer which is L llbm I think okay but you know you go into like you know compared to something like py on you go into like numpy let's say you're you find a bug in numpy which is probably not super common for like most people to find a bug there but yeah let's say it happens right and you you just want to try to understand what's going on you may end up in a world where you have to understand you know something other than python to be able to understand what's going on yeah Julia didn't really have that problem you could just every layer you would go into was like oh it's more Julia more Julia and so there was a sense of like oh I can I can really understand things deeply okay and I thought that was really cool They al also their onboarding experience their first-time experience I found to be really smooth just the tooling around getting Julia installed and the to the tooling around like managing your they don't really have environments the same way python has like virtual environments but just the way projects are managed and all that kind of stuff is a lot more intuitive do you know the age of the language like how long it's been around roughly a decade a little more than a decade I think they celebrated their 10 years okay so it's more modern yeah yeah exactly yeah no I mean and if you ask Julia developers they'll tell you you know they've benefited from yeah you know most of them are developers in other languages besid Julia they come from some of them come from python some of them come from mat lab and c and C++ and thing and so they'll tell you you know they've relied heavily on drawing from the experiences of other languages you know what I mean it's like yeah that's partly why I want to talk about it on the show because like that is a common theme that I hear from you know definitely like the core devs people like Brett Cannon and wuk longa and you know people like that like I've talked to where they're looking at other languages and they're kind of like you know deciding as we migrate and go further into the language like what what do we want our language to be able to do and and to approach and where do you see Julia being used is it you mentioned the mat lab kind of thing so that makes me always think of like universities that's like such a common place that you see those sort of installations of it but where where else do you see Julia being used yeah so I mean it's definitely the at least you know when I was more actively thinking or trying to keep up with where it's being used and everything I think maybe it's changed a little bit but the the biggest group of folks using Julio were definitely like in the academic space you know and in science yeah and it was in a lot of areas where I saw a lot of stuff like applying machine learning in science and stuff so those kinds of things yeah I would say that's probably the biggest group of folks that are using Juliet that said there's also a Julia web framework called Genie okay which is I wouldn't say it's analogous to D Jango or even flask it's I'm not sure the best way okay I'm also not like an expert on it maybe kind of like a stream lit sort of thing but more more powerful like apis and things like that or like kind of uh yeah it allows you to build like dashboards and stuff really well but it can do a lot more than that I know it's a much more full-featured web framework than just something like streamlet but kind of targeted towards you know you've got some data you need to display and you want to have some interactivity and those kinds of things and yeah you know that kind of stuff also I I hope if anyone from Genie is happens to listen to this and I'm doing a terrible job of describing it I'm sorry but that's the way I understood it at least like a year ago yeah yeah yeah how's the packaging in like the other libraries and stuff like that I would guess it probably wouldn't be quite as diverse as what's happening with python not yet but there was massive V velocity I saw a lot of velocity of you know new packages coming out and that kind of stuff so definitely you know the ecosystem is trying to to get there and you know honestly I think for numerical stuff I my takeaway was I would almost prefer to work in Julia for most numerical stuff that I was kind of interested in than than python but getting beyond that it kind of felt like there's just not not quite the ecosystem there yet to to tackle the stuff or like the maturity you know I me python the other like big thing that python has is just the maturity now the ecosystem has been around for a long time and and everything so that's a hard thing to beat but yeah cool well and I think that that audience you know they've been really receptive I saw a ton of people coming from R yeah yeah coming into to Julia right we talk about the cross-pollination there from Python and R yeah that's interesting too yeah my feeling was I I might have been I might have seen more people that were coming from an R back background than python going into Julia but there were definitely people coming from both both backgrounds regardless the other thing that Julia offers is pretty good interop between Python and stuff so you can actually call python from inside of Julia and so I was wondering about that yeah yeah and so I don't know it it's definitely something I think if you do a lot of numerical stuff it's worth considering like just is it yeah does it have a place somewhere in your in your pipeline yeah I wondered about that like I you I've said this many times on the show how I when I got into python was just starting a job it was a kind of a cross shop it was R and Python and so very often they would come up to me and say would you like to do this in R you because that's what it already exists in I'd be like sure you know it kind of made sense because it was like already kind of like a project that was there but that was probably the hardest thing for me at the time was like oh I know that this thing you know like this particular tool I know what it does in Python and I if I was going to try to run an R it was it was some work it was it wasn't an easy task in that sense so it's good to hear that because I think in the data science world or even mathematical different areas there is this sort of Crossing between boundaries often and having to work with other people and collaborate but it's interesting yeah because I yeah I try not to be too religious even though it is the real python podcast I hope people can forgive us to talk about another language but I I always feel like there's lots you can learn about languages and about ecosystems and like you know stuff to pull from there yeah for sure no and I would say that for me having been so deeply steeped in Python for you know so many years it was actually it was really refreshing and really beneficial even as a python developer to Branch out and just learn a different language see some different ideas and everything and I think it impacted some of the way that I think about and write python I I don't know like specifics or whatever but you know it just like anything else it just opens your mind to new ideas and things that then when you're solving problems in other languages it's it's a new mindset a new way to look at things new paradig and things like exactly you find a new solution that you hadn't thought of before so yeah so you you did a podcast for a little while did talk Julia and I found that really interesting because it I think that helped you maybe land the job I'm not sure like how that kind of got connected together it connected me to relation yeah so definitely need to bring up the podcast so that was kind of my way of learning Julia was yeah yeah yeah it's a great way to learn I definitely do it here and the co-host on me with that Randy daa so he I I've known Randy since College we met both of us graduated from the University of Houston downtown so he was the one that was bugging me like for years like you got to check this language out you got to check it out ah okay and so he was kind of ahead of me you know knew knew a little bit more about it and everything and I was just just kind of you know trying to learn it as we were doing the the podcast and everything yeah that was a lot of fun a lot of work mainly because we decided to do the video route and yeah the video editing was just so much work yeah I haven't quite done that yet that ultimately I think well that in a combination of then getting the job at relational AI kind of you know led to the the end of of the podcast and we're not doing that anymore but kind of that so that story there I don't think it got me the job but I don't know that I would put it that way but it connected me with the people at at relation AI so one of the yeah early guests actually I think he was the very first guest we ever had on the podcast guy named Logan Kilpatrick was kind of like the community manager or I'm not sure what exactly his title was or whatever for the Julia language so he was working with the group you know that develops language and just kind of managing social media and the discuss forums and you know that kind of stuff yeah we had him on to talk about the language and stuff like that and and a couple weeks after that he invited me to a Twitter space that's what they were called right spaces yeah yeah yeah do they still have those I don't this is at the time they were doing Clubhouse and they were kind of making a competitor of it yeah exactly so this would have been 2022 maybe or 20 yeah I think 2022 sounds about right yeah so he invited me to space they were talking about getting a job using Julia and I mean I wasn't interested in a job as a Julia developer but as you know the host of a podcast it was like I should probably go and maybe there's some companies you know to invite and stuff like that so we go and uh moham the CEO of relation I was one of the guest speakers and he spoke about what they were doing and if I'm completely honest at the time I wasn't 100% sure what they were doing but you've learned a lot more since then well yeah but he kept mentioning graphs and I was like I've got to go you know figure figure this out so I went to their website and kind of got some more information and everything and it was like this looks really interesting kind of up my alley so I applied and and now here I am two years later and uh it's been yeah a really cool journey in fact it's brought a lot of things together so there's Julia involved in relational AI yeah however our our product the the interface for our product is all in Python so you know I'm kind of brought both of those worlds uh together in a way although my you know day in and day out I don't really work with Julia much at all but yeah so there's some cool stuff to talk about there and then also just the the graph and graph theory side of it so it it was like I don't know it just felt like man this is it's a cool fit really cool fit yeah and you've been doing a lot of writing too right which is you know kind of your background from real P so my role yeah so I'm a a technical writer is my official position at relational Ai and currently I'm kind of managing all the documentation not all of it and yeah lots of writing yeah cool yeah yeah definitely Sentry provides endtoend distributed tracing enabling developers to identify and debug performance issues and errors across their systems and services their tracing offering focuses on proactive debugging rather than passive observability it introduces a span-based tracing model allowing developers to track the complete endtoend path of requests pinpointing issues across distributed systems and microservices with the trace Explorer users can search specific traces using High cardinality data and debug efficiently with full context in the redesigned Trace view which provides detailed visibility of application requests and metrics this approach helps identify and resolve performance bottlenecks quickly installing Sentry is simple like five lines of code simple use the code real python during sign up and new users get three free months of the team plan so when your code breaks you can fix it [Music] fast I've wanted to talk to somebody about graph theory for a long time and you had mentioned it and brought it up I don't know how many times back when we were doing the show together and pulling up articles you're like oh I'm you know fascinated in this and and so I was like okay that's David's thing he's goingon to go to town and and and so forth and so I didn't really dig much into it and so I think most people know that I did not go to school for python or for you know programming like my school days were a long time ago and I was at the time you know looking at like electrical engineering and so I was touching on Fortran and c and and so forth so I learned some fundamental concepts but I didn't really a lot of these things are newish comparatively and so maybe the first question is did you study graph theory in in school at all I did yeah so I almost got a PhD in graph Theory okay see you're the right person yeah I left or say in graph Theory right I mean it would have been in mathematics but I mean I was focusing on you using graphs and and solving problems and stuff so I did not finish my PhD I ended up leaving to go into industry I had a family and just PhD stifen was not not cutting it and uh I had to it's it's hard yeah yeah had to make a decision and anyway it worked out great for me so in the long run yeah so I studied graphs I have a lot of background academically thinking about graphs and everything yeah this is a you know a different situation where now we're trying to apply graphs to business problems yeah but the people that are running into those problems and tasked with solving them know nothing about graphs or or or really have a limited knowledge about graphs so you're explaining them a lot and through the documentation that helps I guess too so uh not quite yet um our docs don't have a lot of explanation we're finding that to be the case so we may end up you know having to do a lot more teaching of that but there is kind of this gulf that exists in this world of people have been talking about using graphs for quite some time we have like graph databases and everything right and some of them are really great and they've you know had a lot of success but one of the big issues that you constantly run into is people I've got this data I see this cool stuff that people want do a graphs I just cannot figure out like how do how do I do that with my data what you know and and the issue is that graphs are an amazing tool that help you solve countless problems like it's such a general framework for representing a problem and then you know doing algorithms on something the issue is to get any real benefit from it you have to really have a good understanding not only of the domain of the problem but also the graph background so you need sort of like a domain expert who's also a graph expert which is like a really tough combination to find there are people out there like that yeah but really you want some kind of you know you'd hope that you know you could make graphs approachable and everything and so yeah that's still kind of an open question I think like we're we're working on tools to help make that easier for business folks to approach those problems right now we have more of a low-level kind of API to um to graph tasks and everything but um but yeah just to I guess maybe to kind of back up a little bit and just explain like what is a graph maybe some people I think if you're a programmer you may have at least if you've been doing it long enough you may have some notion of what a graph is because it's a data structure that eventually I think comes up if you do certain kinds of tasks in programming yeah but it's it's just a general idea I think you know that's one of one of the things that makes graphs a little bit challenging is like there's just this General notion of I have some set of objects and those objects have a relationship with each other yeah and so visually you just think of drawing a bunch of dots on a page as these are all the objects that you're kind of considering and the Rel if there's a relationship between two of them you draw a a line and connect them um and so we call the dots nodes and we call the lines edges and that's what you hear people talking about nodes of a graph I think Edge is such a weird weird term to use for it in in my opinion you know because anybody thinking of geometry or whatever you think of that always being like the that's the edge you know that's that's as far as we can go you know you're not gonna go beyond it this is a great point and it's something so if you look in academic literature they don't call the dots nodes they call them vertices which makes it even more confusing because you think vertices and edges now you're like what are these like polygons or something and so yeah exactly that's actually no a really good point because yeah it can be confusing graphs have no they're not geometric in the sense that like you can draw them a particular way right you could and and and that's one way of drawing them and then you could sort of Shuffle things around and as long as you keep the connections the same you can move things around as much as you want and it's still it's the same graph like you haven't changed anything so there's no like geometry involved with them it's just connections just connections which is interesting to think about the one that I think maybe most people would be familiar with is social networks like the the idea that yeah yeah that that's where I started to hear about this idea that quote unquote social graph that like I join something like I don't know let's say LinkedIn and so I have this group of people that I friend or connect to that are you know people that I worked at this one particular job and so I have these like sort of connections there and those people have their connections from that same job and then of course people move on to another job and then they start developing this other tree from there whereas like maybe a simpler one would be like you know like a Facebook thing which is kind of a friends and family kind of set of graphs which are interesting and so there were like these startups and stuff that would try to jumpstart their social graphs by well let us uh get your contact database would that would that be okay like let's have access to your contacts you're like oh okay or you know and so suddenly they've imported all this stuff in and they can go oh it looks like this person knows this person this person and you might be friends with this person do you want to add them to your whatever friends list and so forth and so there's like all these like you said you know individual nodes people in the case of those things and then the edges are all the different types of connections you might have and the types of the relationships may be different and I don't know if that's something that's defined in graph Theory also it is yeah yeah okay so like I don't know if you want to jump from there but that's the one that I can think of and then I you know I know there's other ones like we talk about I had Calvin Hendrick Parker on recently to talk about Rag and the idea of augmenting llms by adding additional information and so this idea that you have to vectorize quote unquote this data that you have and so then it has to build all these relations between the words and the concepts in the documents that you have there and you use a tool like something like chroma DB that can have a database that's designed like that that understands the idea of you know maybe a little bit uh more than a standard relational database I think a relational database is a little I don't I want to call it crude but it's maybe missing some of the the the tools and yeah maybe we can talk about that like what what's different in the in the types of databases there yeah absolutely so I mean I think to so I think I there's couple of questions right so yeah I think I threw about five at you I'm sorry I mean so uh I definitely want to get to the the last one about like yeah the relational databases and everything that's that's an important topic but just to maybe talk a little bit more about graphs and like so the idea of like a social network and everything that is kind of a classic example of a graph and okay you see that all all the time in like intro articles about you know thinking about graphs because that's something it's not only a graph that we you know belong in that everyone kind of you is in some kind of social network but it's one that we actually use every day you use social media you're like actively traversing a n graph basically you know when you're clicking on on uh profiles and stuff to go see and everything and then who gets shown different things and EXA that sort of stuff yeah you know but that's just one one way to build a graph of people to give you an idea of how General the notion is and and maybe why it can be challenging to figure out like how to apply graphs to this because theide is once you have a graph then you do something with the graph itself doesn't really answer anything it's more of just like a tool that now that you've got it you can do some kind of algorithm on the graph to answer some kind of question okay but if the graph isn't set up properly if you don't have the right objects and the right connections you may not be able to get the answer to your question so it's it's not like okay I've got a social graph I've got you know here's the people and these are like the edges but let say it's LinkedIn where you can ALS you can follow someone which is like a directional like you follow someone they don't necessarily follow you back but then you can also connect with someone which is like a mutual thing where you both connect with each other and you establishes connction right between both people yeah yeah so which I think it was always sort of the Facebook kind of model is like you know like this is agreed that you know we are connecting together whereas the following was kind of a a Twitter thing so it's weird that have a a network like LinkedIn where it's like both yeah exactly yeah so you know that's that's like one sort of Distinction there's different kinds of edges they can be directed right they can go from one node to another but not necessarily in the opposite direction or they can be undirected where it's just like a it's a connection that goes both ways okay and then in in addition to that you can think of like okay if I've got a graph of say the LinkedIn Network and I know this person a follows person B and then I've got other types of edges that are connections that aren't directed and everything depending on what you're trying to answer that may not be the right be the right graph and then also you may not have that information so let's say your business I'll give the example so I can't claim responsibility for this example this comes from a demo that was developed by some people at relational Ai and some people at snowflake which is our partner where when we when we talk more about relational AI you use it in conjunction with h with snowflake but we we'll get there so the example is they call it their Tasty Bites example and it's basically like a fleet of food trucks in San Francisco I guess or what some City okay and it's like a fictional business right they manage all these food trucks and everything this example they gave was you've got a food truck that's been serving cold burritos and now people are complaining on social media about these cold burritos and everything and you're trying to put this fire out you want to try to figure out like you can figure out maybe who the people are that got served to cold burrito because you can just look at orders at that food truck right like you've got that information so okay you know the people who sered got served a cold burrito you can send them a a coupon for free burrito or something say we're sorry you had a bad experience whatever you want to do but those people are going to talk to other people right right their friends and everything and say Hey you know this food truck really sucks I got served a cold burrito don't go there so the idea is maybe you want to try to get ahead of that and try to figure out do we know any of their friends can we sort of like preempt that and send like a a free coupon to their friends as well right right but you don't can we make sure we hit the community with this stuff yeah but you don't have their data like their social network data you don't know who their friends are all you have is customer order data but there are ways you can sort of infer well we might be able to figure out maybe we don't know that they're friends but we know that they maybe know each other maybe they're co-workers or acquaintances or something because maybe we've seen that they have come and and eaten together at the same food truck they've bought they've both purchased something at the same food truck within a short period of time multiple times over the last year like maybe they've done it four or five times over the the last year or something and so we'll just we don't know for sure right you just flag okay there's a couple of transactions that are within say like 20 minutes apart from each other these two people and the same people have that we find that same pattern at the same food truck with the same people like multiple times throughout the year we're going to guess that they they might know each other and and so what you're doing is you build a graph now where the nodes are the people still your customers but the edges are like you know maybe know each other based on the data that you've got just from you know orders and everything so you can build up a graph of like a you know potential social network it's you can't verify it or you know whatever but it's a it's a decent sort of proxy right for all of that and so even you you don't have all that data you can still kind of infer some of that stuff which is maybe kind of weird right like it's makes you wonder about like what what else can you infer you know from that kind of stuff but but these are but it goes to show you that even if you say like oh I don't have a social graph well you don't necessarily need a social graph to be able to answer questions you can sort of build a proxy to that which is still a graph but it's just based on kind of what you do know you can come up with some good rules to help you make those decisions and everything and then you can you can still get a solution to the problem that may not be the best like perfect like you nailed it you got everyone's friends but you know at the same time it's like okay well we sent some wrong people a free coupon whatever they'll maybe they'll come and still get a get a burrito and and eat eat there some more but um yeah can I ask a question on the database side like you know I'm so accustomed to standard relational database tables and so forth and so I I think of like columns and I think of these types of pieces of data that would be in quote unquote a row and I feel like this is not structurally the same like you're not looking at a node and the edges that are attached to it the same way like I I don't understand how that stuff's stored like yeah like do we have to have a completely different Paradigm or or is it the same yeah so that's that's a really good question traditionally you've switched into a different Paradigm so you've got data that's in the relational format like you have a table of customers right and you've got customer ID their name right and other information about them that you that you know right yeah phone number address blah blah blah yeah that's not in a graph format like you don't have a way to know that like oh these two customers are related to each other and so traditionally what you've done is you've exported that data you've turned it into a graph data structure that has the right tools for managing the connections and that kind of stuff whether that's putting it into a graph database of which there are there are several or it's just doing it in memory like if it'll fit with there's something like networkx is a python package that can do graphs you know in memory and those kinds of things so yeah okay I've heard of that one yeah and so that's kind of the traditional route and and honestly if you have something that fits in memory I mean you know Network X is probably a great the best tool for that if you're if you're in the python land like that's not a it'd be hard to beat you know something like that if it has the tools you need so you and you might do that but the the other kind of issue with all that is in order to get it into the right format you may have to move data out of your database somewhere else and this job at relation AI is the first time I've really been exposed to like the Enterprise world of of data and everything so much was like quote unquote large data data whatever you want call it it was kind of like a like a culture shock at first to me of like how big of an issue this actually is for for people and you know in some cases it's because of the size of the data right it's just I have too much data to move in some cases even if you could move it easily it's it's not the size of the data that matters it's you know related to ideas of security and governance of data and like this data there are regulations that tell that say that I cannot move this data like I have to do it here right and so right you know all these issues kind of prop up when you start to try to apply this stuff to like an Enterprise level where you've got massive amounts of data or you've got you know regulations that say it has to only be in certain places or things like that it can't yeah you know and it's like okay well if we do want to move it now we've got to start a whole process of getting things approved and and all this stuff I did a little bit compliance uh for just a short while and so like filling out vendor management forms and answering every every single question inside there to say you know this is what they do this is the software that they use this is how they're connected and it's just very very very detailed and you there are Fields where that right is required you know you have to have done all that and I can imagine that's a huge part of you know quote unquote Enterprises you know yeah well I mean any anyone that's handling medical or financial data I mean there's just all sorts of stuff they have to keep in mind and and yeah it's really it's really difficult and people for good reasons take that really seriously and it kills projects I mean there's things that you look at it's like there's a great off-the-shelf solution for some of this stuff and it's like well you know it would be too expensive to go through the process of getting all this stuff approved like we're just not going to do it this is becoming more and more of an issue with the rise of like all this AI stuff and everything right I mean companies are feeling massive amounts of pressure uh to try to to provide something but then like they're running into like these kinds of issues right like I can't I can't just send my data to open AI like that's that's not really an option for me I need I need something in house or something so yeah all these these kinds of issues and then going back to like the relational database idea is like okay I've got all this data already it's in a relational database that is kind of just the standard format right and if you're an Enterprise you're on a data Cloud somewhere you're either on you know Google or snowflake or or Azure or things like that again it's all you know you've got most of it's in relational databases and you can use relational databases to represent graphs pretty easily though you have to kind of switch your thinking from nodes and edges to relations right like just tables of rows or just tupal even if you want to think of it that way okay you could represent a graph as nodes are just a single columned table right like that's my set of nodes and then I could have a table that just has two columns where I have like a from column and a two that has the ID of the node and you know that it's going to in the tape it's a sort of connecting table that that is looking at you know these other you know quote unquote relational databases but this particular you know new table that we've added is all about the connections it's it's all about the graphs yeah exactly yeah and so you don't even necessarily need yeah like I said a single column for the nodes it could just be like your existing customer table that's like okay where the node information comes from and then you have this Edge table that connects them with like two columns just using their IDs or something yeah now so okay so that helps you represent a graph in a relational database but then how do you do stuff with it right like you all you have if you want to do something in the database with it you have to use SQL and SQL is going to struggle with a lot of graphy kind of operations it's just not optimized for that it's isn't going to work that you can I've seen some amazing SQL through the course of the last couple of years of people doing some of these graph things and it's just it's like you see it like wow okay they proved it's possible but like who in their right mind is ever going to write anything like this like no one melts your brain yeah exactly it's not you know not good and um and so if you want to be able to do something directly with the data in the database and not have to you know move anything around then you need a system that can work with relational databases and that's where relational AI kind of comes in that's the whole thing that the relational part in the in the name so we have you know before I joined the company there was a group of researchers that done a ton of work on new kinds of joins and all this kind of stuff I'm not the right person to talk about all the the details and I'm not sure I don't know that the real python audience would necessarily want to get into all the those details either but it's real you know database Theory having everything but they they came up with some novel ways to do joins and things like that where you could now tackle these kinds of problems just inter a relational database and we've put together a python API that gives you that can access the database and everything and and then run the the different operations you want to to do with it and everything so it provides a way right now it it only works on snowflake the snowflake data Cloud yeah yeah and uh We've we've partnered with them and that's kind of we're just that's that's where we live and if you use snowflake then they have this new feature called native apps which they announc well I think they announced it like a year ago it's now available they've announced the availability of it with their a couple weeks ago in their big Summit that they did and so where we have the relational AI native app that you install people can start running graph algorithms and doing stuff like that on their on their data yeah you were telling me before how it kind of connects to this snowflake instance so again going Beyond this idea that we we've mentioned several times of like we not only do not want to move our data we by compliance reasons can't or should not do this and so the idea that it can be in place and then this tool can kind of connect to it and do the rational stuff on top of it and and kind of give you that that sort of functionality which is kind of neat I I I think that might be a a model that is definitely forward thinking you know a lot of people in this field are looking at all these different like types of tools and and I don't know how many times I've heard of like take all our data and let's let's put it in a big data Lake and move it over here and and that sort of stuff it's like there's only so many times you're going to go through that process of moving the data again I think yeah for sure that I mean it's so much pain that yeah it's it's not something companies want to do very often and on on top of that it's interesting it requires a lot more than just just something like relational AI at at this stage so there's been a ton of work on snowflakes part to provide you know what they're what they're doing is they have it's called Snow part container services and it allows you to run application containerized applications in your snowflake account within their whole security perimeter so you can actually run all sorts of stuff in there like you know anything that you can put in a in a container and run it and and that's how you know we're doing so we have our service that's running as a container in snow park container services that's like installed in the snowflake account and it's all being spun up all within that you know that security perimeter and everything what we're doing is we're we're attaching to the data so that the data never has to move outside of snowflake but then all the actual compute and the processing and all that is happening in the like in in our application but it's all in your account like it everything is just running inside there and yeah it's it's a model you know it's a platform for building applications literally on top of the data without having to move data anywhere else you don't need an application database at all you just kind of the Le source of Truth so yeah it's I think it's all in you know service of moving towards this picture of your data is just in one spot always yeah and then everything else just kind of connects to it or connects to you know views that have been prepared for that that application and things like that yeah okay yeah yeah that whole uh bit of a warehouse kind of formatting kind of thing um potentially yeah okay yeah cool um so you mentioned Network X as a potential tool that maybe people could explore if they're interested in diving a little deeper into graph Theory you know using python I mentioned chroma DB because I know that one's really popular especially with tools like Rag and so forth and I I'll link to some resources um from real python that talk a little bit about you know vectorizing things but is that part of the process of what what you're doing of I use the term vectorizing I know that the relations may be very different depending on what people want to do with like a tool that you guys have we talked so much about yeah people in the relationships there but is it just a matter of like sort of changing like okay well what are the relationships that we're trying to set up of it could be I guess events that that could have happened that you you want to look at you know the graph there or we think of again vectorizing language into large language stuff so you see all the connections like you know again the common thing of people saying it's just selecting the next appropriate word you know or whatever um is that part of that is that I guess I'm trying to formulate a good question for it I think I think I know what you're getting at and I think I can okay provide an answer so we we've talked a lot about graphs and everything and that's a big part of relational Ai and what our tool does but to maybe to talk a little bit more specifically about what you know what it is that we do to giving idea so the main concept in our tool is called a model okay we have a python package that you install inst called relational Ai and that is uh that's on Pipi you can just pip install it you are required to have a snowflake account to actually be able to do anything with it so if you're just a single developer I don't it might be a bit of a challenge to get into something like that but that's all right I mean our target audience is for people who AR snowflake so yeah but uh the the main concept is a model so when you when you install our our python package you do you know import relational AI as R AI That's the what we call Ry is how we pronounce it and then you just do you know ry. model capital M it's a class and you give it a name and you get this model object back okay what you're able to do with a model is the or the way we think of a model is it's composed of three things it has objects which are like the nodes in the graph we just call them objects because they're just kind of a general you it can be anything yeah yeah sure objects it has types which are sort of collections of objects so you can sort of categorize objects by type and then objects can live in multiple types so you can use a type for example like if you have a person you could say okay this is a objective type person but that person is has you know age 18 or older so I also have a type called adult that's going to hold all the you know people who are 18 or older in this adult type yeah whatever so you have things like that okay categorizations yeah across the board categorization yeah kind of a let you build a hierarchy and everything okay and then you have rules and rules that's that's really the the key thing here so by by having objects and types actually there's one thing I should mention before I get into rules is objects have properties and you can think of this like a python object in fact that's actually very much the way we want you to think about objects in the model as if they were just kind of python objects and so they can have properties you can have you know object. name right if it's a person object and that's that's a name property right and so what what we're doing is we're transforming in our container that's running in Snowflake you can point to specific tables that you have in Snowflake and say I want to import these tables as objects in my model okay and it'll pull everything in and the what what's going on is we basically create like a graph index over that data so it's still a relational index but that's what we're sort of storing in the the container that's that's running in your in your account so we've got this graph index now of all the data and then you can add rules into it I should say when we have by graph index you can think of it as we're basically setting everything up into this the way we talked about where you've got these tables of just pairs of like these are my edges defining my edges and my relationships and everything and then okay indexing it all and everything to make it fast and stuff to access so that's kind of what's going on there and then you can add rules to the model so this is what kind of makes it unique in in sort of the graph world so by having objects and properties and everything you kind of already have a graph structure where because properties don't have to just be values properties can point to other objects so you can have a property that's basically like an edge in in the graph it's just saying okay this property connects you know objects with this um together and what rules allow you to do is sort of put constraints on the graph and build new edges and things like that on top of it to expand the graph without having to do everything manually so you can have a rule that just says get me all objects of type person check that they have an age property that's greater than equal to 18 and set the adult type on them right and just becomes a rule in your graph and then or in the model I should say we really want to use the word model so because the the the idea is that we don't want you necessarily having to think in terms of graphs we want you to be able to think in terms of just objects and properties right which are already and modeling what all that stuff is yeah exactly and so the graph stuff is kind of happening in the background uh for you in terms of like you know that structure and everything okay and so rules are really powerful because they let you express sort of like the the model of your business the rules sort of Define like the way things are or supposed to be in in the business or you kn