Statistical Relational Artificial Intelligence with Sriraam Natarajan - #113

Key Takeaways

[Music] hello and welcome to another episode of twirl talk the podcast why interview interesting people doing interesting things in machine learning and artificial intelligence I'm your host Sam Cherrington [Music] all right well listeners this is your last chance to join the conversation and share your take on home and personal AI entries for our my AI contest closed on Sunday February 25th at 11:59 p.m. Eastern so go ahead hit pause right now jump on over to twill a I comm slash my AI check out some of the existing entries and submit yours before we move on I want to give a quick shout-out and thanks to everyone who has submitted a video so far this week so cheers to you Matt Bradley Laurent and Mohit in this episode I speak with Sriram Natarajan associate professor in the department of computer science at UT Dallas while at nips a few months back Sriram and I sat down to discuss his work on statistical relational artificial intelligence star AI is the combination of probabilistic and statistical machine learning techniques with relational databases we cover systems learning on top of relational databases and making predictions with relational data with quite a few examples from the healthcare field Sriram and his collaborators have also developed boost srl a gradient boosting based approach to learning different types of statistical relational models we briefly touched on this along with other implementation approaches alright let's do it [Music] all right everyone I am here in Long Beach California I'd say sunny Long Beach California but it's actually fairly late in the evening my first interview here at nips in fact the first time I've attended the nips conference and I've got the pleasure of being seated with Shriram Natarajan Sriram is an associate professor of computer science at the University of Texas at Dallas Sriram welcome to this week in machine learning in AI hey great to be here why don't we get started by having you tell us a little bit about your background and how you got interested in machine learning and artificial intelligence so my interests are in machine learning artificial intelligence primarily in this field called statistical relational artificial intelligence about which we are giving a tutorial tomorrow which is why I'm here at nips and the application and adaptation of these algorithms to many real problems focusing mainly on healthcare type problems but also on natural language understanding finance types of problems but mainly focusing on healthcare problems how did I get interested I think it started from a grad school when I came here for master's at Oregon State my interest was primarily just computer science in the broad area of computer science but I took artificial intelligence courses under professor Prasad sorry Polly who went on to later become my PhD advisor all right and I know I just like the style of teaching I like the fact that the artificial intelligence and machine learning techniques combined two of my favorite topics math and statistics with computer science okay so the combination of math and computer science kind of glued me into AI nice so my thesis was primarily in artificial intelligence and machine learning okay didn't have too much healthcare problems when I did a postdoc at University of wisconsin-madison professor David Page and professor charlyk introduced me to the possibility of using machine learning and artificial intelligence to solve healthcare problems we are looking at electronic health data electronic health record data from Marshfield Wisconsin that got me interested in kind of like personalized medicine no it's called precision health right so these so I got interested in adapting our algorithms for for these problems so I think that's how my journey has happened so far okay okay great and so the you mentioned that you're doing a tutorial tomorrow right and it's on statistical artificial positional artificial intelligence or star AI yeah this is a great acronym Thanks what is story I so star AI is the combination of probabilistic or classical statistical machine learning with more logical or relational artificial intelligence the idea being that so this traditional artificial intelligence techniques and machine learning techniques make a lot of assumptions on the data they need a lot of pre-processing they read a lot of engineering whereas what the story I feels starry-eyed methods try and do is kind of look at the data in a kind of a more holistic manner look at the data in the natural form in the relational form so okay a classic example is most of these machine learning algorithms - what is called as an iid assumption you know what iid independent and identically distributed mine which means each of us is described by the same features or same set of attributes right and each of us is drawn independent of each other and then you learn a classifier you're not a predictor on top of us whereas if you look at real data it's not true the problem you have you having diabetes or me having diabetes depends on your parents and in my case my parents okay and my parents parents and so on so the family history matters or I can see that's particularly not true in the case of diabetes or disease are there other examples outside of the healthcare realm or that that is also she'll be here here right now some like even some small thing like the probability of somebody smoking depends on the fact that their social network friend smokes are not okay how popular a particular a particular person is let's say for in a scientific field depends on who his or her coitus his collaborators are the popularity levels of the collaborators so if you take classical machine learning methods they kind of project these multi relational data into a single fixed flat form and they operate on that statistical relational learning methods on the other hand all of the data to be in its natural form and we we try to learn using the power of first order logic and relational logic so think of learning in a in a relational database using statistical methods okay so that's why it's called statistical relational artificial intelligence because it takes the classical machine learning methods but kind of upgrade them to learn on relational data allowing data to be in the natural form so you take any data electronic health records are classic examples but you look at social networks or you look at hot air citations you look at movie databases everything right most real data now Google stores everybody stores the data in a relational database but somehow when we are learning it they are transformed into a different format and they are aggregated they are somehow shortened and they are all compressed into one form and then the algorithms run on those compressed form in our case what we are saying is well let the data be in its pure form which is relational somehow the models be learned at a relational level which is why we sometimes call these models as lifted models because they are they're lifted from a flat representation to a much more richer representation and when when you first started going through this the first thought that jumped out in my mind was relational database but then as you described it from the context of you know different distributions and kind of moving beyond the independent identically distributed assumption I thought oh he's not talking about relational database stuff but then you come back full circle to relational database can you elaborate on the relationship between statistical relational ai and relational databases so relational databases are representations so relational databases are how the data is actually stored on whatever you are storing them on the statistical relationally I says I'll take the structure of the relational database and now the model upon the structure so what happens is in so let's just take a simple example in an iid framework in the classical machine learning framework each of us you and I your father my father we all become individual examples for a classifier right okay in these relational models however your family becomes 1 mega example my family becomes 1 mega example we may be connected I don't know bike you know when we go like 20 levels up but but at the for whatever data we have we could be different sets of mega examples and and we we have so what these statistical relational learning does is learn at these mega example levels then at individual example levels so then how are they related to databases if you look at a database and I start let's say with you or let's say we start from Shriram that's easier to explain in a in an academic context then you go to my advisor my advice is collaborators my collaborators you can form a small network of people around me and basically use that information to predict something about me whether I'll be successful whether I'll I'm gonna have a paper next year or whether I'm gonna have a grant next year you may predict that by looking at whom I'm working with what topics I'm working on how popular it is and so on and so forth when it comes to relational data you are basically looking at all the relations that are connected to me and figuring out how you can make some predictions about me this is true not just in citations it's true in many of these health care problems it's true and natural language problems it's true in any any real problem that you look at you have objects people are objects and they are related and you have objects everywhere it people are objects things are objects and emails are objects and projects are objects and then we are all related to emails and projects and papers and so on and so forth and then you start talking about relations how are these relations maybe two people work on a project right there's two people work for another person and that person reports to an organization and the organization is managed by somebody so there's always these relations now that you can tease out from a database so it's very related to a database and I can think of what we do is learning on top of a database so I'm envisioning then well I guess there's two directions that I want to go with this one is the extent to which what you're specifically talking about is from an implementation you know related to learning on databases meaning you know taking advantage of you know schemas and primary keys and things like that that's kind of one possibility but then I'm also hearing something that sounds to me like where as we you know might have in traditional machine learning a set of independent examples in statistical relational yeah you've got you know this set of examples and kind of overlaid by some connectivity graph and it sounds like what you're you know trying to what you're doing in a sense is maybe you know finding different ways to kind of feature eyes that graph and use it in the training process yeah that's actually one once reasonably simple way of understanding what we're doing reasonably simple is a great place to start that's the reason I'm saying reasonably simple is because we really do not construct too many features have no net we let the features be defined based on the relationships themselves so for instance let's let's go back and take a classic example of movies okay I want to predict that it the next Marvel movie how much money is it going to make and then you want to look at well typically people look at history how things change and so on but you could also look at the actors and you could look at the different actors that are involved in a movie and you could say well these lead actors have typically tend to get you know a lot of box office collection and so on and so forth so what we are trying to do is we are looking at the fact that one Marvel movie might have five stars five five big entertainment stars yeah like big actors what are and then another movie on the other hand might be carried solo by a single you know Iron Man this is Iron Man even Marvel ladies that's Marvel okay I got that right so Robert Downey himself carries one movie with him but but let's say the Avengers are some other one is carried by five six people all right that's it necessarily mean that it's better than one we don't know that right all right and what typical machine learning methods tend to do is kind of collapse them into one group we on the other hand say well how about we let the data speak for itself maybe each of them combine some way and I look at the previous Avengers look at who is directing who's imparting so from that perspective I'm not doing too much feature engineering but I'm figuring out how can I use this graph of people and make that use that to make a prediction that that I cannot with with a normal machine learning algorithm okay the other problem is here is the problem okay so let's say you you in classical machine learning you make an assumption no I'm going to look at the top five actors but let's say suddenly Marvel decides to make this mega when just moving with 20 right okay so you got to go back to your model and say oh man even from five to twenty nine and he was right just you're multiplying exactly you might want to you cannot generalize that easily right V on the other hand make no assumption on the number of individuals or a number of objects we can say as long as you know they are actors in a movie we are going to use their information to make predictions so that's where the power of relational representations come in and they are kind of tied actually you asked me two questions one is on the implementation aspect divided and I don't think they are too different because when you think about how these are ultimately implemented they are in some sense implemented based on either a logical representation the power of first-order logic as people know for a long time in in computer science or from a relational representation as a data base so it's I so which is why in u.s. this is called statistical relational learning because most of the work is done on top of relational databases okay in Europe this is called probabilistic logic learning because most of the machinery underlying is a logical representation and both of them do minimal feature construction okay they kind of learn at the level of the databases the cool thing is they are statistical and probabilistic because you can all have the data to be noisy so that's what happens so our data can be noisy our models are robust tool our models are robust to changing a number of individuals number of parameters number of objects in the world and and so we learn at the level of objects and relations not at the level of specific specific people okay and I think that's why these models are powerful okay and so how does it work how does learning work or how does kind of what's the you know the method that you apply there so what we do for instance I'm going to talk about one specific method that we do which is called a relational functional gradient boosting okay that is this famous gradient boosting technique inside machine learning what we are doing is we are elevating it to the relational setting so it's called lifted relation lifted gradient boosting and the idea is very simple so let's say I'm interested let's just take a simple example I'm interested in predicting if somebody has a heart attack okay and we look at the database let's say I'm a positive example in that I have a heart attack Sam you're a negative example you look much fitter than I am so you don't have a heart attack and what happens is my model uses my attributes and it starts - lon and then it says who is the father of this person diabetic turns out my fatherís okay and then as the is a cholesterol level of this person pretty high if so then the probability of a heart attack is let's say my model comes up with me having a probability of 0.7 100 I the same model comes with you to have a probability of 0.1 3 okay now my because I'm a positive example it should say that I have a probability of 1 to have a portrait heart attack but I said 0.7 which means one - point seven point three is the mistake that the model makes on me your probability should have been zero because that's you don't have a heart attack according to the data right but my model said 0.13 as your probability say you are weight so to say it's minus 0.1 three that is a mistake so every positive example will have a weight of greater than or equal to zero every negative example like you will have a weight of less than or equal to zero so what we do is we learn a small model regression model it could be a class which says if the father is diabete follower of this person is Witek and if this person's cholesterol level in the last three months is greater than I don't know if let's say the HDL cholesterol is is less than 30 then the probability of heart attack is this it could go do other things for instance that says the relational power comes from the fact that your class could say something like if any of his close family members and this close family members could mean my dad my dad's dad my dad's brother my mom's brothers and so on and so forth and you will have completely different number of uncles and aunts as I do hmm okay so you could say something like if a close family member has a predisposition for a heart attack then the probability of heart attack for me is so much and you could say that because of the fact that I'm not defining I can define close family member to be either an uncle or an aunt or a first cousin or you know my grandparents on either side and so on and so forth and we are not restricting the number you just look into the data and you can learn it so what the model does is it tries to find out these factors that matter put them together and make a prediction and then it finds which mistake where it makes a mistake learn something else to fix that mistake so for people who don't have low HDL and whose father don't have a let's say diabetes they could be that they smoked a lot and drink a lot and for them there is a different issue so what the model does is oh okay I've got serum mostly covered there's only a small mistake on sridham but there is this other person who has a heart attack whom I have not covered so let me focus on that person and so it kind of may fixes these mistakes repeatedly and learns one robust model in the end and it scales up pretty well we have the code is available online so people can look at that from a web page as well okay a couple things so one you you talked about heart attack smoking cholesterol levels these are all things that I would associate as being features in a traditional model but yet you said that you don't really get into features as much with this how do you how do you reconcile that so I have to make it clear in the sense that these are all features so everything is a feature what I'm saying is we don't really do a lot of feature engineering I don't we don't say who for instance beautiful question right so let's say you go to the hospital again you're looking much fitter than I am you'll go to the hospital maybe once a year you do your hand roll checkup I go to the hospital three or four times a year okay mm-hmm now think about how a classical machine learning algorithm will do you will have one measurement of cholesterol I will have four measurements of colostrum mm-hmm I have I might have my a1c record at every every every actually four is good for a year so for a one C's which is every three months I'm going to record my blood sugar level and so I have multiple measurements and you have only one measurement how do we do this well classical methods a min/max average so it takes an average of my cholesterol level it takes a minimum maximum and puts them as features this is what we call as feature engineering right okay we don't do that we say if the cholesterol level in the last one year has ever been greater than this and that can be written by logic so a logic statement says there exists a cholesterol level and this is called quantification in logic and so they kind of so we don't even do that min max etc they can be done if needed we can do it but doing that is a subset of what we can do we can just let the data as it is which says that any time in the last five years Sri Rams cholesterol level has been this anytime after the age of 55 this person's a1c has been this and so we can tease that out so what I mean by feature engineering is we don't construct specific features for specific problems you just let the data be in its natural still sounds like feature it sounds like you the the if statement is like one heart encoding on whether they've their cholesterol has been high so cholesterol is a classic example okay a restaurant might not be the best example to illustrate the difference or fake the smoking properties of a friend okay all right if your close friend smokes yeah the probability of you in having a heart attack is point three or point four how do you define that in in in in in a propositional or in a normal setting okay I want to say if we're close to and features fall down because they're not related to the individual entity whereas you've got this broad universe is captured by all these relationships and it's all features but they're not like features of me it's features of this network its features of this network or the size of the network varies between people right so in the end everything is a feature you're right right but what is the feature of my point is that we don't do specific feature engineering continuously we let the data in its natural form which is I could talk about objects and relations and features of those objects and features of those relations right okay I just have to say oh sighs again you might have five close friends I might have three close friends mmm how do you encode this right right right right whereas our models can just handle them because it just says who is that one friend is that two friends you can just look at it and you can learn from it it's easy to do that I thought you were going someplace that you didn't end up going it strikes me that you gave the example of the multiple you go into the hospital multiple times a year and get your four a1c readings I go in once and get my one and then you you write your rule that basically looks to see if I have been you know have had a higher a1c reading over the past ten years yeah it strikes me that you've you've kind of lost a lot of information maybe there's information and the fact that you actually want four times and I only want once that's a great point it's it's not necessarily that we lose that information okay if you want to take that you can record that - mm-hmm okay so you actually do so two things that I want to clarify first I don't like the rules uh-huh because of this boosting algorithm rules are automatically learned from data so we use only data and we use some domain knowledge actually we I work on what is called as knowledge based machine learning okay now we call it as human on the loop of machine learning there is a human expert we work with cardiologists we work with radiologists we work with neuroradiologists and and so on and so forth we have a lot of collaborations in in medical with Medical Group's okay slang lower the rules learn algorithm the loons the rules are learn by algorithms what they give us is a little bit more knowledge they can say things like oh this this guy is much more difficult to predict than this person and then we tell the data you focus on this person or a person could come and say I'll come to that about the experts in literature after I answer this question okay so we don't provide the rules the rules are learnt automatically from the data okay and the rules themselves are capable of learning that the rules themselves are capable of saying ooh the number of this is the number of misses matter then it can say actually if the number of this is it's greater than four and in that number if there is a one see reading greater than this much so when you say you don't learn the rules does that mean you don't learn no we don't write the rules right right oh sorry the London rules when you say you learn the rules are you learning the the parameters of the rules nor you learn these rules themselves you learn the rules then effects along with the parameters okay it's just why so so if you think of like a complex neural network or a deep network where I'm writing these features out yeah the features themselves can be learned by us right so we learn these rules which are basically if-then-else think then in statements yeah and then we parameterize them the probabilities are real numbers depending on whatever interpretation you take but we learn the rules from data so yeah so Bilan it from data okay now going back to the other point that that I wanted to make about experts experts can provide a lot of things experts can tell us things like in some cases false positives are more important than false negatives okay okay so for instance in in in recommendation systems when I'm recommending a job to you right or I get these LinkedIn recommendations which keep telling me there is a postdoc position open oh yeah and I'm like come on man I did my time that was seven years back I don't want to do this again right right so that is a false positive it's a false positive they're from a recommendation system perspective that needs to be eliminated if you tell me for jobs only for jobs and those are all relevant to what I am looking at I'm going to toast your system so out of the potentially 4,000 jobs the human is truly four jobs but those are important to me so I look at it but if you give me 20 jobs out of each only four irrelevant the rest are all false positives then I'm going to lose trust in your system now think exactly the opposite side of an epidemic you are interested in predicting Ebola for instance mm-hmm right it's it's okay to quarantine for more people what is the first thing they're gonna do they're gonna sue you sue the city mm-hmm million dollars each at most ten million dollars is what we're going to lose but think about releasing for people with Ebola into the general immunity right then it becomes an epidemic and costs us billions of dollars just to talk in terms of numbers all right so what I'm trying to illustrate here is in one case of recommendation system false positives are more important than false negatives in another case false negatives are much more dangerous than false positives an expert could come and tell us oh no no this is the case where this is more important than the other one okay and and what we do is we think of it as a knob and turn this knob on the classifier and say oh you know what that's what it is and in another example a sister domain expert could say Shriram for people what what I have observed in my experience is that for people with high cholesterol if they have high BMI then the risk of a heart attack is higher than people with lower cholesterol so even people with high BMI have two different effects they are not exactly telling me how these things influence except I give a qualitative understanding between between two things and a target of heart attack so that's called monotonicity synergies they call qualitative relationships we can take that you can take preferences and how do you how do you encode that kind of thing I'll come to that in a second I'll just illustrate this one thing and I'll come to that so the third thing for instance is I was in a college town Bloomington Indiana let's assume that you're trying to build a robot and the robot is sitting on top of a building you know having a donut in one hand and observing how people stop and talk science how people stop at stop signs I'm in a university town which means only 50% of the people stop at stop signs now and then you you the the machine could ask the human hey I'm looking at this data but like only 50 percent seem to be stopped what do you think I should do then the expert could say well I prefer that you stop at stop signs if it is safe to stop maybe there is an extreme case where you know you don't want to stop in a stop sign because it's dangerous actually to stop right you might put you in danger or somebody in an errand I can imagine such situation so animals cars for instance and ice you say something soft that says I prefer that you stop and stop signs and then it says okay great then what what what our algorithm does is it takes these statements as constraints and then it combines that with the data that I have okay so there could be regions in the data where there is a lot of mistakes like stop signs then it says oh the data has a lot of mistakes but they expert has told me that this is what he or she prefers mm-hm so what I'm gonna do is take what he or she tells me combine it with my data and wherever the data conflicts the expert or the expert can fix the data I'm gonna weigh that lawyer wherever they agree I'm gonna be a bit higher and kind of incorporate that into my model so what I do is I get these human inputs as constraints to the learning algorithm and and we we have shown across several publications that such constraints improve performance mm-hmm and very recently what we have been trying to do is also do a little bit more which is let the algorithm ask questions okay instead of the expert telling so when we started this research the expert has to say everything that he or she knows about the problem and then we take them as constraints and learn but now we have gone the other way and we have said we'll start looking at the data in the regions where we have extreme uncertainty about things that we are trying to learn we will solicit information from the expert so we call it ad with actively advice seeking advice okay so the so the key is for the machine to know what it knows and solicit information about what it does not know and then the expert could say something it takes that back into the model adjust it again because it's a it's an iterative learning method that we have it can go back and try and fix its mistakes so let's put this on the stack after the enter to the encoding question okay but it strikes me there that there's a there's a huge gap between what the algorithm is likely to surface you know some factor is you know there's a high degree of ambiguity and some factor and something that you can present to a doctor that you know makes that intelligible and it kind of elicits useful advice to feedback into the algorithm which goes back to the encoding question like how does that so that part wonderful question that is a fantastic question and that is another reason why these logical models are more useful okay here's a problem if I if I now use a very complex learning system underneath then it's going to say feature one seven six three five four two two one right seems very Fiat for me and I go to the doctor the doctors are going to know anything right right I on the other hand mine is a logical class it's an if-then-else statement so it could say for people who have slightly lower HDL level but on the other hand the triglyceride level seems to be good there seems to be a lot of distribution over these heart attacks and diabetes what do I do hmm so what happens is because of the fact that we are learning these these knowledge at the level of the data themselves that the relational schema themselves it's easier to present to the doctor because they know what the schema looks like now it's not just the doctors we work again with financial experts so again our legal experts so we have these documents and we can say well you can't say something like if the parse tree comes up with this noun phrase no but we can extract the sense that says this part of the sentence that seems to be always conflicted with this part of the sentence why is that and we can ask that question and and that is only because of the fact that we are learning a much more representative model a much more interpretable model and then than a typical complex model letters that's because of the fact that you're learning with object objects type objects value objects relations like this then something right okay so then you can present this to the user so that's another power that that comes from these relational models that you don't get with standard models yeah so can you talk a little bit about how you benchmark this relative to other approaches and what kind of results you've seen so that's something that we do all the time extensively what we try and do is we take these models and try to win so for instance if I have to run a deep belief Network or something we try to create as we spend a lot of time engineering as much as possible and and and try to engineer this a lot and then create many many many features and you try to use a standard machine learning algorithm as your as your baseline now what happens is for the current data set it might learn a good performance right on your training set it could actually give pretty good performance because most of these machine learning algorithms can learn well if you engineer your features very well and that's doable if you do that right the problem real problem of Playa comes when it comes to generalization so most of the benchmarking that we do is on generalization so for instance you learn about whether a student works with the professor from one university right you just on that university most of these algorithm works great now what you do is you go to a different University and apply this model right I kind of break down because you have to create new features new things whereas these relational learning algorithms because of the fact that they learn not at the level of the individual professors but at the level of these groups of people and in sets of people it gets much more easy to adapt and transfer okay so when we do this benchmarking we try to figure out let's say other networks which is way a classic example is let's say you've used all of Sri Dom's family network data what I'll do is when I'm testing it and go to science data and I see the Sam's data work on my model mm-hmm and and then you do that with these classical methods some of the times you don't get them sometimes if you know what you're going to test on you can certainly engineer and try and do that but most of the times these these relational models work beautifully on these problems as well yeah I hear people talk all the time about like overfitting on imagenet things like that in the the industry in the field do you feel that this notion of kind of focusing on generalization as opposed to performance is like underappreciated or under pursue this basically definitely but the the good thing is that there's been a new conscience on looking at generalization as an important aspect but initially there's been a lot of work on on on the science of X experimental methodology of machine learning there's been a lot of work on empirical machine learning I think that'sthat's an extremely important research direction right people kind of lost a little bit of that oversight when when we when we when we went and I developed aggressively more and more and more newer models but I think now people have started realizing that because of this abundance of data sources and you have these multiple data sources that we have to somehow integrate and work with certainly generalization again has become an important issue so I from my limited whatever 15 16 year experience I have seen that there is a lot of interest now in making sure that we build models that generalize across populations across datasets across diverse data sources and I think that's that's certainly an important research direction and which is one of the reasons we want to make sure that any model that you develop it's not pipe to that particular thing which is why when I see many of these people talk about AI we're all a little bit one it because it is specific domain specific solution that you are always worried when it's gonna break and it could burn very easily right if you don't think about generalization are there standard and well accepted ways of measuring generalization or do you do do you feel like everyone kind of figures it out you know presents their own results in their own way in papers there's a lot of work on understand generalization errors and understanding generalization everything cross validation and and the by hole bias variance theory is trying to figure out how the bias on on a particular data set is going to affect the variance across a multiple data sets so there's a definitely a lot of understanding in statistical machine learning on this notion of generalization error and people have definitely looked at it but I think when it comes to the practical implementation people didn't ignore it they tend to kind of evaluate them in a very narrow field I've even had students both in my class and in my research group sometimes build these so called tuning sets where you fix your parameters using these tuning set or your learning algorithm from test sets you should never do that's like cheating that's like having a practice question from the final midterm or the final exam that you have yeah you're gonna have a practice question on that so that's cheating but students do that all the time okay so we we tend to try and teach them that's not the right way to do it and we try to show them a proper way to do it so there's this definitely a lot of work on generalization and understanding generalization but remember it's it's within a certain domain can we do it across domains maybe maybe not like can you take something like learning how to drive a car and figure out how to drive a plane or fly a plane no but but maybe driving a car in u.s. was India I can't imagine doing that easily but you could do that you could bind could you could try and understand how how does it generalize across multiple countries and it's right to understand how the rules of the of the domains change right at some point it gets into this artificial general intelligence right if you can realize across every domain that's what that's the goal yeah well that's a goal that's a goal but I don't know I mean I'm not saying that you need something that is very gentle I'm saying that you need something that is generalizable enough inside the same problem domain adviced yeah like we say if I if I if I build a model as I said for driving in US that should be able to drive in in Europe if not in India let's say that that level of chaos may be difficult but maybe at least in Europe maybe in London it should be able to drive a car if I teach it how to drive a car in LA and and that amount of generalization should be there at the very least all right would be cool to drive a car in India that would be awesome to achieve it but I'm not really looking for this one system that drives and flies same time right in fact I'm worried that that might lose some specific knowledge that that a driving agent will have that the flying agent will not yeah yeah so so that's the balance how much general knowledge do you want and how much specific specific I guess what's the right word here specific skill set you want for solving the problem I think that's the difference that that we have to find and I the sweet spot depends on the problem domain in some problems it's so think about it like you you have a problem and you go to a doctor and this doctor is a neurologist okay and now you want to talk a little bit about diabetes this person is going to tell you something but they're going to say this is what I know and you better talk to somebody who knows this right right and you go there right or are like a cancer right you want to go to an oncologist right and so so but before that you have a family friend who's a radiologist radiologist is going to give you a lot of information all right but then you say I but I still refer you to this oncologist for your treatment plans yeah and and that's what you want even with systems you don't really want systems it says oh I can solve this and this and this right and so not to generalizable right but so artificial general intelligence is great but we also have to accept that that comes at a cost is contextual yes exactly you mention yet code up on your site yeah tell me a little bit about the code is it is it kind of code applied to the healthcare say positions you know a general algorithm or it's a general algorithm it's a lifting algorithm that anybody can download it's a gradient boosting which operates on relational databases okay we have an extensive tutorial on how to convert your data to our format how to run the code we can learn multiple types of models relational problems take models we can learn multiple we can take the human inputs on this we actually have a wrapper that can do natural language extractions so you have text data but you want to let's say figure out who who's married to whom based on paragraphs reading okay some some CNN articles or something okay and you can you can post that problem we have shown how to do it so this is an extensive tutorial on how to use this it's available off of my web page okay eyes if you go to my lab that page through my webpage in in the software it's there okay and people can download it and use it it's a general purpose software not not only applied to health problems okay when it comes to health problems there is always a catch we have to be very careful on what we release and what we don't and so on and so forth so yeah okay and then we've been talking about relational databases but there's a whole set of a whole type of database called graph databases yes does your method apply to graph databases is fantastic question yes the answer is yes because graph is a relation for us right right notes and it just our relations and so relation is this common word that we use yes the graphs are just relations and actually because one of the most important step inside our optimization function is to count you have to count the number of instances account the number of papers that somebody wrote count the number of friends that somebody has in facebook and so on and so forth and that counting is actually a complex problem so we actually use graph databases to accelerate our counts okay so we do use sometimes a temporal sorry sometimes intermediate representation of a graph database okay waxing right are lying so far as graph databases and databases graphs they're all structure problems that we can handle using the formalism of logic absolutely so the answer is yes we and what we are trying to do is actually do exactly that a build a version that is more optimized for graph databases because graph databases are much simpler sometimes than the full logic like psych our knowledge base or never-ending language alone or Nell knowledgebase those are logical knowledge bases and those are huge there as a call psych ARP psych our yeah psych art was this company that's been running a learning that's been building a knowledge base for over 35 40 years okay and so those are like giant knowledge bases okay whereas graph databases are much easier to manage men and work with so they may be even faster for us to learn with so yeah absolutely running the graph database is something that we do and so you're out doing the tutorial kind of evangelizing this approach what's the end goal for you is it trying to get grad students and postdocs are you trying to get industry to adopt this what's what's the motivation and vision I think we want people to be aware of this big goal of statistical relational AI you want people to know that deep learning is not everything and we want people to know that there are other learning areas that focus on some richer probably more important questions turns out that the four of us who are giving the tutorial actually have large groups ourselves so it's not like we are trying to recruit people here ok bye we are going one conference at a time to kind of show that that this field is now actually much more matured than what it was ten years back okay and we are trying to tell people that you know there is a lot of opportunities inside our field so combining deep models with logical models combining let's say matrix factorization with relational data so is there's a lot of opportunities for this and that's what we are highlighting in the tutorial tomorrow is inviting more people to work on similar problems like we do and that's the ultimate goal for us is kind of show people that this is an important research area and problem and that all of us together can contribute to so you want more people to work on these problems and that's our end goal great great are there specific examples of like case studies or like you know health centers or products that use this or yeah that's a that's a very good question so I think one of the case is is this deep dive by Professor Gilles tre from Stanford okay that's now been borrowed by Apple so this is this probabilistic databases which which kind of is statistical relational ai you know in some sense that that automatically extracted information knowledge extraction from videos and images text and so that's a classic case we have been working with hospitals on trying to see how we can put the data back into their learning system for making predictions on the hospital of hospital readmissions the number of procedures that need to be done on some persons all right we recently have got some very good success on using such models on predicting postpartum depression by looking at the network of women sorry network of people that the women are in touch with and so we are trying to talk to people to see how this can go out to the market we work with a particular Bank on on looking at their legal documents and figuring out if if if a new document comes in does this match the standards of the of the company of the bank and and we can do that automatically with like 98% 99% accuracy oh and the ones that we fail we can flag it and show it to the to the human expert who currently or so well currently looks at it so yes there's there's a lot of these case studies out there that but but again we are trying to do the deployment at this point so maybe in hopefully if I talk to you in two years I will be able to well I probably will not be able to say company a uses it but I can say there is a connector I started the same thing with recommendation systems the job recommendation system is actually a real project that we have done with the real company and okay and they are looking to use our product in their collaborative filtering system okay so yeah so there's a lot of success stories on this actually which is what we're gonna highlight tomorrow you should do a boy okay yeah awesome well sure and thanks so much for for joining me I really appreciate having an opportunity to learn about statistical relation on AI thank you any final words or places that you'd like to point folks to or anything else oh thanks um there is a book on statistical initially I of course that time Accord that off actually the four of us who wrote the book are the four who are giving tutorials tomorrow okay one and there's a series of workshops that happen every year and that the four of us again founded ten years eight nine years back I don't know eight nine years back I'm getting old and but now people are running at it's kind of self-sustaining in its own so I invite people to look at this problems of statistical relation I and contact any of us if you need any directions or anything that's up I'm very very happy to help awesome thanks so much thank you thanks alright everyone that's our show for today for more information on sriram or any of the topics covered in this episode head on over to Twilley Icom slash talk / 1:13 definitely remember to submit your thoughts on AI in your life at twill a Icom slash my AI and of course thanks so much for listening and catch you next time [Music]