Conversational interfaces for model understanding- Talking Language #5

Key Takeaways

foreign [Music] so with that I will probably spend just a few minutes talking about the next part of this talk which is natural language interfaces for model understanding so now that we have seen there are these different methods they all the zigbee and there are some ways to sort of navigate through that disagreement either by using reliable evaluation metrics or by relying on some of the theoretical results which clearly show the conditions under which each method will recover the underlying linear model so both of those can be employed when choosing a particular method for a context or an application right now coming to Jay's earlier question of okay so looks like there are lots of methods you know can we somehow get rid of this even question of like which method should a user employ can you just directly give me an answer that's likely to be correct do I really need to Wade through all these methods or can you just combine them somehow right in fact that's exactly what is being done by by this latest work that we have in addition to that we are also trying to improve the usability aspects associated with understanding models and all that is happening in this work and I'll just start by first showing this demo which will give you some intuition about uh what this tool is uh like so with this tool you can literally chat with a model in natural language and try to converse with it and understand what predictions it's making and also the explanations for different kinds of predictions it is making right so in this case when we say could you please explain these predictions it's saying that glucose BMI and age are the top three most important features that are driving the predictions of individuals whose whose age is higher than 20 for instance right and so on so this kind of interface uh which is both natural language uh conversation like interface to understanding a model but is also as you can see there is no more details about are you using lime or shap or what's going on it's doing a lot of that under the hood and I'm going to talk a little bit about how it is doing that and it's relieving the end user completely of the responsibility of picking a correct method and figuring out what is good and what is not right okay so we have just talked about all this which is there are several explanation methods today uh and there are also several implementations of these methods and different libraries and so on right so it's a huge hodgepodge of a lot of different things uh which actually adds to the complexity in fact we did a survey of different kinds of end users such as healthcare workers doctors and policy researchers and ask them about the challenges they face with explanation methods and also what is the desiderata that they would like to see in this field going forward right and essentially what they told us is the following so in some sense they are puzzled by questions such as which explanation should you use which method should you use because there are several options and they just don't know which one to rely on what to do again going back to our disagreement problem that we were discussing and then the second one is like okay if I pick a method which implementation should I use you know is it one Library versus other and they again have you know very little idea as to how to approach that problem and then the last one that they said was that uh whenever they were running into these kinds of problems they also had additional questions lot of times these methods would just provide a single explanation and they are done right so they're like here is Future importances and then we all move on but typically these folks have follow-up questions they want to ask maybe about another prediction or about you know points that look like this instance that they were thinking about and so on and again there is very little that they have control over in terms of like asking those questions uh because either they have to do that coding on their own or they have to rely on very specific libraries that may not be useful in other aspects and so on right so there are a lot of these kinds of challenges and they just felt like accessibility was one of the biggest issues when it comes to uh them being able to use these explanations and understanding models right okay uh so and this is where this idea of in fact one of the practitioners we interviewed in the study actually just had this anecdote of what if I could just talk to my model right so what if I could just or what if you could just talk to your model and that's where this idea came from to have this kind of an accessible no code natural language interface towards model understanding right uh and this talk to model uh which is the name of this tool that we built has two parts to it one is having this like language understanding engine which takes the natural language inputs provided by the users and then the other side is the ml explainability engine which basically picks an appropriate method and explanation that is most reliable and then communicates that to a user again in an accessible format right uh so I'll give very brief overviews of what we are doing on each site uh and then sort of show some more results uh so essentially what we had was this kind of here is a user utterance and you know what the user is typing in we have an underlying parsing model which sort of maps this utterance to some commands uh which are like input commands or the commands that will map it to specific explanation method and then we get our output in the form of oh these are the important features that are driving the prediction right and the main thing that we were doing underlying this uh is use language models in order to map these natural language utterances that uses input to some kind of a parts or a command that we will internally use in order to generate an explanation right and the way we were doing it was essentially we had a bunch of training data that we created ourselves which have this input utterance which the users will provide like the questions that they will ask and then you know this kind of a parse which Maps it to uh essentially commands that we can use internally uh and then you know sort of like try and uh fine tune a language model that can do this kind of mapping uh in fact we tried both in context learning as well as fine tuning and found that fine-tuning is a lot more effective when it comes to the language model kind of mapping a user utterance in natural language to some sort of a command or an internal representation that we could use right uh so in fact a lot of these parses were straightforward to deal with but the most important thing for us is to figure out which feature importance explanation to pick right so we could always for example when an end user says you know explain predictions for people younger than 30 we can map it to something like explain this or a command that says explain but which method would you use and in order to sort of work around that problem the way we deal with that is essentially look at this sort of a metric where uh we compute these explanations from the different methods for example lime sharp gradient based methods and so on but then we compute this metric on each of the explanations where we take the feature importances like the top K features output by each of the methods we put up those features and then see how much the model prediction changes and basically an explanation where the change in the model prediction is highest that is the best one right so let's say line says age and income are important right shap says income and debt are important now we take these and then in case of lines explanation we take age and income of that data point and change the values or massage the values of that feature those features and then see how much the prediction changes similarly in case of Sharps explanation we take income and depth values uh for that data point massage those and then see how much the prediction changes higher change means you have captured the important features correctly right so we were picking an explanation uh for which this metric caused the highest change in the prediction so that's essentially how we were automatically picking an explanation that is likely to be the most accurate right and we did a bunch of evaluations on this with like different domain experts in healthcare and also folks who work on machine learning such as ml engineers and Industry as well as graduate students working on machine learning and I'll just maybe give like one or two insights here given that we are running a little bit short on time what we found was that essentially this kind of if we were asking these users to do some tasks such as tell us what the model is predicting on a particular instance or tell us if a model is likely to be correct in its prediction and so on the accuracy on such tasks was much higher when participants were using the stock to model tool versus another common dashboard that is out there that's also considered a popular tool and again when we think about time spent for per question we see that the other tools were taking a lot more longer than the talk to model interface that we have right so here's more details and Dylan who is the first author of this paper and the student who let this work has put up a nice website so you can go look at the demos code and other things and play with it okay foreign [Music]