The DataHour: Explainable AI Need and Implementation

Key Takeaways

so now let me start the session uh still the speaker has not joined yet but let me give you a brief introduction so hello and welcome everyone to another session in the data hours series we are thrilled to be here with you this evening for a session full of action tag learning i am part of part of the data science team at analytics video for those who have joined us for the first time let me give you a brief introduction of the data hour sessions the data hour is a series of webinars conducted by analytics vidya and led by top industry experts it is a fun way to understand the concept of data science from the leading players in the data type domain so as the name suggests is just an hour dedicated to data we are hopeful that these sessions are going to be great source of enrichment and value addition to our community members now on to our session which is going to be today which is on explainable ai its need and implementation today ai powered by machine learning is mainstream in several businesses across verticals however there is skepticism in certain verticals on the power of machine learning to be able to perform complex tasks without a rationale of why is the machine learning model giving this result the area of explainable artificial intelligence was introduced to provide explanations for the decisions made by the machine learning model in this data hour nilima bugari mam will provide a hands-on intro to explainable artificial intelligence i hope you all are excited to attend this data hour with us well before we kick things off and i hand it over to our speaker a quick recap over certain things so we are recording this session and we will make the recordings available in a few days on youtube channel please use the q a session section for asking any questions you might have during this session or any question which comes up to your mind during the session of the data hour as the late hour progresses also we will share a poll about the feedback of the session towards the end and i request you all to kindly fill it up now on to our speaker in this session of data hour we who is the ceo at tara artificial intelligence and ceo at ellen shaw with us let me give you a brief introduction about her neil mom is season entrepreneur with extensive i.t experience of over 20 years she is certified data scientist with a degree in data science from john hopkins university maryland usa she is currently ceo at tara technologies a multi-artificial intelligence and machine learning competency and consulting firm she has recently been conferred the prestigious woman in artificial intelligence award which only which is only given to a select women leaders in data science and artificial intelligence my friends she is a strong innovator has filed three patents in the area of artificial intelligence and provides innovative solutions to over 20 artificial intelligence consulting projects for marquee clients for those who don't know the meaning of marquis marquis basically means well known and well publicized well now over to our speaker mam the virtual stage is all yours hey thank you thanks for the warm introduction and as as a representative of av analytics vidya has introduced i am neelima i have two companies one is where i am the ceo and that is tara technologies where we are fully into skill development and consultancy on the ki projects and aia and related technologies and the other company i'm into is ai ensured where our product ensured is into helping ai products build trust for the ai and there we use explainable ai left and right and that is the reason this topic of today's is of great interest and passion for me thanks thanks analytics video for giving this opportunity so shall i start the session uh yes ma'am uh you may start the session if you wish to or you can even start the session after two to three minutes so that we can wait for two to three minutes more for more participants to join if you don't mind i don't mind i wouldn't mind but uh hi everyone i have given a brief profile of myself right uh i would it would be great if i get to know generally if you people know what is here i'm not asking you fully hands on expertise but do you have a minimum understanding of what is ai and what is machine learning things like that just to understand the profile of yours yes yeah a good uh srinivasan someone this very same many of you stephen says yes nebula is wonderful and it also says that's great and this helps me understand you see when i'm talking to people who do not know yeah i may have to be a little more detailed that is the reason i was asking you guys yeah it is part of him in this automation wonderfully it's great yeah yes says yeah is to make the machine intelligent and smart and of course humans dumb that is the reason we want explainability yeah i'll explain that more in detail but we are dependent on their system so much that sometimes sometimes we become helpless but again we have to understand ai to make it more we have to leverage it here right for that we need to understand ashish is ml for sentiment analysis yes one of the applications of ml is sentiment analysis mostly typically sentiment analysis is done as part of text and using uh sentiment analysis is typically applied on the text well it can be applied even on the images that is still as a research area so data analytics using ml from great learning rajendra maestri says yes he is using ml from great language spontaneous that's good so keep your questions open and yeah once i explain explainable team here then probably even ask more questions yes ma'am so i think if you want to start with your session you can now yes okay that's good wonderful i see so many people 80 80 india every second is ticking and then i see more people that is wonderful thing so to start with the session let's have a big round of applause to yourselves all of you applaud for yourselves because you come to the right place to understand the right technology just learning ml and yeah at that time and then getting into job does not end the purpose if you have the real passion if you have the real passion attending sessions like this and upgrading your skills is definitely a wonderful thing you may not be able to upgrade your skill just by attending a one hour session right but this shows your passion to know more and probably if you're more passionate you may get into that particular core subject that'll be good yeah so big round of applause again yeah so maybe you can just say clap or you have clap symbol right that will help you understand you're really approaching yourself great someone has uploaded that's good the reason yeah 92 people have job that's wonderful this is this not just one but plus one are on your over effort so is it okay i don't know that okay rajendra yeah someone has raised the hand i guess right who's that yeah one of that and he says raise the hand ashok is it is it by mistake or what is it like yes ma'am that person might have raised his hand so guys you can raise your hand using emojis you can wave your hand using emojis in order to interact yeah yeah did it by mistake that's that's fine to worship wonderful yeah uh that is the most important thing that is needed when you are trying to learn something new okay you should not have any inhibitions in asking questions and yeah yes hands on is also part of this webinar vijay lakshmi but again due to the time constraint i will only be doing hands-on on my system and you will not be able to do it on your system because uh there are many installations that are part of it of the simple example it's not a simple example it's a it's an example but for you to run it on your system it will take good amount of time so one hour may not be enough okay that's good so we have hit the count hundred so shall we start the session now okay the most important point i want to tell you is in machine learning i know most of you have learned machine learning right you can represent the data in multiple different formats whether it can be in the form of images or it can be the form of text it can be the audio it can become a video or it can be in the form of structured data or it can be a simple time series data like symbols like now whatever is a type of data machine learning algorithms will detect the type of data and understand and derive the insights and use it for the purpose of the business it it may not be i mean when i say it's a business it need not be a business which is earning money it can be it can be the target whatever you're trying to do ah i will share raju just i'm just giving out so in some time i'll share the screen definitely that's good in the meanwhile analytics uh i don't i have missed out your name whoever is anchoring yes my name is but would you please stop your skin share one yeah so in ai which is again machine learning is part of here right all of you some of you have smartly answered that before when i ask that but what do we do we derive insights from the data to derive insights and then sometimes you just stand by there in creating dashboards and these dashboards will be helpful to understand the data and use it for a business purpose but many times artificial intelligence especially you're not stopping at simple data analytics instead you're applying algorithms on top of it to build models ai models these are models are the softwares or the softwares right effectively these air models when you apply algorithm on this data on the data that is given to you which could be relevant again these build air models these models are the softwares on which we rely we give input and then get output we just trust these models right how do we trust the models because of experience probably we have known that these air models give good output example example when you're using google maps if you have to travel somewhere from from i'm giving some location in bangalore because i'm from bangalore let's say i'm traveling from banarata road to dairy circle or majestic i'm giving some location if you have to if i don't know the name if i do not know the truth what i would do is i'll just trust the google maps and give the location and it would give me the output whether the output is optimal or not i only trust it because it has been working from long time many people are trusting it whether i'm okay i'm okay even if it shows me root which is five minute slower than the actual optimal route but it claims that the route it has shown me is the optimal book even then i'm okay with it right that's okay five minute delay is okay for me but sometimes it happens that when we are relying on ai models like this like the way i have given you example of google maps right the way we are relying on google maps in the simple example that i have quoted in this case probably i am losing a time of five minutes that is okay but when we are relying on models where it will predict my life like a products which are used for detecting cancer if just an x-ray is given as input to the ai product and the product will detect whether i am cancerous or not and if i have to rely on it how trustworthy should the model be don't you think the model should be trustworthy right i want an answer here please yes bhusha says yes wonderful that's good if only that is that happens only if i completely rely on the model now the question of how trustworthy is the model how do you know the model is transferred or not i can't say let me use the model for on thousands of people and then can come to a conclusion whether those were trustworthy or not right even if there is a loss of one person that is really very risk-taking so here comes the need for explainable ai so we need an ai which actually explains what the model is predicting and how the model is predicting so to list it out explainable ai actually helps us understand what the machine is learning okay model is learning not the machine but the model and what part of the input data is actually important and and also how the model can be built more robust when we look at it in a data scientist perspective because it is we who build a trust for the a model right it is not the end user end user is only using it but vs data scientists have all the responsibility to build a trustworthy model so that at a time when the user wants to know how the model is working we we should be able to explain it and for us just for the explainable ai is a tool to go ahead okay trial and error yes perfect which i like me in the case of google maps trial and er works perfectly but are you okay to go for the software which which will predict whether you are cancerous or not on a trial and error basis no right because that is putting our life at risk let's say you go for a product which will just take your x-ray and send predicts that you are not cancerous and then would you stop it there i wouldn't have done that because i love my life right election that is thing calendar works good when it is more like maybe maybe recommendation engines on e-commerce it's okay even if the either the recommendation engine on an e-commerce site is actually hinting you to buy some product which you may not have actually liked it that's okay even if you buy it you probably lost thousands or hundreds of rupees but what do you mean by many algos vijay lakshmi here i didn't understand many algorithms yeah i understand model outcomes often convey confidence level along with prediction yes now coming to confidence levels confidence levels are only limited to the data with which you have tested right alec right is it on all the data is it in all the possible data now when i'm let's say i'm i'm building a product and i have uh typically unless i have one lacrosse so i'm only relying let us say i have split the data into 75 percent for training and 25 for testing now out of one lakh i've used 75 000 for building the model and yeah data must be given you're perfect data must be given but if the data that is given to you does not cover all the corner cases don't you think you're a trouble right the 25 000 data that you have does not cover all that corner cases you are not sure if it covers or not as a data scientist you are not sure you because that is not your duty you say you are only relying on what that client has given right why do you worry about it but there are techniques to understand that also again our product a insured that does does that so that is a reason i mean you can go to anshu.com to understand a little more on that but coming back to this part if you have explainable ai then you run the model when input is given to the model and then output is generated you clearly understand what is that part of the data which the model is you will see it when i'm running at last at the end of the session so don't worry just keep your questions to yourself no you can open up you can ask me but again if you're not clearly understanding how is that the model is focusing will will understand which part of the data is important that part you can see later yes services more the data better the prediction but again remember here that is a wonderful thumb rule more the data better the production but the data should be relevant the more the relevant data the better the prediction perfect again when you say relevant data the data should be covering all the corner cases that is important yeah a in a insured.com you just type dot com there and www before that we have built a product called yeah ensure which handles the security privacy and explainability of the products and right now it is only we data scientists who worry about explainability because we know that ai is not perfect it is only probabilistic right the output is probabilistic and we are also worried when we use applications for let's say for detection when you're doing doing fraud detection also if you have prop yes sashes that's perfect yes so mushy and stiff sugar pressure that's the right one there so when you're using the air model uh you know it is probabilistic it is not hundred percent sure but again you as a data scientist how much can you do you can only do whatever is the possibilities within your limits the data that is given to you will work on that and yeah you will see that if there are explainable i mean i'll show you how to use explainable a when you have the tool of explainable ai explainable ai is not a tool there are algorithms i'll explain it to you okay ma'am please share skin i'm not able to relate okay perfect travel till now i was only giving the introduction now i haven't shared the screen or those of you were thinking now i'll start my screen share again this is a powerpoint presentation you will be able to understand it even when i share the screen now okay yeah yeah can you see my screen is my screen visible right my screen is visible right perfect i'm not seeing slides are you able to see that here now come on champ manish rahul okay perfect yeah so today we are explaining i mean i'm going to talk about explainable guy as i told you machine learning is the core because of the recent advances in science and technology every other company you talk about uses machine learning or ai to either enhance the product that it has or build a new product because machine learning is giving is working like magic and why is this happening this is only happening because of the just once again i want to check the time so that i know okay fine okay ma'am i have no i will not be able to share the powerpoint yes okay yeah you got a response then you can see the recording here so why is that we are able to use machine learning from long time machine learning is there from 1960s okay artificial intelligence has been there along with machine learning from 1960s but it's only in the recent past that we have leveraged machine learning quite a bit the reason is data is exploding and you generate data heavily now i give you a statistics which was around four-year-old facebook alone was generating two million bytes of data per minute four year back not the recent status now it is predicted that yes along with the data generation advancement in computing power has added to that and that is the reason we use big data technologies on top of the data using artificial intelligence to build wonderful projects everything is fine as long as you see the project is working the model is you created is working the way you want but but when it does not work then comes the question why is it that it is not working when it's a simple example like e-commerce recommendation engine that's okay that's okay maybe there must be some intricacies uh there must be some relations within the data which has made the recommendation engine to suggest that right if that is something like a recommendation that's okay for you but if it comes to an example like cancer prediction or a fraud detection if the production software that you have implemented detects a an eligible map or an eligibility a legal transaction is a broad transaction don't you think it is it is it's bad right even if it is in fraud that is there and it was not detected as fraud then it is a huge loss right so this is what falls positive yes false positive and false negative both are very important for us to have it right yeah computer vision i'll come to that i'll show you an example on that okay now use it so we as data scientists try to do that and users are asked to trust the model how long will the users stress the model as long as it does not hurt them and how do you know if it doesn't hurt them even if it does not hurt them sometimes when it comes to health care or finance of course users are proactive because they can't risk their life or they can't raise their money and reputation right that is the reason we have to go for explainably okay now like one of you was asking right i don't remember the name accuracy aggregate measures and there is uh there are many measures after you build the model to tell you whether the model confusion matrix is accuracy and the great measure these are all part of concrete matrix these give you whether how accurate is your model but but again if the data that you have taken does not cover all the corner cases what is the data that you've taken the test data that is data is a part of the data that was given as input to you right are you generating data are you generating test data are you making sure the test data has all the corner cases no that is the reason we cannot at all rely on accuracy and aggregate measures or the configuration matrix or any of the statistics yes to prove to show it to our client maybe you use that but again that you cannot rely at all okay and now i told you the need for explainability but there is also whether we need it uh when we whether we think it is needed or not now there are regularly compliance standards that are coming across the world in multiple countries which are mandating every software every as of the company which is using it to be able to explain the relation between the input and the output and when this comes it is only extendable layer that you have to rely on again i want to tell you one other thing you would have heard about auto ml products right site google has come up with automl where ml is generated automatically so going forward it is possible that it is possible that data scientists will mostly work on explainably behind the actual level because actually ml is anywhere we have tools like automl that are there which take the input and understand there are some complications there and which builds an aim so we have to rely more on the explainability going forward explainable layer is very important okay yeah now when i say explain play there are some ex some explainable algorithms like decision tree linear equation random forest how can you tell me or this are these deep learning algorithms or simple algorithms hello can you tell me what is decision tree perfect they are simple algorithms right you know what it is when a decision tree is built i'll even show you the screen of decision tree a little later but okay yeah so this is decision tree i am unable to my money but there is some pop-up that has come on to my system i don't know i'm trying to close it okay perfect yeah this is an example of decision tree i know all of your data scientists so you know what is this right decision tree takes data and understands interprets the data and comes up with tree like this based on the tree structure it understands what the output would be when an input is given from this the rules are built in the simple decision tree that i'm showing here if the outlook is an overcast is overcast then it will definitely rain and if the outlook is rainy and windy it will rain too the outlook is sunny and the temperature is either mild or cool it'll rain too right so that these are the rules that are built so this don't you think this will help you understand why is that the model has predicted whether it is rainy or not whether it will rain or not do you think so uh geetesh just hold on i'll come to that i'll come to that i first have started with decision tree yes okay it looks like a project perfect yes even linear regression and logicalistic regression are explainable here they also use the equation okay i'll come to that part and also tell you why it is okay yeah we have seen dictionary now let's go to linear regression linear regression is the first regression algorithm that you would have learned by default in linear and logistic regression right these are called interpretable algorithms where there is an algorithm that a model mark algorithm there is a model that is built and the model is something like this this is an example where i said 3 x 1 plus 4 x 2 minus 6 x 3 minus 20 will give me the output of y y is the output and x 1 x 2 x 3 are the inputs this is an interpretable model it will also explain me why exactly that what is the relation between the different input and why is that i am getting the value of y right so this is already existing this is also an explainable model decision tree is again an explainable moment i think most of you would have known about random forest right what is random boris what is random forest yes it's an ensemble of decision trees thousands of decision trees like this would be there as part of random forest along with that so when we say decision tree is explainable random forest is obviously explainable because random forest consists of an ensemble of decision tree and it's not just that you also have in decision tree different variables and what is there uh there are different features right which feature is actually adding more value that will see that this again these are all self-explanable algorithms but if you noticed if you noticed most of the problems that we are solving uh i know most of your data scientists but again uh because i i interact with the clients and we get many many requests from kind we handle we do consultancy and projects for uh yeah on ai so there are many times where we have shied away from using deep learning and instead limited ourselves to using random forest you know why because of the lack of explainability now in deep learning we have line that is what we are going to explain a little later i'm going to explain but again line is not self-sufficient in explaining all the data types there are some complexities in some of the cases it is possible okay so i think did you all get it till now we use linear regression decision tree ma'am i am an m tech student from mechanical bank but wanted to shift my domain we'll talk about it a little later i don't know yeah rahul okay we'll talk about it later let me just first finish that okay now coming to feature importance in random forest along with the fact that random forest is an ensemble of decision trees it's also that we can understand the future importance like in this example we have gre gpa and rank okay now if you see the graph here if you see the graph here it shows one is what gpa zero is gre rank is two from these these are featuring important graphs that are there as part of random forest we can do that and from this we can understand which feature is of highest importance so that also helps us understand right why is that the output you're getting is also so feature importance can also be explained can you can we use bootstrap sampling for feeding the data into the model for classification issue we can but that doesn't solve the problem but random foresting have a thousand pages wouldn't be considered a black boss no it is not a black box random forest is not a black house when we go to deep learning it is a black box and there we go with live approach decision trees fundamental resource perfect yes that is the reason that is the reason we are going to talk about lime tree sorry lime yes we can use plot tree method yes we can do that but again the limitations of using random for random forest is the best among the decision trees and linear regression okay when the data is complex when there are some situations where we have gone for deep learning there is no way out because the deep learning works the best over there though it is a black box and there we use line and we have our own our own uh product air and shooters tool right okay now any questions before i go there i'm open to question yeah okay that when i don't get any questions in the chat i presume you understood it if i'm not answered because i see many of you were asking questions sometimes i would have missed out but we have around 15 minutes right okay nor is it easy to understand how the different pieces yes nothing is easy yeah but there is something existent right rajiv this is for you okay it is not easy exp model itself is a complicated one but there is a tool for you to understand so using the this features we can develop okay and we can come up with nothing is easy as such because it's a complicated model when artificial intelligence in artificial it is trying to mimic human intelligence so things cannot be so easy but there are tools and techniques which make it simpler for us and i'm only talking about those tools and techniques here yes we can see that plot underscore tree method is uh when to choose what models map yes value i'll come to you there are ways there are there are model agnostic explainable tools like lime lime does not rel is not worried about which model is used in the mod which which algorithm is used okay it is model agnostic and there are also local and global explainable ais random forest has how many types which are there okay uh random forest you can look into but random forest has is a set of decision trees okay ma'am can you put some light on hyper parameter tuning hyper parameter tuning there are you know about hyper parameters right when you tune them then obviously we the word tuning means that you are making the value of the hyper parameter so app such that the output is more relevant to us okay i cannot because of a limited time i may not be able to explain you what exactly but energies i will tell you uh aren't you using macbook yes i am using ashish uh is interpretable different from explainable yes explainable ai only explains between the relations between input and output i'll show you interpretable see some models are interpretable like our random forest decision tree and linear regression when it comes to explainable ai uh i'll show you an example on the image then you will understand it better for future selection other than statistic analysis do we have other techniques too it is mostly statistical analysis and about domain expertise so any ai application we need to have a domain expert okay can we consider auto model is the best method not really auto model is never never ever the best method when you rely so much on automobile you should be ready to take risk so no definitely not but yeah when companies like google come up on that there will be a good amount of research there will be and auto model is probably helpful in simple regressions like where you're using linear regression or logistic regression where human intelligence may not be of greater value it automatically generates inter what the different graphs that show the relation between different attributes okay how do you define the values of higher for the hyperman uh we will define again that also needs expertise and we need to there's no hard and fast rule over there okay now let us come back to line what is line line is locally interpretable model agnostic explanation when it is locally interpretable that that means that we can understand we can apply explainability on one single instance and it is more like agnostic explanation when i say model agnostic it only means that there is no hard and fast rule that the model has to be built using this algorithm or that algorithm so line works on any of the algorithm any of the models that are built on any irrespective algorithm that is used in the model building what is life okay its objective is to explain the result from any classifier so that the human can understand individual predictions perfect line test what happens to the predictions when you give variations of your data in the machine learning model line generates you know what line does it along with the data we only give data to the line right input data and then the output that is generated but line actually process it such that it it actually creates divisions in the data okay someone was asking the question x explainable has a method value sharply values how lime is different lime is different from shapley there are these are the two most commonly used explainable models but today i'm only explaining lime shapley is different you see if i ask you how is svm different from i give some other algorithm time features are different right it is apt for us to understand which algorithm is better only based on the context and the data on the dome on the problem you are solving very similar to that okay lam is part of deep learning is lime is part of uh ml such as regulation forecasting using different timely guidelines about that uh i would like to tell one thing that you can take questions at the end as well so that your presentation also doesn't get hindered you know because uh students or the uh the attendees will keep asking questions but you can continue with your presentation you know because there is time bound as well yes as much ali is also saying the same thing so that's good okay fine so these use the data is split into and it's like permutations and combination the data if you take an image the image is cut into different pieces and then it will create new set of data and the output it maps the data input data to the output and comes up with a model uh i'll show you how it does don't worry when you see this yeah the original model is approximated by simple and interpretable use usually a linear model you don't need to get into the details of line when you're working on it so i'm just giving you what is done internally okay simple model is approximately for the input we are interested so it is said to explain the original complex models local behavior not the global behavior when you say local behavior local behavior is only applied to the data item that you are working on global vp behavior when you say global behavior it is like applied on all the data on which it is processed so we'll work on a real program line where i apply line on an image and show you how it works so shall i i'll show the program okay and and after that let me open up the questions is that okay close your chat box so i'm stopping my screen share it's yeah is my jupiter notebook visible now i will not be able to share the code john okay um i have my constraints there okay this is my company code and yeah but i'm showing an example that should be okay i see uh all of you know right i'm working on a google collab on my google collaborate basically so i'm mounting that that you know right that's the first thing i'm doing and after that i have my program there so i'm doing that too hmm they have got another hero can you give me a couple of minutes i'll just stop my screen share fix it hello hello hello yeah and there my program there is an issue there i'll try to explain okay my code is open but the program is not working because there was a crash so i will just share my screen i'll explain it to you okay yeah no worries you can explain it without running without running it yeah yeah exactly but you need to have the images only then you'll understand it better so i'm trying to see the images it's collapsed [Music] that was here i will share my screen in just a minute i'm not going to run it i thought i will run it because i'll just share and then show you the output i wish you will understand it okay now this is a simple mounting ah the only thing is there was not much of a gap and before that what exhibit i did this so google collab will give this error to me okay and now i have another instance where it is working i'll explain this to you and then come to you there okay ah now this is a simple part where i'm mounting the google drive and then because my files line that is my program is here i'm going to that location okay and then i use tensorflow after tensorflow this particular tf underscore slim package is important which i have installed i'm also installing line okay i would have run it but now i have to take a different uh drive and then do that first i'll explain it to you and then from lime image let's see line for images detail line can work on images it can work on text data also now i'm trying to show you the example on the images okay now we are loading a pretend slim model this slim model is a model that is built which will identify dogs and cats in an image okay this is a model that is already built i'm not going to touch it i'm only applying my line on top of this so i'm here loading the model okay after that i am applying transform image for inceptions so hello can you hear me yeah okay uh i'm actually transforming the image because when i apply lime i i need to have some transformation so there is a method for that which i have done it after i do that okay after i do that what i get it is i can get the predictions for the image now this was the image that was given as input to the model the model that is already built i have i'm not using lime at all okay this is an image that is given as an input to the model which detects whether it is a dog or a cat now i am only taking the top five predictions what is the first prediction here can someone tell me it is bernie's mountain dog right what is the first prediction yes but it is bernie's mountain dog please read the blue thing yeah name of the dog yes it is 73.19 there are different types of dogs right the other one is ant labor and all the top four are predicting that it is a talk but the fifth one says it's a cap though the percentage of prediction is what we rely on right generally the more accurate you look into accuracy there taking taking into consideration that we have considered all the corner cases and the data is relevant and stuff like that so if you go with this model we will probably predict that this is a dog the image of this the image of the image this image is of a dog but the fifth prediction also says uh it can be considered as a confidence yes but again you cannot fully rely on it it can be considered the confidence only when you are sure that the data is covering all the corner cases the data is very genuine and and covers all the corner cases code repo this is the code republic probability same yeah okay now if you take this i'll explain you how explainability of line comes into picture we'll just scroll down basically i want to look at it this way the top four predictions are saying that it is a dog the different varieties of dogs yeah but both are appearing no man what yeah both are appearing but why is that one of the predictions uh jai lakshmi you raise your hand i'll come to you i'm taking screenshot for later no issues you can do that no issues here i'll come to you jai lakshmi right uh so he's asking velour cesar kumar is asking but both are upbringing yes both are appearing but why is that our model is saying that this is dog and not a cat the first four instances it says only dog right and the fifth one says it's a cat it is just a simple dog and cat scenario but we should understand why it is like that so if we come down now this was the model with five predictions now we are applying line here and for that again you use numpy and your processing image by line this method where i'm giving the image as input and processing it this will take good amount of time now i also say show lime expectations for image and i have given idx equal to one here idx represents the top five outputs okay and when i'm giving one i'm worried about the explainability of the first one see i'm worried about the explainability of the first one right so when i apply line when i apply line it says that it is only looking at this part of the image and that is the reason it has predicted it as doc okay now when when you look at something you probably look at one part of it and then decide right as a human being also but it's not just that one part you probably look at many many more things if i if i if i give a screenshot of uh let us say what a sofa maybe rectangular shape so far the image classifier we are not sure if it classifies it as a sofa or a bed just because it doesn't know the size it it is only looking at the shape but where is that it is looking exactly it is looking at this part and so now you understand why is that why is it why is it that my model has predicted this image as a dock because your model in the first case where you said idx equal to one your model has considered this part okay now this part is again in the whole image this is the part that is the reason it has predicted that it is a dog and not a cat now let's go to the fifth prediction where it is cons it is counting it it is predicting it as cat right so let's see what it is now i say idea is equal to 5 and then when i give 5 it has highlighted this part which is the part what is this this is a cat right and so your predict your model is predicting it as cat now coming back to if you can share the lime zip link it will be highly appreciated for later reviewing lime ziplink okay i will share it i'll share it with analytics with your team okay that you can okay now one of you was asking right ma'am it has two right why is it predicting only one right who was the one who was asking that question the image has both cat and dog why is it that it is predicting only whether it's a character one of you was asking that well basically our model that is written uh well zip file ashish i'm not sure if i can i'll talk to my team okay whatever is possible i'll share with you okay don't worry i'll make sure that you understand it i will talk to my team and i may not be giving you the overall code but the code this particular one i'll be able to give it to you i'll uh analytics with your representative i'm sorry i forgot your name okay i will share it with you can you share it yes ma'am yes ma'am my name is so guys don't worry uh we will share everything which we can we will coordinate with mam and we'll share it to you later on so don't worry you will get it yeah okay and now uh one of you was asking that question right why is it that it is predicting only image because the classifier we are calling the classifier here right here print drop it drop print top predictions per image this is only the the code inside this particular logic or the program is only classifying for one image not all that is the way it is designed yes it can also predict for blurred distorted images yes but maybe the prediction is not correct and so if explainability is there let's say this part this part is blurred or maybe the dog was blurred let's say that though there is a dog as a human being you are able to understand that it is a dog and a cat together but this was highly blurred then probably it is possible that even the first four uh predictions say it is cat okay below say okay now do we have time i will try to see if i can execute it we have only four minutes so i am trying to see if i can execute it i have it located in a different drive too i can log in and show it to you is that okay or you are done with it no i'm you can go ahead i'm stopping my screen share i'll try to log into the different drive and share it yes apart actually i have joined i've given take consider i've joined the same link with a different id as a participant so can you give me the share access over there as a participant yes ma'am uh i just did it will take a minute in this meanwhile someone has raised a hand right what is that who is that yeah tell me reporting in progress yeah i'm now on two systems so i'll share the system yeah now i'm on a different system can you [Music] yeah i know i'll just be little yes yes now it's okay so here i'm executing it okay i'll be little slow just to make sure that this previous error doesn't come into picture here i'll put it as in a different folder okay lime explanations for image classifier nice it will take time yeah and do you have any buffer time but okay fine yes ma'am you can take a few minutes if you want to okay so this part you've understood right now i'm mounting the drive there i'm going to this particular location on my drive where i have the code and that is the reason but again i'm printing the dir now i come to importing tensorflow and printing the version this is important where i say tf underscore slim this is mandatory yeah you can take screenshots and if possible i'll give you the code also but otherwise no issues even if it takes the initial that's okay yeah now i'm installing line okay now we are loading the preteen inception underscore v3 this is an image classifier available this is a pre-trained model the model is already existing you understand that when i say pretend okay i'm loading that just loading that because i have to use it oh i get an error again this is because i think let me let me try problem is with the resource variable the problem is i am trying to access it from a different system also right so i'll just explain it to you again here we are transforming the image and giving this image as an input right see one of you was asking can you show the location one of you was asking where are the what these images present right it is there in my drive here do you see my dog dot dog image here can you see this dot dogs right that is what i have loaded i have transformed it and loaded and then i'm just displaying the image that's what you do right once you load you check to see if it's correctly loaded or not i am doing that and then and then this is a method that is there already in the load and model where i am trying to print the top predictions again here i have mentioned it as five i cannot change it because you know there is an issue it's not working the way it has to now when i give five yeah it is only printing the top five i have limited it to five hours or i have taken five just because the top four were predicting it as a dog for me and the fifth one was predicting into the cat i wanted to show you why is that it is showing like that that is the reason i've taken five again contextually you can change it not initiative after that i am importing line okay sorry numpy you import numpy and you process the image by line so you give the image and then number of variations you can specify the variation that is 1000 times the image is processed when you specify the as i told you right every input that is given will be divided into multiple pieces and then uh aggregated together and then it is mapped with the output just to understand how the different parts of the input image in this case or input text are relevant to the output how they are mapped that is what it tries to do and for that in this case i have given number of variations as thousand the more the number you give the better it is but you know the more the number you give sometimes it happens that it if you give very high it runs off sprout runs out of space and also sometimes it comes to a threshold value so you have to limit it at that level and with experience our team has realized that thousand is a better number for this [Music] and when i sorry i'm sorry just it's not in my hand i i live in an apartment complex here i think it's equal to one when it space id is equal to one that is the first prediction i'm giving the first prediction i want to understand i want to know the what is the explainability for the first prediction and then it says this is the explainable game have you noticed it is only looking at it is only looking at the dog image right two parts of the dog face that is the reason it is predicting this as dog now let's come to the fifth one and the fifth one here says it is looking at only cat right that is the reason the fifth iteration here it says it is an egyptian cat there are varieties of dogs and cats right it's an egyptian cat is what it predicts us so this is the way line works i am going to share all the code with the image because the image is like that in spite of that in spite of that ashes have you noticed the in the model is able to predict it correctly because both the images have together in one single way dog and cat are probably sitting together okay how do you train the model here do we do we provide multiple images yeah we provide multiple images because the model that i was talking to you about that is a pre-trained model where we have trained it for multiple combinations of dogs and cats it's not just the combination of cats and dogs different images of cats different images of blocks also it's a pretend one i'm not explaining on that part right so why didn't touch it okay feature what is what do you mean by feature visual experience if the image is mirrored will it record yes it will uh yeah i'm sorry yeah we will try to mail everybody does the image have any labeling no who is that muhammad it doesn't have any labeling if it if it has labeling then obviously you don't need to worry about it right there is no leave in here that is the way i think you would have all worked on image detection when we are training very little label but not now muhammad if you are asking about the training time uh yes when we have trained we will label all the images cat and dog will label them but when we are testing we will not really believe okay did it answer the question uh as a hint what is the kashish ma'am give us more details as in what intel do you want me to give please explain more about concept and working of line uh madhu it's like this now tell me one thing uh if when uh when you are trying to explain let us say dog to your kid i don't know if you're married or not but a small baby who is just two year old and the way you explain it to him or her would be have you seen have you seen the ears of the dog if it is like this there are dogs right you will you will try to explain him part by part do you do that madhu right every part you will try to explain him if it is like this you can say it is a dog yeah there are some cases where it may not be dark you will give him explanation also it is in a similar fashion lime does lime learns it by itself now this dog let us say this image is there right what it does it cuts these images into multiple parts internally programmatically and then in this image let's say i had another image let's say i had an image of a dog and i labeled it as dog because when i'm training i label it right so it understands if the nose is something lik