Why Machine Learning Projects Fail In Production

Key Takeaways

my name is Krishna come welcome to my youtube channel now recently based on the survey done by Kegel last year and a company called as Gartner they have actually found out that 85% of the machine learning project usually fails after you put them into the deployment so this particular video will be discussing about that only why machine learning projects right after you deploy it into the production film usually fail what are the common reasons right we'll try to understand this and we'll try to also understand some of the scenarios which leads to this particular failure scenario basically so first of all the main reason why machine learning project fails is that first is always important that you will not be having sufficient data now suppose I want to solve a business problem right now there will be three teams that will be basically involved I'll not say three instead I just said two so one will be the business requirement team right who will be actually creating a business requirement there will be some domain expertise that will be involved in that particular team to discuss with the team that are basically solving that problem so that disappearance data scientists data understand many more things right so when this discussion is done first of all if the communication is not that good right if the combination communication is not that good if the collaboration is not that good between this particular team definitely that all requirements will not be captured by the data scientist team right and it is very much important guys because those requirements are most at most important because in that particular requirement the data scientists will then understand that what kind of data needs to be collected right so if this communication and collaboration is not proper definitely most of the things will get missed in my personal career I have seen this particular scenario when we are not able to get a proper requirements from this business team at that time the project is just extended because unless and until we don't understand it data and just understand guys a single data scientist cannot be a domain expert in every field right tomorrow he may get a problem statement in some specific domains and next project it will get some other projects with respect to some other domain right so it is very very important that the communication and collaboration between these two team must be very very good if you are able to capture this particular requirement properly right properly then what will happen is that then the data scientist will know like what all data is actually required to solve that particular problem and if we miss even a single important feature that may lead to you know decrease in the accuracy of that particular model so this is very very important so the first one I'll say communication and collaboration of work between various team members is very very important and because of this I have seen many machine learning projects fail in my real-world experience with them that basically means the work that I am doing in my day-to-day life right I have seen this kind of scenarios now the other condition is that even though we get the business requirements some time it is very very difficult to gather all the data that we require you know we may be dependent dependent on some third parties data some other data right and it is not possible that I we get the complete data at once right we have to wait for that sometimes we do not get that particular data also and privacy is also one very very important issue right privacy of the data is also very very important I am just not talking with respect to machine learning I'm also talking with respect to D planet right so if you know that there is whole lot of face recognition algorithms that were basically coming up right in deep learning and at that time there was some privacy issue concerns that was raised already so this is very very important and obtaining the data or collecting the data it is very very difficult guys it is not that simple and you also need to understand with respect to the business like what all data is required right so this is very very important the third important thing that I would want to say even though the data is collected it is not 100 percent accurate okay now let me just give you an example suppose I have made some I've done of some survey and some people have filled the form right and in that particular form you will be seeing that some of the people will not put some information they will put some wrong information also and those kind of scenario may happen so suppose if you have a 1 million records of data that you have collected and out of that forty percent of the data is just they have despite something right so those data is not that accurate right you know if you don't have that accurate data it is not possible for you to create a model with high accuracy and because of that most of the machine learning model will fail after you put them into the production environment because the new data that will be coming suppose if it is a proper data and those kind of data is not already captured right in your training data set or a test data set right so at that time those kind of scenario will give - will lead to a decrease in the accuracy so it is very very important guys the data collected should be more almost at most accurate at least you know more than 80 percent from that whole data set that you have right so this is very very important thing now the other thing is that data like you know that data is getting collected more and more data will also get generated more and more right now when the data when you have a huge amount of data suppose initially I just have 1 million records I've trained my model and after training my model have deployed it right then after some time still more gen data will get generated right more data will get generated it will never stop right and some of the data will create it will be created in such a way that they may not be present in that particular data set which we have trained our model so for that what we do is that we continuously train our model what we do is that we see the accuracy for every month and then we if we see that if the accuracy is deteriorating right we try to increase that data we try to increase our training data more and more now you should understand one thing guys as the data volume increases and you're trying to train your model that mature effort will be taken place right you have to put more efforts to train your model and with respect to that new data that has got added right then that may lead to you know a decrease in accuracy also because the algorithm is not going to capture some more new information from that some more features may have got added right so all this kind of information comes into picture when we are actually having a huge volume of data there is one very very important term that is my fourth point that I am saying and this is basically specified by most of the data scientist the term is something called as garbage in and garbage out that basically means that in the complete data science in the complete life cycle of a data science project feature engineering plays a very important role now this particular term that I'm saying garbage in and garbage out will be respect to the feature engineering if you don't you know clean your data if you don't make sure that the data is in the right format before passing to a machine learning model it will definitely give you a bad accuracy now see the term guys garbage in garbage out if you are giving a garbage kind of data you'll also get a garbage kind of output so this is pretty much very important to understand right so it is much more necessary that you do the feature engineering process you do the feature selection you invest most of your time into that particular scenario and you do a whole lot of work with respect to that if you are not doing this particular stage properly guys I will definitely tell you that whatever model you generate after some time after you deploy that into the production that is definitely going to fail okay because after some time it will start deteriorating like the curacy will be going down and down and again you have to retrain your model with someone new data you have to go back again to the future engineering stage and do a whole lot of things so this is pretty pretty much important again there are some more points that I basically want to discuss is that lack of domain knowledge now when we are considering domain expertise also since I've seen many domain expert II right when we are talking with respect to that particular use case he may be lacking some more information right if that information is not conveyed to us and if and that information may be a very critical information to design our model and if that is not conveyed and if that data is not collected for that particular information right definitely we will be getting a bad model after all right so business domain expert II I mean the domain expert it plays a very very important role so the domain expertise should have a huge amount of knowledge with respect to the use case that is solving let it be any kind of use case right if he is able to provide all the inputs with respect to all the scenarios that are involved data actually leading to give you the dependent variable output and if he's sure that this many features will actually correlate to that particular dependent feature if we are able to find out that particular information our model will definitely good accuracy but due to lack of domain expertise knowledge disturb the whole information is not basically captured that is the most important thing the last point that I would like to say is that which I have written again over here according to the Gartner survey and cattle survey also they have specified that 40 percent of shortage of data science engineers now this is the other major thing now your domain expert is fine right he's giving you the perfect data but still do to qualify data scientist right there's a there's a whole shortage of qualified data scientists in charge even though you don't have if if you are a data scientist and if you're not that good right you may just use the data you may just create a model right but if you don't know how to work with respect to that data in depth and definitely your model will fail after some time initially you won't be able to see that but after some months right when and always remember guys you always have to create a generalized model whether the model is in the training or in the deployment stage it always has to give you a good accuracy so there is a shortage of more than 40% of qualified data science in review and whole lot of you know recruitment is still going on data scientists and people are looking for more qualified dataset it's not only qualified guys I'll just say that those who have good experience those who have done very good work so these are some of the basic points that I've actually mentioned why your machine learning projects usually fails and if I say about my work life experience I mean the work experience that I do I've seen this kind of scenarios I've seen some projects moved to n number of days like it has just been postponed to n number of days because we are not able to get the data that we want because we are not able to get the information from the domain expert II for that particular reason and because of that you just projects are getting extended and extended right it is getting delayed so this is all the points that I've actually mentioned please be free to put up your points in the description box or in the comment box sorry and definitely I will have a look onto that and definitely I'll also try to reply you back in that particular comment that you have put up I hope you like this particular videos please to share my Channel please to subscribe my channel and come up with more interesting media in my next month I'll see all there have a great day ahead thank you one and all bye