Why Automation Will Not Kill Data Science Jobs?

Key Takeaways

hello all my name is krishnak welcome to my youtube channel so guys today in this particular video we'll be discussing why automation will not kill data science jobs now this video i had to upload guys because recently i uploaded a video on exploratory data analysis with the help of an amazing automated library which is called as detail and as soon as i uploaded that video many people started commenting because that with the help of those libraries i can just write a one line of code and i can do the exploratory data analysis that basically means we'll get all the visualized reports statistically right comparing all the reports will be easily able to just get with one line of code right so many people started pinging uh in that and even they started commenting in that specific video then what is the use of data scientists you know will data scientists lose their job because this kind of automated libraries are actually coming and i think this is one of the most major concern many people are still having right automation may impact some other industries you know with respect to some other work but the thing the point that we are going to discuss you know it will not impact that much data science jobs but in turn it will try to increase the number of jobs in data science how we will be discussing about the specific points again guys this video is based on my research and apart from that my experience itself yes some of the discussions that we are doing may vary but trust me in this the amount of years that i have spent the amount of things that i have seen around me definitely automation will tend to increase in data science jobs that is what i also feel so i've actually created this specific video so let's go ahead and let's try to understand why automation will not kill data science jobs so this is the first point that we i want to really take up the most important thing in data science project is domain knowledge okay let me just take of some amazing example let us take that same detail library example where we are doing exploratory data analysis let us consider a normal person and let us consider a data scientist fine the normal person knows how to do the exploratory data analysis will just use that library with the help of one line of code you'll create all the you'll get all the visualized reports but to interpret those reports to understand how the data is behaving in those reports right to understand that reports we definitely need to have some amount of domain knowledge right what is domain knowledge domain knowledge is basically understanding about that specific use case the most important thing again i'm telling you guys data science is not about model building it is about understanding that specific use case understanding what kind of data we'll be using in order to solve that specific use case and trust me like how we see kaggle data set right so simple in a csv file we we see some normal data sets you know where we are practicing things where you are practicing machine learning and deep learning in real world scenario the data set is quite complex instead of complex i'll say it is quite notorious why i'm saying you this because just understand to derive when we are actually working in real-world scenarios guys the type of data that we get it is not at all clean we will not be even knowing what kind of features to use and sometimes with the help of donaire knowledge only will try to understand that what are the new derived feature we have to create right now suppose let me take a very good example and give it to you guys suppose if in your data set you have two features like longitude and latitude what do you think what we have to do with that longitude and latitude right whenever we want to solve that specific use case some of the domain expertise may say that we have to convert that into some distance you know suppose i have two longitude and latitude of a single record right i have some starting longitude starting if i take an example of uber use case right in uber use case suppose i want to predict what is the price of the uh or what is the taxi fare based on the distance and suppose there your the data that you have is like pick up longitude pick up latitude drop off longitude and drop of latitude so what you'll do with this kind of data right this is a notorious data itself right at that time a domain knowledge will person will come and who is actually working in that specific knowledge will tell okay just calculate the distance and based on that distance will try to create a new feature this is just one example guys and in life you will be getting various new new scenarios even though you are 15 years experience in data science 20 years experience in data science 10 years experience in data science unless and until you don't have that domain knowledge you will not be able to create this kind of derived features which are pretty much important to solve this specific use case automation over here is mostly okay once you did all these things right and always remember guys in a life cycle of a data science project right if i consider from data collection till feature selection lifestyle right more than 40 to 50 percent of your time will be gone over there now suppose understand we have some amazing automated libraries to do our eda to do our uh automation i mean feature engineering automation feature selection automation how helpful it will be for the data scientist to do the other part of the work just imagine this because that point i'm going to take up in the last point in this the reason that i'm actually giving it will help data scientists to complete their work pretty much faster they don't have to worry about it because just understand a new person a new person by default he cannot just directly use an automated library he needs to understand how that things work but a person who has already worked in that life cycle of a data science project if he knows all the technique he knows how to interpret those things so if he gets the help of that automated libraries he'll be able to do his work faster people right now just they have to focus more on the domain knowledge things they really have to focus on how they can perform those things right part of coding is there and part of the coding can always be automated the whole process cannot be automated trust me in that again i'm telling you why it cannot be automated because the data set of the real world problem does not look like a kaggle problem statement it is far more complex than that we more probably have to use 100 different sources you know when i say sources it may be a database it may be a third party apis it may be iot sensors data it may be some kind of diagnostic data and once you get just a raw data in you right you really have to think what you have to do with that specific data right so the most important thing in data science is the domain knowledge okay so unless until you don't have this one and definitely automating some part will never effect never affect this kind of jobs you will be requiring an experienced person you'll be requiring a person who has a lot of domain knowledge in data science industry so if you are working in that specific path i'll tell you no one can take up a data science job in turn the hiding of more data science jobs will happen coming to the second point automation removes the chances of human errors okay as you all know guys in data science industry whenever we are working on various use cases we do a lot of statistical analysis we do a lot of hypothesis testing and when your data set is huge you know at that time you cannot just do manually guys it will take a lot amount of time again right and when you're doing that calculations manually you may tend to make mistakes because we are human why we are making machines just understand why we are making machines to make our work easier and to reduce the chance of human errors that is the reason why we use calculator right otherwise i was taking a pen and i was actually doing all the manual work we are trying to remove the human errors the chances of any human errors so any kind of statistical analysis probably a phd guys tries to do it he may also make a mistake even though he may be an expert in all these things he will also make a mistake guys he will tend to make a mistake because human minds are developed in that particular way every human being makes some kind of mistakes no one can be perfect right so that is the reason why automated libraries are there for those right we have chi square test we have p test we have this test right and just by using those kind of libraries we'll try to we know the theoretical concept we know why we are doing that right yeah and then you may be thinking wow what is the use of data scientists but understand to become a data center you should know all those type of concepts right so this was the second point it removes the chances of human errors the third thing is that which is the most important point automation just makes you work faster i just like to limit automation till this level guys yes i know how to write the code now i want to make that code little bit more faster i don't have to invest that much type in writing that i know how to do i i understand each and every step now i may use an automated library and do it let me take an example of hospitality data analysis we have libraries like detail i know all the steps of exploratory data analysis like i'll go and first file down missing values i'll go and fight first and check imbalance data set i may go and check how to handle category features i know all those techniques right now i really want to make it work faster because i don't have that much time because i have to also focus on other part like domain knowledge i have to focus how i have to build the machine learning algorithms how i have to perform the hyper parameter tuning and as i said that in life cycle of a data science project tasks like data collection feature engineering feature selection will take 30 to 40 percent of your time even exploited data analysis will take some around five to ten percent of your time now if that library is able to help me to do that particular work faster then why should i not use it right so automation just makes your work faster remember these guys whenever i feel that yes my task i'm doing some specific tasks i can do it quickly i'm not judging myself whether i know everything or not by default i will be considering i've been working in that particular stage i know all the steps that are included so i will try to just try to use that automated libraries to do that particular step so in short automation makes your work faster okay so this was the three points i really wanted to mention now i am saying that why automation can really really increase you know the data science jobs because understand in case of automation what do we do we try to automate a sub specific stage and on top of that many libraries come as the easiness happens as new techniques come you know those kind of automated libraries can be built who are building this kind of libraries again a data scientist only building right you'll be seeing in pipeline there are so many libraries with respect to automaton who is building that a data scientist is obviously building that right you have seen so many uh automated libraries till now why they are building to make some of the tasks easier we know how to do that specific task now we are doing that we are creating new libraries to make that task exam probably as we go ahead more easier method will come more optimized technique will come so that we will be able to do our task and but again the base is that you have that domain knowledge unless energy you don't have the domain knowledge you take the automated libraries also you do nothing will happen you will not you'll not be able to take that model to the production only and when model is not going to the production you're not doing any kind of work you're just playing with the data that's it so coming to the next slide automation advantage is this i'm talking with respect to automation and data science right i'll give you one example in my previous to previous company we were actually working to increase the efficiency of the server and for that we had actually created one ai application automatically we'll be able to uh the server will be able to efficiently work i'm just giving as an example i cannot just tell you the real use case now in this scenario when we are able to create that ai application and now if the server usage is being handled smartly now what will happen the cost will obviously come down right initially this much cost was coming now the cost has been reduced by half just imagine the cost has been reduced by half or we are performing some tasks that is given to some third-party people third third-party company and then what we did is we built an ai application which is automatically doing that particular task right definitely the cost will come down now that saved cost can be utilized in some other data science project obviously the company will find it interesting right because they're trying to reduce the cost that cost can be properly utilized for some other data science project so in turn there will be definitely boost for the data science roles and in turn there will be an increase of demand in data scientists that is the reason why i'm saying you guys automation will also lead to the increase in the demand of data scientist again it depends on scenario depends on situation right but most of the time it does happen so here you can see as the automation increases productivity also increases you'll be able to solve that particular use case in less amount of time and again i'm telling when i'm telling about solving i'm considering that you're good at domain knowledge also this can bring the cost down this will boost the company to use data science for more use cases this will boost more ds roles increase in the demand of data scientists is a final goal and definitely there will be the rules and that are what i have seen guys i've seen this so i upload videos on automation also uh automated libraries also i upload videos on solving the use case step by step also the reason that i tell you both the ways is that because i know the process of how to do it now i'm trying to do that whole thing with the help of automated library and i'm also able to understand that interpret that and provide that all information to my stakeholders right that is the major aim so don't think that automation will take up data science jobs specifically yes it may take out jobs with respect to some other companies other sectors you know specifically in factories and all where the machine can do all the tasks like what human being does but with respect to data science that was the most concerned query because everybody was worried see since you're learning data science and all right understand the most important thing is to understand the use case properly understand about the data set that you're using understand about the process that you're doing that is the most important thing and by that thing only you will be able to fine-tune your model in a much more better way get amazing accuracy and then only you will be able to take that into production so i hope you like this particular video please do subscribe the channel if you're not already subscribe i'll see you in the next video have a great day and thank you bye