Practical MLOps #mlops #datascience

okay so hello and welcome everyone to another session in the database series uh we are thrilled to be here with you this evening for a system full of action-packed learning I am Tamil silver multiplication part of the data science team at analytics Vidya for those who have joined us for the first time a brief introduction about the data work the data was a series of webinars conducted by analytics Vidya and led by top industry experts it is a fun way to understand the concepts of data science from the leading players in the data Tech domain and as the name suggests it's one hour dedicated to data we are hopeful that these sessions are going to be a great source of enrichment and value-adding for our community members so now on to our session today which is mlops the Practical implications going forward in this data Mr ruju will cover the impacting points about envelopes and then we'll demonstrate how to actually use flai envelopes soon I hope you are excited to attend the data over with us before making things off and I hand it over to a speaker a quick recap of housekeeping items we are recording this session and the recording will be available on our YouTube the link and you can find section for asking any questions you might have during the session and we'll do our best to answer them as the data our progresses are towards the we'll be sharing it all towards the end to run requested before leaving the session Mr Rogers he's a B Tech from the strategy and holds extensive experience in creating and deploying since assemble Ops engineer create starting off with the idea of flair flia is an end-to-end low code SAS platform to design create build and deploy a lens and scale so over to you sir the virtual assistant hello hello okay all right shall I start Ed sure you are good to go all right perfect so welcome to the podcast everyone um today's topic is envelopes and the Practical implications going forward so before I start off a little bit of introduction about myself uh this is me and you can follow me and connect with me on LinkedIn I also write medium blogs from time to time so I started my career a little bit early I started in second year of college itself with a data science job and I was doing a Consulting for foreign startups and even larger companies by the time I was in third and fourth year and that's how I got to where I am today where in between my Consulting and assignments hello sir sorry to interrupt you uh sir your voice is still a bit low audience are just putting up the point can you please increase your microphones volume or anything like that from your end let me just check up yes sir is it better now uh guys sure okay foreign guys uh the speaker will join shortly it's trying to resign foreign uh hi is it better now yes it is much better now okay good good try to change some settings all right so let me share my screen okay foreign yes sir yeah uh you can interrupt me anytime if the sound goes bad again so all right sure so a little bit about me uh I started off working uh a proper job job in third year I started off with data science and second year of college itself and that is that's what has brought me to where I am today because I started off early and then I went into the roots of how to figure out what uh the core challenges is in in Ai and ml and I understood that the core challenge was more so about the infrastructure and the deployment part than the model part and companies or industry uh industry people were still much more focusing on the model part so then I had an idea and I decided to build on top of that idea to solve that infrastructure problem and we ended up with fly so fly is a ml Ops product company and we have our own Flagship platform the Flyer trains sorry to interrupt uh the PPT from your screen setting from your side is flickering like it is blurry sometimes it is not the clarity is not good um okay one second here um foreign hello sir yeah yes I'm back so yes is the screen up yes sir now it is much better okay so um hi this will be my second introduction and thanks to the internet connection uh it's called fly and um here are my LinkedIn and medium links I do occasionally write blogs so the way I started off in AIML was I started pretty early and I got into it pretty deep so why my final year as doing projects on kubernetes where we had a pipeline level deployments directly on kubernetes based on scratch uh I mean by scratch Pipeline deployments and that's how I got the idea for my product to automate the infra part infra side of the problem of aimf and that's what culminated into fly so feel free to follow Flyer's LinkedIn page or write to us on demo at the time blogs.com now starting off with the topic of the day I want you all I'm going to keep this picture on the screen and I want you all to focus for one sec and think about it this is a picture of what could be possibly the future okay how does the landscape of a tech of cyber look like 10 years from now five years from now you'll have drones flying around I mean that's what I'm stating the obvious we'll have autonomous drones will have autonomous planes we'll have we already have cars that are driving themselves we'll have ai taking key decisions in the industry sphere we'll have ai taking key decisions in the medical sphere where it the basic diagnosis is done by Ai and the later stage or the more critical stages are then uh attended to by doctors will have ai in sports Villa we are in we already have ai in marketing so this wide scale deployment of AI has been talked about a lot and it was previously also talked about a lot five years before but today we see that only like 10 or 20 percent of it is actually in reality we barely have a natural language processing algorithms that can translate from one language to another or understand the words of what you are speaking but then they go bad or they they go wrong when you change your accent just a little bit we have ai which can say detect objects in your camera but it goes wrong a lot of the times the virtual background which I was not able to put up today is also Ai and virtual backgrounds are made up of a different kind of image processing algorithms and you might have experienced that they go wrong a lot of the times bringing in items from the background so it's little things here and there just like that where you can see that AI is in its infancy stage in the real world deployment larger companies the fan companies or what today is called as man they do have a pretty good or rather I would say really good levels of AI deployed but when it comes to the lower rung or when it comes to the layperson the percentage of AI that is he is getting in contact with on a day-to-day basis is hardly like 10 percent and that is what will change tremendously in the coming 10 years so when we look at change we are at a curse point in the history of Technology where industry 4.2 has just finished and Industry 5.0 is starting I'll come to I I'll explain it a little bit in detail in the upcoming slides but I want to go first to a case study about touch screens so everybody assume knows very well about touchscreen technology sir sorry to interrupt uh can you try changing the settings of your volume like previously what you had like the PPT is clear now once you rejoined again the audio went flow the audience here foreign yes sir now uh is it Audible sir yeah sir sir can you hear me yes sir yes sir uh we can hear you sir all right yes sir please continuously sorry for the inconvenience please confuse okay uh all right so I'm recapping um when we look at a transformational change in technology say when the internet came on in the 2000s or when computers themselves came on in a big way in the 70s and 80s you have to look at what kind of Technologies were there at that time already on the top of which that technology was upgraded or built so let's look at the case study of touch screens everybody has a touchscreen phone today there will be nobody in the audience who does not use buttercream 4. how did this transformational change come along so the first touchscreen believe it or not was uh in place more than 60 years ago and that was a capacitive touch screen the touchscreens that we use today are most of them are resistive touchscreens and that came along in the 1970s the first commercial product with a touchscreen was the HP 150 with a kind of resistive touchscreen which you could control and yet for almost 20 years or 20 25 years the touchscreen technology came up and then it died down and why did it die down the first popular touchscreen Hardware uh thing was the iPhone before the iPhone there was no technology there was no gadgetry or piece of technology whether it be a computer whether it be a phone which was touch screen and which was widely widely popular and adopted by a lot of people so why did the iPhone become popular and it all breaks down or it all boils down to what kind of other Technologies digital screen need to be successful by itself so it needed the phone to have an excellent battery it needed to have an excellent display it needed to have all those softwares in place which would enable the software the application experience behind the touchscreen and it needed to have a sufficient RAM and processing power because that is how the touchscreen experience in all became fluid when we look at mlos the same thing we can see happening today mlops or AIML to become widely and widely used needs basically five things that are the pillars on which it is built off the first one is data collection and storage where you need massive and massive capacity of collecting storing and indexing the data and being able to access it very fast the second one is networks so just a couple of days before 5G was launched in India and 5G is supposed to have speeds in gbps and not Mbps and uh that will bring a transformational change where from the data store the data can travel much faster to your device so your device does not have to be very strong in itself in terms of computing power or it can actually take on and do functionalities which were not intended to be done on that device another part of what another continuation of what I just said is cloud computing where most of the processing happens on cloud and only the end results are conveyed back to you vipi so in this kind of cloud processing data and networks jumble you add another thing which is called The Edge devices or the iot devices where each device has a small microprocessor everything all the data is then collected and stored to a third place everything is interconnected via very fast networks and finally in using cloud and the power of cloud you can process much bigger much larger uh things and when you add AIML in the mix that is where mlops comes into the picture ml Ops is the glue that is binding all of these five things together which I try to explain and as simple terminologies as possible which will make AI take off in a big manner where you'll use AI on your phone on a daily basis you use AI or you will gel with AI uh similar or very simple example would be the way today uh the metaverse is taking on where you have to wear those goggles similarly you will have devices say something like a Google specs or a wiser where AI will project different information to you just the way it's shown in movies while you are going while you are running while you are going someplace and you can take decisions on the Fly based on that in technical terminologies the last four industrial revolutions happened exponentially building on top of each other and happened at for a much faster rate than the last so if the first Industrial Revolution took 100 years then the next one took lesser the next one took even lesser and the 5.0 Revolution where everything will be embedded uh embedded computers with AI assistance will go even faster in technical terminologies this is what is the best point we are at a cost point where MLS will enable the enter connection of all of these Technologies which you are seeing on the screen and using AI into it it will be a powerful and uh something so Splendid which you might not which might have implications that you have never imagined before thank you so that brings us to the question that AI has been around for around five six years in a big way it's been popular the python processing library is just that they've all been there in place for at least the last five six years so what is the challenge when we go from doing AI to in what is called as experimental conditions where you extract a small set of the data process it and uh get the results and then get the predictions and so on so forth to doing it in real life and the mass or the scale of processing is so let me start with going back to a little bit of the basics where your traditional program looks somewhat like this you've got the data incoming you've got the rules that are coded in by a programmer by human being and you've got the output which is pretty simple in machine learning this is how it goes and this small change in this process at the beginning where you put in data and the set of outputs that you have and the ml program actually learns the rules is what causes the disruption when it comes to employing things at scale in devops for example you could develop a website very nicely and you can you can deploy it at scale so in Dev if it is catering to 10 users in the deployment phase it can easily cater to 200 users without any significant changes in the application and that is because at each stage when you add on these traditional program blocks you are going to get something that is still pretty much in a set pieces of configurations so let me go back to the last slide a little bit so this is your machine learning program where it will output the rules itself and those set of rules is what is called your AIML model now your AIML model takes the new data and it applies the rules again and it keeps on giving the predictions and that is how AIML models work now mlos is this where you have large scale data multiple AIML models multiple predictions probably multi-stage predictions where a stage one of models is feeding to Stage 2 and stage two so on and so forth and then the final prediction and this is where because in a traditional program at each stage stage one stage two stage three you know the set of combinations you know the set of rules and the rules and the outputs are fixed whereas in AIML the rules and outputs are not fixed so you don't know the end number of permutations and combinations that might come out from the start of your program to the end so that is where the unpredictability in deploying AI at scale comes into and this is where ml Ops has uh big issues in deployment so some of the current issues are there's a high model failure rate so some test case studies say it's 70 percent others say it's 85 percent uh whatever it is it's it's it's definitely not a desirable success rate where data scientists are working um in the experimental conditions they're developing the algorithms they are developing the models uh the models are working perfectly in testing conditions but when you go to the um production deployment they are failing miserably the second thing is because I'm sorry because the um because the scale and scope of uh one second this class model so because the scale and scope of the um projects in devops is always whatever you do in Dev if you replicate the same thing in production then you get pretty much the same kind of results but in ml Ops and AML it's not like that so the tools that were used for the dev environments say your Jupiter notebook or something like vs code the same thing cannot be used as it is as a tool in ml Ops deployments a third big problem is when you go for large scale mlops deployments you need in you need replication at the infrastructure level you need modularity and kind of Lego blocks which you can build a Lego block and then plug and play it anywhere else and that is how you save time for the next iteration this capability does not at all exist at least pertaining to ml uh AIML platforms the last part is that there are so many Platforms in the mlops ecosystem or space that users have to often juggle uh in between a lot of a lot of them so I've I've also been a part of this problem because uh I've had to juggle through multiple platforms through cross-platform Integrations work the issues between two different platforms when that should not be the need to do so all of this in the end affects your time and cost to a production grade machine learning deployment a ml pipeline deployment at scale uh if there is no so I'd like to introduce you to the fly mL of suite uh fly is an end-to-end drag and drop SAS platform for creating and deploying machine learning pipelines directly to kubernetes okay uh just let me know if there's an issue with the uh video that I'll be playing so here's a small glimpse into fly uh this is a recording of fly in action foreign foreign thank you all right so uh that's a short Glimpse uh into one of our demo pipelines on the Fly mL of Street our end-to-end SAS platform for mlops and the way we envisioned it is that data scientists should have be focused only on the python code only on the AIML code and they should not have to worry about the devops deployments when it comes to deploying a large scale pipeline a heavy pipeline but they should be able to focus on their basic python code their machine learning skill set and the rest of the process the provisioning of cloud resources optimization of cloud costs uh the auto deployment of certain resources everything will be automated by fly that will allow different entities different companies different individuals different coders data scientists to build bigger and bigger and bigger AIML pipelines to eventually create what we look at or what I pictured uh what I showed you in that picture as the future of the next 10 years so that is all uh I like to take some questions now uh yes Bharat Kumar hello foreign the different stages of uh prediction are first it is Data extraction which is traditionally called as ETL uh the second stage is data cleaning where you your data is never in a state format it might come to you uh in different formats even in different file formats uh different encodings you have to clean it and get it in one set format the next stage is your pre-processing stage so I'll just go back to my PPT uh thing and I'll show you so the pre-processing stage has uh operations like scaling different columns or eliminating different columns or uh hello yeah so scaling different columns eliminating different columns uh getting uh say pivot columns out of the data so on and so forth that is what the pre-processing stage is and uh one second the last stage is actually uh as part of the pre-processing stage is encoding so that might increase that might include label encoding one hot encoding so and so forth so that the data gets in the exact format that needs to be fed into your model and then you have the AML model which you train on a training data set do the train test split uh train it on the training data set test it on the testing data set and then if your client so requires they'll have a data data set in the blind which is called as the dev sorry deployment data set which uh you can which is fed to your model where they test your model how accurate and how good it has been made so storage the data input cleaning does not happen automatically on fly uh because there is no set way to have all of the data cleaning functionalities uh kind of jumbled into our scrambled into one platform so that is that's not optimal so we suggest that data scientists do the data cleaning on fly itself uh using their own python code okay so Bala subramanyam uh the question is what is your thought on getting high-end compute in gpus one of the major problems in deep learning projects so that is one of the major problems that fly automates uh I'd like to show you one slide let me see if I can pull it up foreign [Music] demo pipelines which we built for a prospective client and the function of this pipeline is to entirely text the input of 3 lakh data input samples and it predicts the loan approval ratings for a trains four different models to predict loan approvals for future incoming customers now this pipeline uses heavy uh so it has got one neural network model one XD boost model and two clustering models so it uses a pretty heavy workload and gpus are especially needed for the neural network and the k n a clustering algorithm so what fly does is it completely automates uh This Is How We sync with kubernetes clusters in your cloud and it completely automates the provisioning of gpus to your pipeline so as to say so that is how the gpus are Auto provisioned within fly okay foreign so one question is how to integrate our data pre-processing and pre-existing models into your tool so with fly you have the capability to import certain uh python codes and pipelines directly so just bear with me for a minute and I'm going to actually take you to a demo of our platform foreign thank you okay just uh can somebody answered in the chat if my screen is visible now spin is visible sir all right so this is the pipeline that I was talking about now this one takes one and half hours to run even with gpus uh it's a pretty heavy pipeline I'm not going to run that what I'm going to do is open another pipeline uh which I can run while I answer a couple of the questions and you will also get to see it live in action so let me tell you a little bit about this pipeline uh this pipeline is a road quality prediction pipeline so basically it takes the gyroscope data from the two suspensions of a car and when that car is going on say 20 kilometer road and in the end it gives out a clustering model plus a neural network lstm model that predicts the quality of the road that that car has driven on without knowing that historical data without knowing the actual data so let me give the run for this pipeline uh and I'm now going to open a new pipeline to show how you can import uh existing models and code blocks so basically we have uh developed the functionality to import python files directly into your system so you can directly upload dot py files uh let me just share my full screen here uh okay so you click on import dot Pi then you add some python files and well there you have it so you can import your python files directly as they were from uh from your system and you similarly can import an entire jupyter notebook or we have developed a special protocol which is the dot fly protocol using which we can say export and import so I'm just going to go to this pipeline suppose and I've already built an entire block which is to say the code inside the container definition which is this and the entire input output files configuration and I can just click on export block and that block is downloaded as a DOT fly file I can also select like multiple blocks say I want to export all of these five blocks export them and they'll be downloaded as a zip file and I can import them directly into here so I'm just going to do a little bit of cleanup I'm going to delete this and I'm going to import blocks so I want to import multiple blocks here and there they are so these are blocks the lstm write is the exact block as it was in the previous pipeline or the pre-processing as it was in the previous Pipeline and you get it entirely with the full configuration with um the the coordinate and the configuration setup all of it ready made now you just have to connect it with say other blocks configure the files and you are good to go so that is how we manage uh importing existing applications or things into fly so another one of the question is time saving comparison statistics so the empirical studies that we have done so let me actually give you the answer based on the pipeline on my screen uh the loan approval prediction pipeline this pipeline does not run on a single instance a single compute instance deployed Azure ml notebook it does not run on Google collab uh deployed with gpus but on fly it can run around in one and half hours so that is the power of what we use under the hood kubernetes architecture method with some of our optimization stuff because of which the time reduction that we have estimated is around three to four times with comparable costing platforms or comparable to costing Cloud infrastructure so that is the kind of time saving and time saving Plus cost saving done in that order so how can University students learn this platform do you have any training program uh so we don't have a training program uh as yet right now we we would be happy to collaborate with a university to design it uh uh and uh we do have a rich uh demo pipeline catalog so I just showed you two pipelines out of that we have around six which are different pipelines for different sectors and we also have excellent documentation which will be provided along with our platform so it's fly is almost intuitive so in my team we have got some interns who could pick up uh the usage of this platform as long as they knew that they knew their data science or AIML Basics they could pick up the user of this platform in less than a week all right uh can we compare this with Azure ml pipelines uh bhavesh madhwani okay so yes we can compare this with Azure level pipeline so fly is a no code or a low code solution whereas in Azure ml you have to spend or invest I would say significant amount of time and resources to actually get up get uh into using that platform effectively whereas with fly you don't have to put in any time or efforts to learn ml Ops per se and you can get started with a basic data science experience and the ml Ops is completely out of it uh any resources to start for mlops for beginners um I would uh so I would suggest uh I I never learned from any particular book or any resources I learned on the go so I'd suggest start get into a good uh company get into a good internship where uh you are given the chance to explore mlots you are given the chance and access to kubernetes Cloud platforms Etc and uh that is how you can start because there's no uh there are very few courses on mlops essay which which do the job perfectly uh so does this handle cross a functional function I think okay so uh Anonymous attending okay does this handle cross platform functioning is that what you mean so cross-platform functioning is something that we customize for clients we don't do it unless it is required So currently fly uh integrates with certain platforms as per the requirement of our current clients and if your requirement or if you uh if the need for that requirement is sufficient we can add that functionality into our next version of fly itself so that way we are we're not open source but we are open suggestion platform all right uh Ajit R I am currently learning ml algorithms and I don't know anything about envelopes so ajith I think that uh ml Ops is one step above ml algorithm so you should at least understand the different ml algorithms first and uh different kinds of applications not just understanding the algorithms in theory but you should have applied it for at least um six to eight months or a year before you can get into mlops effectively because ml Ops is you have to understand your basic effect for it foreign [Music] fly essentially does all of this to final form of data which can then fed to the ml model so fly is not uh and I had emphasize this again and again we don't want our customers and our users basically to be um restricted in any form of usage which often happens with a lot of no code or low code Solutions hence we have completely uh automated the ml Ops part where you don't need to do the ml Ops you don't need to provision any kind of cloud infrastructure you don't need to install anything additionally it's a complete SAS platform and you can get started on the go your pipeline will be Auto uh Auto engineered I mean it will automatically pull GPU CPUs and everything if needed but we don't want to uh emphatically interfere into the AIML code into the python code of the data scientist we want to make work easier for them where they are not experts where they are experts well let them do their work all right bhavesh thank you bhavesh okay I've got a funny question here um if these platforms automate all the processes where do you see the need of humans in the ml space well uh at least for the next 20 years that's not going to happen uh for the next 10 years it's definitely not going to happen so it's a fun question uh and uh regarding the next next question uh you can please contact us through our website uh you uh I just put up the website slide here so please contact us through our website and my team will get back to you soon uh are there any more questions okay uh how to incorporate feature engineerings in the pipelines via your platform so uh we do not incorporate feature engineering feature engineering can be easily done via using pandas and other libraries already existent in Python and that is not something that needs ml Ops to do foreign platform which format is required into fly uh I'll have to check on this one sec So currently uh we don't directly connect with altrix ETL tool but you can offload the altrex data into CSV format store it in your cloud storage bucket whichever it is AWS DCP or Azure and you can pick up the data files from there uh all right thank you Martin so do we have any plans to incorporate some modules which will help users do better feature engineering yes definitely so with feature engineering there is no one answer for its all that is what our design so during our design phase we had thought given a thought to feature engineering extensively and the problem with feature engineering is there is no uniform set of say feature engineering functions there are some that are used commonly but then there are lots that are not used that commonly or not used in the same fashion and those are very difficult to integrate into a UI based platform so that is where we leave it to the data scientist we say do your pandas thing do your uh I don't know numpy thing and uh that is uh we can include feature engineering if it becomes a pressing demand for uh or say a pressing suggestion from many of our uh plans so all right uh other animal questions I think that is all really oh yes sir hello sir uh yes thanks a lot sir thank you too sorry from my side for the uh speed or whatever issues they were uh it's fine sir it's worth that like whatever it is it is worth it like I can see from the chat section that audience had absolutely loved the session okay great so it was fun to demo fly I hope that uh we can again uh demo uh say one of our uh POC or one of our demo Pipelines on your platform thanks a lot for hosting me sure thanks a lot sir and also on behalf of analytics Vidya and on behalf of our audience we thank you a lot for your time hopefully we can conduct more such sessions in the future so also uh I would request every one of the audience to participate in the feedback poll also I have shared the upcoming webinar links which you might be interested in there is a session at 8 30 today I have shared the link in the chat section please check it guys you might be interested in that as well so yes uh thanks a lot sir and thanks thank you guys hope the session was used to you sir uh anything you want to share sir [Music] and we really hope that uh as the audience gets more and more knowledgeable about ml Ops per se uh we can have more people participating um into um into ml Ops into building bigger and bigger pipelines and uh we'll be starting free trials on fly somewhere around the third week of October so follow our LinkedIn page uh and stay tuned for the news we'll announce it on our LinkedIn page all right guys uh it was nice uh it was nice conducting this session thank you all for coming and uh have a nice day yeah I said thank you sir thank you guys thanks everyone yeah