Complete MLOps Crash Course | Machine Learning | Community Webinar

Key Takeaways

and welcome everyone again um i can start with it with a more detailed introduction um i'm as ml developers and ml engineer by training so someone who came from a software engineering background and move towards machine learning pretty early in my career and have spent a fair few years now deploying machine learning models in production and i picked up a lot of these envelopes you know learnings which i'd like to share today um i also in my spare time love to work on side projects so i did a few i do one every year actually um so i am an indie hacker by heart so i love like playing around with new frameworks and tools and um yeah creating products with value um yeah so i'm uh i would say i'm i'm an ml engineer by training i'm also co-creating xenomel which is an open source mlogs framework we're going to get into that in um in a few but i would actually like to start the presentation today before we get into mlaps just to gain an understanding of who i'm talking to as it's my first time here um who exactly is the audience constituting off so i set up a poll in slido um this is a tool that lets us let's just basically ask a question live on the poll the qr code is on the top left so if everyone would be so kind as to take a few minutes to start voting um and answer the question the classical question what describes you best um i know that there are some people out there who are not going to be um you know fitting into any one category you might consider themselves um a bit different but just choose the one that resonates with you the most um it would really help me cater the the um the entire presentation towards you guys so as expected data science so the data scientist category is winning this is of course because it was marketed towards data scientists but a lot of others i i wonder if somebody could put in the chat who the other category represents if you could just pop it in the chat and read it out and maybe there's a category that i missed okay students sure that make sense if you're a student maybe you could vote for what you want to be um as you as you study in your career marketing biomedical engineering okay interesting cool i'll just keep this up for like 30 more seconds ai consultant link could someone from the team share the link with with everyone i can try to pop that over that is all right but we're almost so i'll just give it a few more seconds just to settle down data scientist is winning so about half the audience constitutes a data scientist and a lot of students i i believe in from different different fields in different regions which is which is perfect i'm i'm going to go ahead and close the poll now thank you so much for for that i i really find it useful to understand where everyone's coming from when giving these sorts of presentations because ml ops of course in machine learning keeps on getting commoditized and it's it's becoming easier and easier from everyone to get into machine learning and specifically even if you're from a non-traditional software development background machine learning is getting more and more relevant so today we're going to be talking about mlaps so i i thought i would start with the canonical mlops slide it is basically like a prerequisite to study this slide when you're starting your journey with ml ops it was a slide introduced by google scully specifically who was a pm there pretty cool guy and i think it's pretty evident what the slide is saying it's it's the fact is that machine learning as as used in the industry is quite different from machine learning which is used in in research or like for the students in university it's actually a very small part of what you learn at university is actually instituting these components that make up ml in production and everything around it is actually way more work and way more complexity and you're going to have to learn it as you go into the industry and if you took it take a look at this slide this was this was something that happened in 2015 so it's a pretty old understanding but more and more in 2000 onwards especially there has been a boom of um people wanting to understand what exactly this means and therefore the um you know like the word ml ops is getting more and more popular for google and it's it's it's a buzzword that everybody wants to learn about excuse me so as you can see a lot of people in the in the industry who actually do machine learning properly like elon over there at tesla with andrew carpety they they are quite serious about the fact that 90 is actually just software engineering and only a small amount is machine learning and that's essentially what the diagram before saying and you can see how that translates into how roles are formed in the in the actual industry as you go and progress in your careers or maybe some of you are already data scientists or um i know that 40 of you are then a data scientist is increasingly um shrinking in the value they can bring across right so you have the data scientist who's essentially doing the data science stuff meaning training the model etc and then you have the data engineers ml engineers product engineers people who are basically you know sres who are trying to fix everything around it and data scientists need to be aware of this and need to understand that if they are to create an impact across the entire value chain as they progress in their careers and they would they would need to have an understanding of the systems around them which constitute essentially mlaps so i put the slide up just so that everybody has a picture of what you know a classical ml team nowadays in the industry looks like it's it's changing but um you have to be aware of this if you were to move forward in in your career and if you look at the the problem that happens when data scientists try to move into you know other fields uh software development they end up spending most of their time on data so actual data analysis starts getting less and less only maybe 20 of their time and 80 percent of their time is just finding gaining reorganizing huge amounts of data and the problem is this is a very inefficient data strategy because from from the background of data scientists and people ml practitioners who come from a non-traditional software engineering setup they find it very hard to do this stuff and therefore you know it becomes very inefficient for a company to be able to um ask them to do that so they need to retrain themselves and span across the value chain but spending more time on data perhaps is not a bad thing so i mean classically a data scientist work would look like this they would perhaps do it in a notebook maybe a little script um read csv you know pandas fit it use some numpy magic and fit a classifier you can you can use the equivalent of r um and you know you could have ebox you could have deep learning stuff in the middle whatever but essentially that's what a data scientist is trained to do as they come out of university most of the time and the problem is that as they do that and if they if they take that as their final artifact and move it forward and like sort of give it over to engineering to productionalize to put into production so that it serves potentially hundreds of thousands of users this is what engineering sees engineering sees that this is just just a mess i mean how can they take that piece of code and productionalize it because there's so much else that hasn't been accounted for and this is why data scientists need to probably change their way of thinking and that's what's currently happening um and a lot of data scientists who have realized this in the industry in the last five years have rescaled and gone across the valley chain and i'm here today to give you a lesson on emerald ops and maybe why you could go towards that end of the spectrum um to better yourself and set yourself up for a successful reader so let's take a look at why exactly that spongebob name came into life right like why exactly data engineering you know flipped the table why did they think that this was a bad um artifact like why couldn't they productionize it um the problem is that machine learning and production what one might think it looks like is very simple right you have data collection you scope your project you gather the required data you engineer features for the task and you have you know model training which is what we just saw just training a model and then maybe even model deployment where you package the model up you perhaps put it in a rest api a flask application or something deploy it on kubernetes and while all of these steps individually taken are actually getting more and more trivial it's not what the reality looks like if you do data collection and training modeling deployment and production all in isolation of each other excuse me it's it's actually it's very easy i mean you can you can train a model without fit you can deploy a model but don't deploy now the problem is that that's not what happens what what happens in reality is there is a feedback loop that goes into the entire process and affects the output of the system so think about it in the sense that in classical software engineering let's say you're developing a mobile application or you're developing a website and you know the way you do that is you have you know you have a software development life cycle where you have sprints you create a feature you test it you deploy it and then it's deployed and the website is updated so anyone who joins that website would be able to you know consume the latest feature but if something breaks on the website what you would do is you would get some of that feedback you would update the code right run the tests again and deploy it again you can do this with ci cd in an automated manner in the devops world and you know that's very easy because the only thing that's being affected um in terms of the output of the system ie the website or the mobile application the only thing that's affecting it is the code right so if you change one line of code or two lines of code you know it's a fairly um scoped complexity of things in machine learning that's not really how it looks like there is another player in the game which makes it very hard to automate this this um this this whole process and that that other um you know loop and that other player is is the data because machine learning models are actual you know actually software artifacts that are trained on data so inherently they are they have more complexity attached to them than just code so you have now two pieces of things that you have to take care of covered in data and it turns out data is very hard to make reliable right data changes it's unstructured sometimes it's structured but still it's very hard to get a handle of data and that's why the industry struggles to actually deploy this reliably in production right how do you make sure that as the loops happen you can go through this process again and again again and again through all the different stakeholders that exist and not have things break make and not have disasters happen um and that's really what in the in a nutshell where ml ops lives emelops is simply a set of practices that aims to deploy and maintain machine learning models in production reliably and efficiently so it's not a bunch of tools it's not a bunch of you know a bible somewhere like a book which says okay this is exactly what you have to follow and this is what mlap says it's simply a set of practices if they're not codified anywhere there's no standard yet but in essence what it's trying to do is achieve exactly a reliable loop in this entire process and if there's anything you want to take away from today it is this you have to make the data loop inside the machine learning life cycle reliable in order to get any value in terms of business in terms of research out of machine learning in the real world cool so the thing is as i said the deployment part of this pipeline is getting fairly trivial you have you know you have a software's now tools now which can easily wrap a machine learning model deploy it into production it's quite easy you can even use managed services for this all the crowd like all the cloud providers provide it um you can go to amazon google whatever and you know it's getting it's a bit easier the thing is you can't just leave a model lying around after you deploy it right unlike a website it's not just passive it's ever changing data keeps changing and that's that's where the trouble starts so i'm gonna go through a few things which i call the post-deployment wars meaning the the problems that happen after the first model in a company is deployed you know after the data scientist job is finished he he or she walks away and they say okay finally i'm done my work is done but actually the reality is the work has just begun so let's start with number one accounting for latency google says that 53 percent of visits are abandoned if the mobile site takes longer than three seconds right i mean you must have experienced this yourself um if you open up a website if it's not opening and google itself if it doesn't open in three seconds you're probably gonna hit back you're gonna go somewhere else and imagine if you had a machine learning model deployed um on your application whether that's a mobile phone or on a website and it's taking 20 seconds to run an inference on your data you're going to lose perhaps half of your audience immediately and half of your audience could mean millions of dollars of revenue or it could mean the difference between your business surviving or not so as machine learning practitioners develop machine learning models if they do not account for latency pre-deployment post deployment it's a huge pain let's go on to the second goal fairness and bias um i don't know if everybody if anybody here in the zone for watching online ever saw this saw this news where microsoft created a twitter bot to learn from users and it quickly became very profane and racist um and you know very it was because they had a feedback loop going on in twitter where you know in the tutorials they were scraping tweets and feeding it back to the pot and about learned from these tweets and of course that was in retrospect uh set up for disaster no i mean if if any any one of you has ever spent a few minutes on twitter you would understand why but because they had not built in checks for fairness and bias they actually uh were not able to account for such a development and they would it was a huge pr nightmare for them um the problem is that fairness and bias is not just about cultural norms and um you know like political topics like racism um or cultural topics it's also just even from a very purely business point of view very very relevant so let's say you have um an application and that application serves the entire world and you say as a data scientist that 99 um you know your model gear got a 99 accuracy but if you don't inspect where that one percent is inaccurate you might run into a disaster right so let's say you were accurate across the world except you were inaccurate for all your american colleagues um or people in seattle let's say right so you're being very unfair in your machine learning model to the people in america and those might constitute a fairly large chunk of your business let's say right if you do not have any accounting for bias or fairness within your machine learning model training you're never going to be able to understand how to reliably deploy these things in production so that your customers are happy and actually you're being fair across the board and if you are not fair and do not have biases and checks built in you're gonna have um the government coming after you right so you if you cannot explain an audit why your machine learning model behaves a certain way governments and lawmakers around the world are being very aware of the fact that um they are well within their rights to be able to ask you why that is the case and you can you can completely understand that i mean as a citizen we we would like uh machine learning models to be you know safe because they do critical tasks sometimes i mean they drive our cars they you know fly our planes they can do a whole bunch of other things in the future so you want to be able to understand and explain and audit these um these models as they're trained and you want to be able to do that retrospectively right now in almost like a time machine way so if if someone asks me in two years from now that what slice of data was your machine learning model trained on that resulted in the prediction that turned the wheel of that car a certain way i should be able to give a fair enough answer to that question and that's why you know we are facing a huge uh plethora of legislation that might affect in um all of us including the people who are learning data science and you know want to become practitioners in the field all right um another post deployment though is just how long it takes to get there so this is a survey conducted by our friends over there at algorithmia who actually talk about that it takes really long for organizations to put a trained model into scale production the reason is i mean all the reasons i spoke on before and this poll is actually asking whether they can do it reliably and you can see that it takes up to a year for a lot of these companies to even get it out into production in the first place let alone get it out reliably so it's a it's a huge burden on a business to seriously invest in machine learning if they see these numbers so we need to be able to get these numbers down in order to make a coherent um case whether that's you know in any field any um industry in the world who wants to apply machine learning and take take advantage of it and last post deployment bow that i think is probably one of the most important is the fact that the world is not static right i mean if you do pd dot read csv or the r equivalent of that let's say you're just reading a csv file that's a static file it's not changing but the word the world doesn't behave like that right you have models you have concepts you have data drifts um that happen all the time and there are a few examples of this i mean if you if you train a model to predict the best-selling book in the world in 2001 it would be one of the harry potter books right maybe philosopher's dawn but if you predicted it in 2016 it would be trump fire and fury right so the world certainly had different interests you know a new york times bestseller can be equally political and equally fantasy how do you account for that drift how do you account for a pandemic happening that just basically shifts the entire data in a way that your machine learning models cannot cope up zillow learned that lesson really well when they had to shutter their home buying business and lay off two thousand employees i think they lost like millions millions like tens of millions of dollars because they had trained models on housing data which was pre 2019 pre-covered 19 and that caused their emit pandemic predictions to be so far off that they had to suffer huge losses on the business front and this is really the reality that after deployment starts if you don't refresh your models the quality of your models usually goes down so this is a representation of what usually happens i mean this is obviously not true for everything i mean not all data drifts um the same way but as a rule of thumb you should expect that the moment you have deployed your model out then the quality curve is going to go down right and you're going to have to refresh this model over time in order to have a seamless of reliability of quality and this is what all the big companies do um netflix spotify they're they're deploying the hundreds and thousands of machine learning models every every few hours refreshing them accounting for drift finding out if things have changed and of course these things cannot be manual they have to be automated and therefore you need to understand the knobs okay i'm i'm just going towards the end of the theoretical part we're going to get into the code zone um i'm going to end by this last concept which is i believe one of the central tenets of envelopes so ml ops is in many effects encapsulating the transition of an industry from being model centric to being a lot more data centric model centric means something that you might be classically aware of as a data scientist you hold the data fixed the pd.csv and you iteratively improve the code and the model right you iteratively loop over your model you you know pump up the epochs you change the solver you change different parameters and you get some accuracy and you probably need a lot of data for that data centric methodologies are different they hold the model fixed and actually they often start with a very simple model even an if-else condition i've seen in production but it could be as simple as a linear regression or a random forest or a gradient boosting algorithm and then they iteratively improve their data so what that means is let's say you have a bunch of data from the last year and if you want if you were doing it completely model centric you would take all of that data and you would train your model on it and you would just tune the parameters if you if you were being data centric on the other hand what you would be doing is you would maybe partition your data into six months so um let's say the first two quarters of the year and the last two quarters of the year and then you would train your model on the first two quarters and then see how the data is drifted on your model and how your model behaves in the last two quarters right that would be just one example of probably thousands of micro decisions you can make that will lend itself to you becoming a better developer in in the sense that you know i'm not saying model-centric developers are bad but you might get a lot more bang for your buck if you start making the decisions in a data center way given all the problems that we've spoken about and i'm here today to tell you that actually if you want to learn machine learning of or mlaps you would be well served to actually going towards data centric machine learning by thinking of machine learning in terms of pipelines what is the pipeline a pipeline is just a series of steps that happen from the um you know and one at a time um and they're isolated each step is an isolated self-encapsulated piece of logic and you can link these together through an abstract thing which we call the pipeline and once you start thinking in pipelines then you're automatically being more data centric how is that the case and first you can separate environment from code so you can have um you know you can start thinking about hey where's my pandas dependency how's how's my operating system look like you know how about i dockerize these because i'm going to have to change these as the code changes as data flows through these steps you can start to version it you can version the data you can version the artifacts you can you can you can start building in systems like this you can maintain modularity and extensibility so if you want to swap out your random forest with a linear regression in the future you would be able to do that by swapping out one step in the process rather than the whole thing in a notebook you can crack metrics and parameters you can have data quality checks and drift detection plugged in so you don't need to do that at the start but you can excuse me you can actually plug it in and you can do continuous integration and deployment for continuous improvement so what that basically means is you can automate this entire process so once you have this pipeline figured out you can literally set up your company your organization your startup your big company in a way that most of this is abstracted away from you you don't have to think about it and ultimately what you can do is you can provide a reliable machine learning mlaps process with full reproducibility of your results provenance over your results meaning what came from where using data lineage and other techniques and this is all i mean i haven't told you a tool of how to do that right we have a practical lesson afterwards but i'm just giving you a perspective on how you can get into mlaps is almost just shift your mindset to saying okay i need to be looking more at the data than the model because that's what happens in the real world so in in short just to conclude the this part mlm production is different from ml in research and has a different set of challenges these challenges are growing in revelance as adoption uh have relevance excuse me as an adoption in machine learning increases uh mlaps helps solve these problems and amalops is rooted in more data centric in being more data centric than model centric and developing in pipelines helps you to become authentic more data centric so that's my takeaways for today um i think it gives a broad overview of why ml ops is necessary and what it is and how you can start almost thinking in these pipelines to start getting into it i haven't gone into this sort of diagram which you might get in a more advanced lecture which we can do if you if there's interest um this is sort of what a machine learning operationalization platform classically or you know the modern platform looks like you have many moving parts here you have the data engineering parts the data science parts and you can see at the top that pipelines actually connect everything together so you have pipelines which go from the data stage to the training stage to the deployment stage and everything is tracked in external systems metadata stores feature stores monitoring solutions and you have your ingestion system separated as well if you're more interested in learning about these things i have some references at the end which would take you through one by one all of these things and you know make you understand in a concrete way why all how all these systems look like in the industry but given the time today what i'm going to do is i'm going to show you a demo about xenomel and which you know i'm one of the co-creators disclaimer but i think it's a good tool to learn mlaps because it's an extensible open source framework for mlaps and it's centered around creating machine learning pipelines and today what i'm going to do is i'm just going to give you a little you know a few lessons on how to build a pipeline that can is extensible and reproducible and you can use something similar in your life and start playing around with these things so and become more data centric and it can be the first window into mlabs for you um it's very simple to create pipelines in in xml you have the you know it's all in python so people who are from the non-python world maybe can use another tool i'll be using python today and yeah let's let's actually switch to that i have a demo ready in this in this repository so if everybody is is online maybe you can now switch over to it um it's called zen bytes and maybe someone can share the link as well while i set it up on my end and we have q a right so if if you have questions right now while i'm setting this up i would be more than happy to ask you um to answer them what is okay i'm gonna go through some questions that came in in here so what is scale production ashish asked i am and someone already answered it's a dynamic style of production whereas where your processes and infrastructure automatically provision as workload increases or decreases um yes you this is a very good answer exactly that's what it means so you have classically for example kubernetes is a tool that is used as an orchestrator to manage workloads of different production use cases in in in the industry so that also dynamically scales ie adds more memory and more computers as your workload increases if more traffic comes in okay any recommended materials to learn about ml ops i do have some there are two that i three that i like there's the cs329 course by chip human um really love her work josh tobin has um full stack deep learning that's a great course and made with ml by goku mohan that's a fantastic resource to learn about him so these three i would recommend highly i have them in the links at the end are there people are there people who are able to do all of this i.e all the production parts first deployment yes these people are known as unicorns so they do exist um but they um you know it's rare and if you're able to do it you will be very valuable to the company okay edward asked airflow like sorry i just want to say at the end of that so yes i mean you can learn all of these things but of course you usually you know it's too much for one person so you have teams develop schemes that help but you can decide where to go air flow versus q flow um air flow versus q flow is a good question i might get into that while we actually um do the process any reps from databricks here uh vassan says i don't know but i have a few quality i have a few people in databricks um they and we are going to be showing a databricks tool today called ml flow so perhaps we can um get into the demo now and get our hands dirty with cinema so let's go through it all right so we we answered all the questions if for those of you who have cloned the zenbytes repository it's xml dash io is less than bytes on github open source you can give it a starter just to remember it but you know if there's a star you're going to give please give it to the main cinema repository um then this is the first notebook that you start with um so you have you know just the imports and the first thing you're going to do is you're going to initialize an event and you're going to set the default local stack you don't need to worry about this these are just um commands that you can copy paste for now and we'll see the need for them later um and this is what your first pipeline will look like already so it's just a bunch of python functions that are linked together through this add pipeline function and let's go through that slowly so you have here you have the command a the function a it returns two you have b which returns three then you have the add function that adds these two together and you have a pipeline which says okay the first and second function's output goes into the third function and then you actually plug and the abstract representation into the into the actual functions and as you run them you can see that uh like xml realizes that you run a pipeline and just locally runs these these functions and you might be wondering why is it so hard to run a couple of functions that add some things together well we can see that with some integrations of xml which actually lets you visualize what's happening in the back so i'm going to use the dash visualizer which is a inbuilt thing in xnml which allows you to actually visualize a pipeline so you can see that the way it looks like right now is you have you have this pipeline so you have a b [Music] um the squares are actually the outputs and you can see this the things have persisted across the running of the of the pipeline now um this is going to be a path in my local folder which is um you know storing the result of the step b and then they're going to add and then they went into the last output which was just adding these things together somebody asked me in the chat where the notebooks located if you go into zenbytes there's we're starting with zero zeros dash basics.ibmp it's in the root of the folder cool so let's go back so that was pretty easy right and now what happens if i want to leverage the artifacts that have been produced in these pipelines so what happens if for example i instead of using be as my result so let's just scroll over with b which return 3 i wanted to use c and c returned actually 10. so i could use the same pipeline and rather than having the second function be b i could just say c run it so this is the modularity i was talking about right and then visualize the result again and we'll see something really interesting happening so just let me let's do this again so you can see that what happened here is there are some colors that changed so you had um the same so the the function a was the same and it's actually still id 87 so it's it actually understood that example understood that actually these um [Music] like this pipeline was ran before and you know we had already calculated a so we don't need to rerun that step but what's new is the c branch and then all i have to do is run that branch and then store the result and that's what's called caching and it's really interesting to see as you start developing in a data centric way in pipelining way this really scales up and gets really really really interesting um you know as you develop more complicated pipelines like machine learning pipelines and this is what we're gonna do in chapter one so this is the bulk of the lesson i'm just gonna set that up and while people are switching contacts i'm going to read some questions that came in how does mlops fit for asr based pipeline where things are trained on calorie-based styles um i don't know what esr based pipelines are and what cali is excuse me hr i have to move forward with the questions maybe you clarify how is it different beneficial from those like pedro uh pedro and airflow and somebody also asked for airflow so like xenoml is just an ml ops framework so it plugs into pipelining tools like pedro or airflow so you can run xml pipelines on airflow what xenoml provides is an abstraction layer to plug in other things like ml flow and deployment tools and we're going to see that in a second um could you suggest any courses i've already put some in the link and the recording of the webinar i think will be available afterwards it's being streamed live all right let's move forward in chapter one so again we start with initializing xml and xml actually the power of xml is that it sticks it's extendable without the tools of m ops right so you can you install so-called integrations which are basically plugins of xml with other tools that do individual things in the mlaps pipeline and what we're going to do is we're going to start off with rather than a very basic pipeline just use a machine learning pipeline which actually imports the digits data set this is like mnist and rather than returning two or three it just returns like the four classic arrays x train x test y test wide train and trains an svc on it uh um like an svm and just for classification and evaluates it with accuracy so these are just classical scikit-learn methods you can use them in cinema they're supported because i mean at the end it's just python and then we link all these things together and we run the pipeline and now it's actually really interesting because uh the digits pipeline has ran and we have just run our first machine learning training pipeline right this just trains the model and if i actually visualize the result i don't know where this went excuse me i just have to maybe refresh this guy so you have this bar up here which makes it a bit harder and there it is yeah so let me run this again yeah so here now we would see the i mean i'm having a bit of trouble visualizing this but for those of you who are actually running this on the other end you would be able to see the model pipeline that we just trained with the visualization and you would see that there are three steps just like this you have importing of the data you have the training of the data you have the evaluating of the data um let's see how that helps now in our in in mlaps so now once you have this data up once you have this pipeline you can use tools like evidently which are drift detection tools so tools that are built to detect drift um somebody actually gave me a good good tip about hiding ah thank you so much thank you so much melanie you can um you made my life easier so so evidently is a tool which is just a drift reduction tool it detects drift for you and you can actually define pipelines in xynml um which which have evidently step included you just need to import it and attach it to your pipeline and suddenly you can have two more steps in your pipeline on the side which actually take the test and training set and see what uh how the data is drifted between the distributions and all you need to do is you need to add two more steps into this pipeline definition get reference data and drift detector and import a few things define a few more steps again this is you're just appending on to this right use the caching feature to quickly get to that step and then you have uh the pipeline run and if you now visualize it with the evidently visualizer which is different from the caching visualizer you would be able to see if drift is detected or not this was very simple it was just a couple of blocks of code that i had to execute like zml integration would evidently help you have monitoring of your machine learning pipeline because you just plug that in boom and here you can also visualize it straight in your notebook this is very handy for data science to be able to see which features did drift which features you're gonna graft um so on and so forth then let's move on quickly so now that you have drift in your um in your machine learning pipeline what you could do is you could append an alerting step right which would alert your team i mean ml ops is all about visibility of the process so you could alert your team if the drift happens or if it doesn't happen so all you have to do is you have to edit your pipeline again add an alert your step give it the report of the drift right implement that step someone else can do that if you want you just need to import it run your pipeline again and you can see again caching is being used so everything is good you know then you have after the evaluator and after the drift detection you have the discord alert happens and here what i've done is i've used the discord button to have a channel and this works like discord is just like slack you can use slack if you want and you can see that a message has now popped up which said that okay there's a message which came from this pipeline and at this time and it says that no drift has been detected so you could append that with slack you could do many other things with with these cool features all right i'm going to move forward um with ml flow so if someone asked right now any reasons why cinema is better compared to other pipeline frameworks like your door has failure and pipelines so just to clarify xml isn't as such a pipelining framework it's an ml ops framework so it lets you create ml pipelines that then lets you transport across stacks so it's very tooling and infrastructure agnostic so if you want to run these pipelines on kendra or if you want to run them on sqln or whatever i mean sk9 is a bit different but like flow or queue flow you would be able to do so and i i will actually show you an example at the end which helps you do that all right so now we have all these pieces of envelopes starting to come together right one thing that you might want to use is all these experiments that you're doing um you are actually have to track your hyper parameters right you have to track your models i mean how are you going to do that so there are many tools broadly known as experiment tracking tools that actually track these sorts of things for you and with xenomel what you can do is you can just add as part of your pipeline something like ml flow which is a great tool for experiment tracking and the way to do that is just install the integration and then just add three lines into your step function enable import ml flow enable mm flow and import like import the enablement for decorator if you decorate the enable ml flow um if you use this decorator in your python function you would be able to use the ml flow library which is an open source library to be able to log all of your uh parameters so you can use you know the auto like the auto log feature which is a great feature from ml flow that automatically logs everything in your function and now certainly your pipeline has gone from being a very simple three-step pipeline to a fully tracked versioned experiment tracked pipeline and you can see that in the ml flow dashboard i just ran this pipeline 25 seconds ago the name of the pipeline is the same as as that and if i go into the ml flow dashboard i would be able to see all the parameters that i tracked my hyper parameters cache size class weight on the metrics you know 96 percent accuracy so on and so forth then you have a complete lineage of that experiment with the actual pipeline run as well so cinema tracks all of those things for you all right and not only that i mean you could use the same thing to swap out your trainer right i mean let's say the svc was not good let's say you wanted to use trees for that um so i could just swap out like like create a new function same thing just with the decision tree classifier run the pipeline again and then visualize the ml flow back end and i would be able to see i would be able to see now probably a new experiment yes exactly here it is and if i scroll to the right you can see this is a decision tree classifier if i was to select both of these i could even compare them and it's really cool to be able to use ml flow which is an experiment tracking tool right there are many others out there like weights and biases neptune and whatever you want to use please use a tool to track these things with xynml you can select these afterwards um and just plug them in on the infrastructure level so now see what's happening our pipeline is starting to get more and more mature and finally we're at the deployment level earlier right finally we're at a point where we like okay deploy this now so what do we have to do well let's say our logic is if drift does not happen deploy the model right how do you do that you add a trigger you say okay true or false should i deploy this model or not that should be based on the result of the drift and you have a deployment step which actually does the deployment right so i can do that right in my notebook all i need to do is add two more steps right in use this nml ml flow deployer integrations ml flow also lets you deploy models sidenote you could use other tools for that um and run this pipeline and what that's going to do is it's going to do something really cool so it's going to understand that one of these steps is actually a deployer step and now it understands that okay i need to do something which is going to cause an api to be created with my model which is going to live outside of the life cycle of xenomen right so it uses like xenoml is fully aware of these external services and these services you can fetch into your pipeline and actually load them right with these helper functions and see whether they're active or not so you can see this this service which we just trained the model was deployed this and it deployed the service and if i was to use that like my desk set and run a prediction through it service or predict i would be able to actually use that deployment right now locally um with just one function so what have i done i've actually deployed a service it's the rest api and i've just sent numpy raised to it it's returned me the result and now i move closer to production right so i'm sort of i'm like doing production stuff but i'm still local but it feels to me like i can now see what the latency of this model is i can see you know how is performing right now behind the rest api and these things are important because as you get closer to production these are the things we need to know um we can actually see whether this was an 8 or not yeah looks like an 8 maybe it's a zero i don't know but uh yeah so that's that's now we went from zero to a continuously deploying pipeline and if you run this again xenoml will actually pull down the latest deployment and spin up a new one automatically for you you don't have to worry about it and just last thing we have a few more minutes left in the real world this is what what would happen in mmos you would have a pipeline which is actually deploying automatically right producing a service and then you have a prediction pipeline which is running in parallel that's taking the latest model fetching that model's service with the service that we just saw and predicting on on that service and these two things are happening asynchronously right so you can have the model training happening 10 times a day but the prediction can happen maybe once a day with the best model and i'm just going to in the last part of this i mean i won't go through the code obviously set up a simple three-step pipeline it imports some data it loads the service and it actually predicts on the service and you would be able to see that the prediction is zero so right now if you look at this diagram again what you've done everything is local we have set up a pipeline that's continuously deploying the service and then it's fetching that service and predicting on it and voila in about 15 minutes time you have a continuously cicd machine learning process however everything is running right now on your local machine there's nothing which is happening in production per se right you just it just looks to you like you're doing some production stuff and that's what we wanted to do at zana man we wanted to give you that that ability to actually um you know feel feel that you're closer to production and then not only that we wanted to give you the ability to actually go to production so this is just going to be my last thing and i'm going to close for questions uh with xenoml you have this notion of stacks what stacks are are that right now everything was running locally for me my orchestration was happening locally my um data was being stored in a local file system and i had a metadata store which is local as well you don't need to understand exactly what these things are you can read about them later but with just a few commands what i can do now with the same code is i can flip a switch i can say okay rather than running my pipelines locally i want that same exact pipeline to run on aws with q flow or airflow and people are asking you know how do these compare you can use whatever you want in production and i want to store the data in s3 right i don't want to store it in my local file system i have a couple of colleagues who also store it there i want a metadata store which is a sql server not as not a local server and i want it to be all dockerized automatically for me and then transported into the cloud so i don't have to do all of this myself and just as my battery runs out of my computer runs out of battery all you have to do is you have to run these two commands which register a stack and this actually you don't even need to do your colleagues can do this for you um and run your pipeline again and when you run the same pipeline you would see something really cool so i mean i did this in a script now not in the notebook but you would be able to see that xenomel would suddenly understand that as your stack is an awsb stack i'll just stop it here it's going to build a docker image it's not going to immediately run the pipeline it's going to build a docker image it's going to package all your requirements and then it's going to transport it over to q flow where q flow is a proper like orchestrator right like it's running in the cloud this is our aws q flow and then it's going to run the pipeline like this where you have every step is a container image everything is being stored in s3 um everything is you know reproducible auditable but you still have all the things happening you can do all those commerce that i showed in chapter one locally so you could fetch the results you could see what's happening you could debug them and suddenly with xml what you can do is you can transition from an mlop setting which is local to an m to an mlop setting which is based on the cloud in you know in um in aws for example on q pro so i'm going to just stop here for a second and go over to my slide deck i feel like i might be lagging but just one last thing so i know we're out of time and i know i went through a lot of stuff at the end so my last thing is that i would love if if you like what you saw today anyone learn more if you go to github right now um on github.com and just or you scan the qr code you'll get the link you would be able to give us a star which is on the top right and what we love to do is when i'm talking to audiences if you guys can give us that love then we actually can get on github trending which is a huge boost for the project we're open source and it's free so you know it's just for the community and uh yeah i can we actually have a github channel like we have a discord channel whenever somebody gives us a star we actually get notified so my actually my battery died for um [Music] for the other screen but i'm just gonna maybe share this screen can you still hear me all right i'm just going to share this screen and as you can see it anything i need can everybody see it i don't think so we're still seeing the the demo channel ah like not the github channel no not the github channel all right so if you guys just go to github.com can give us a star it would help us immensely um you could create a fork if you want um it would get us on data trending and that's a immense amount of visibility we are also hosting an event next week about continuous deployment with xml and in this nml mlaps hour that's the link that i believe fatima can post in the channel um just so if you guys want to talk to us next week we are going to be doing this on wednesday next week so please join us for that it's an hour long discussion uh just to talk about cicd and ct and all these things and you know showcase more of the cool stuff that you saw today so yeah that's my final words i hope this was helpful i hope this was a good introduction to embellops for everyone please give us a star on github and join us next week thank you very much perfect thanks hamza so before you go into it looks like we have a couple questions for q a before you go into that i just want to say uh my team members uh they posted the link to the certificate of attendance um uh like 10 15 minutes ago um so we'll keep it live while hamza is finishing up q a and then um and then we'll we'll be done here so um uh data science those are team members if you can post post the certificate link one more time to give people another opportunity and then i will finish up here with some q a and you know get out of here all right perfect cool then i see so many stars i'm completely mind blown by by this uh by this um so thank you everyone for attending i will answer the questions don't worry um i see like yeah so thank you everyone who gave us the star um q a all right acharya please let me know what asr is um i don't know what it is i can't answer the first question um rahul asked me what are those courses and resources to start with mlaps i think that um i already suggested made with ml i suggested cs329 from chiphewin and i suggested full stack deep learning by josh tobin these three please uh read them and then we also have the zen bytes repo which i'm going to be developing with the team over there in cinema so it's also going to be sort of lessons catered towards mlops so if you guys want to check them that out give it give that repository start as well um caroline i hope you got the notebook i already answered the cat drawer escolar escalante pipeline um i hope lim that was helpful for you eduardo you you asked about mlaps references and i believe that you have it um yeah so thank you very much i think these were all the questions um the envelopes hour by the way next week on eventbrite you can sign up and we'll send you a zoom link so next week wednesday um pretty much the same time as right now we're going to be hosting it so please join us and like just you know come and chat with the rest of this nfl team perfect all right well thank you hamza and thank you everyone for joining today uh special shout out to my uh team members at a day of science dojo especially fatima rafiq for for setting all of this up for us she does a wonderful job with everything um just a couple of announcements on my end um yeah if you're interested in joining the zen ml opts hour next week uh go ahead and do that i think fatima has posted the chat the link in the chat a couple of times i know my team members are doing it on the live streams um we'll be closing the certificate of attendance submission so fill out that form if you want one um and we'll make sure to get that to you by the end of the day um next week um april 12th at 9 a.m is our next webinar getting started with modern data warehousing using snowflake platform platform uh we have the founder and principal consultant at data couch his name is babu chala so um uh feel free to sign up for that i believe the link's gonna get posted here in a minute as well um and then we also if you're interested in learning more about data science joe we have an information session tomorrow at 9 00 a.m and that'll just be me talking about everything that we do here at dsd so um if you're interested in learning more about that i'll also be talking about our free events just like this one um a little bit as well in that and uh feel free to join us tomorrow again thank you hamza uh thank you fatsuma thank you everyone for joining i hope to see more of you tomorrow or even next week thank you for inviting me