Mlflow Open source framework Hermoine demo

Key Takeaways

so to start you just have to do a prep install I'm gonna use a trace you to update if you already have it proposed Hermione trace up strays ml so it's on its own TP eprints coils from there today Hermione's viewed on top of Conda so you have to have anaconda Python or at least mini called a Python because we made a decision to use condoms and not AIIMS or regular virtual environments so there are already people talking about that and asking oh so what if I want to do not what if I want to use PI EV or something different you you so three things you can just ignore it and implement and build up a pious and work on top of it or you can adjust and use condom but that's not the point we do not we do not want you to adjust to our mind to this to you or you can help us build another integration and contribute in get up you know anyway so it's already here when it's installed I do hermione painful to check if it's installed so we have a nice little logo here with the version we are a zero dot one dot for it now and then we're going to create a new project so I'm gonna create her my any new to create a new project let's call it my okay so when I enter Hawaii is will ask me if I want to implement a brand new project or an implemented one so I can have an example I'm gonna show you did not implement it one but I have one implemented one I have one implement want to show you just reached now it should be done very this should be should not belong to mute okay so that's it and it shows you how do you actively it's the Conda and that was created for you so we're gonna do it Kandra activate homie you so now I'm gonna do my MLM oh my god I am so now I am in our corner and okay so I'm gonna change my I'm gonna change to this new directory that it gets just in my ml demo so this is the project that was built right now so this is the structure we have data a data folder here with the raw folder there's nothing here yet we have the the Conda F the condo fireman here we have the output folder there's nothing here we have the requirements we already put some some some to you as suggestion you know you can obviously you would you will change this we also have a we also have a readme project developed by a three data obviously it's not a three daily it's you you're going to change that no and we have a git ignore with some things that we don't use in kit and the source folder we have a folder for an API if you if you have to build one the config with config files that has already the project name the F path and the files path if you have to log into something you can just put it here if you if you like then you have the tests folder there's a readme explaining to you how to to do the tests in which folder you have to you have to be and stuff and a structure for you this just tests if the project was built correctly and then you have to develop your your tests then you have some some helper functions that we left for you to load data so make pretty prints to load Jason create directories etc and you have an empty train train dog pie file so you are going to set your MX Romanian train code here and the project structure is here so inside ml folder you have the data source this is the base base class the data base class obviously everything is empty for you to to code okay but just structure to guide you through the process and you have spreadsheets if you have spreadsheets if you have for example things in spark you can do another another class to read it or something like that then you have the Moodle motor folder with the metrics class you have symmetric minimal percentage error a custom squirty one to mean absolute percentage error regression regression analysis a method to do cross-validation anyway so it's there you have this trainer a class that we talked about and we left a trainer s colourant as an example to you but the idea is that if you have another algorithms or than other methods you can implement a class like this like trainer there's a flow trainer piperj trainer cat boost or whatever you like and the wrapper that is the object of the model after all the process is done so you have the structure here it contains the model itself it contains the pre-processing the matrix the columns endo in or and you have also when the model was ran when the experiment was right and you have predict class predict methods save methods and stuff this is there and here you have a notebook I'm not gonna open here in DES code because I don't find it very attractive but anyway it's an example notebook I can show you the implemented in a bit then you have sorry pre-processing folder with the normally with the normalizer class this is very useful in the in the projects you have a pre-processing class so you're gonna build here all your processing steps and a text vectorizer if you are going to do some natural language processing stuff okay and then at the last you have these tests our I have already showed you that okay so I'm gonna show you and already implemented and already implemented project yeah and I think one thing that this really shows is that idea that you were going for creating structure around your projects and making sure that things are clear so that if someone is the hero of a project and something happens to that person then anybody else on the team can pick it up and know where things are and what is happening and I think you've done that really well you've been able to separate things and make sure that everything is very clear so it standardizes the process yeah the main issue is that we have lots of people and if we have if we all use the same structure the same idea for building them all those anyone can get to any point and help if you want if you if we have to debug it you know exactly where you are and it's easy to to the back so I'm gonna just looks just gonna get into our what kind of problem is this our Konev oh this is the famous Titanic model would be the model so here we have I'm gonna show you very quickly you have the day you have the data here it raw the train dot CSV you have let me show you okay so first you have data source we have spreadsheet implemented here so it's just reading the data with the with the columns that we need just to save some some some memory then we have pre-processing yeah this is our pre-processing a very simple pre-processing we have a cleansing methods that is cleaning that is transforming dropping a knees dropping missing values those kind of stuff and we have a category encoding step here very very simple project just as an example then we have our trainer so our trainer this we didn't change it's exactly the same trainer scikit-learn class that we have in the other the daanyik limited version okay so it already does cross validation and stuff and we have the wrapper the wrapper is exactly the same okay now what I want to show you is all we have the tests this is very important okay so if we have if we want to do this we we have to have we have to be in source tests folder and it you just like my test we use podcasts itness everything for you we have for pass tests some warnings this is due to to the application regarding some dependencies but anyway everything seems fine so no anyway mornings for now oh sorry we'll overlook the warnings for now oh yeah yeah it's just yeah this is like duplications of some dependencies that people have to to have to have to have two things and then we we have lots of tests of reading data cleaning data if the project is okay if the category encoding is doing fine anyway that's not a there is no rest of no asset for you to do unit testing it's in a data science project this is very customizable no and you have the Train the main of the main your main function here now the real deal where everything comes together so we have here I'm using a server decks in the cloud and here I have all the steps I read the data I do the pre-processing clean Pythagorean coding this is done in a notebook I'm going to show you right away then we are testing three algorithms random classifier gradient boosting and logistic regression random forest classifier and we are I forgot who asked me about logging in here we set the experiment with the project's name and when we are making our mo for run here we are logging the algorithms we are logging all the metrics and we have a function that gets all the metrics for you at once and we log the module so I have a ml full server it's in AWS right now to run it if you have this trained of Pi file you can just go hermione train she's going to look oh I have to be in the right form to be in source folder so do her Miami train then she's going to train the model and save everything to log everything to ml flow this should not take long and then I'm gonna start at Jupiter look just to show you at the example notebook it should not take long anyway okay it's that obviously we're not running in our flow locally so you can ignore localhost and then I have MF already here so we have the Titanic project we have built three models we have some runs that I did before so this is the three that we have just done right now and for those of you who already know ml flow this is good old ml flow so you can get in here compare compare your runs see what's going on accuracy is not a very good metric to classification I prefer f1 or or AUC anyway so it seems that oops it seems that random random forest you know forest gradient boosting is our winner anyway so you can check this ones and you have the models here if you want to use them off already a beauty in API you just registered this model and do like one terminal comment it's very easy and just to show you the last thing I'm gonna show you Jupiter laughs oops oh I have to con the deactivate because I don't have to put all that installed in this environment and should be pro-life okay it should start in a bit there we go yes so while we're waiting to show you Jupiter you spin up if anybody has any questions feel free to jump in or throw them in the chat and yeah so you just want to let us know what you think of Hermione give us some feedback because this looks really cool to me I think you're solving a lot of great problems that people have him and yeah it's very exciting I imagine you put a lot of time into this yeah as I said it's it's not something that we we decided one day that we would start building it was born like for some years project over project making mistakes learning from them and gradually Hermione was made you know this is the notebook that we left as an example for everyone so we have in the beginning this extensions that we have if you want to use like for example plotly or those progress bars on tiki TN for example we just made this as an example for you to do that and also we have to change the system path for the structure to work so when you're going from this point is on you so in the not implemented version code stops here and they implemented you have this example so you're going to hold your classes like this ml dot data source tour spreadsheet you import the spreadsheet class and the pre-processing class and whatever then you get the data check the columns do the pre-processing part and then prepare X&Y do train to split train your model do predictions this is everything that you you just copy here and put in your train in your train high script and then you make your predictions and test if it's going well and do test errors this is a we work a lot in Jupiter in Japan notebooks Jupiter labs so this is just an example for you to structure your your Jupiter notebook so so guys this is this is for mining so this that's it it's awesome we hope to get a lot of feedback from you of what you liked and what you didn't like so much and we hope that with you help we can improve this this tool yeah totally and if anybody wants to use it the link is in in the chat right now we're also putting the link into slack into the slack community that we've got