Reproducability flaws in end to end Machine Learning debugging

Key Takeaways

you cannot have um one piece in a puzzle being reproducible and other pieces are not being reproduced that doesn't work so they have to harmonize and align the rest of the pieces whether it is the data collection piece or the the modeling piece to be consistent and achieve the overall end-to-end objectives and this is still somewhat of a a new uh area and we uh even though we don't uh ask them the customers have uh encouraged us to be a lot more prescriptive encourage us to tell their teams to be you know to organize their work a certain way simply because we we have the opportunity to see many companies and many groups yeah and it makes total sense you don't want one thing to be very shiny and nice and then the rest is put together by a string and it's not working and you have problems with it so let's see how that would work and consistent with what flavio was saying right although this is high risk stuff let me give you a copyright uh example to help understand the situation see that there's a customer of mine um uh that actually is taking inventory bets on the products they have to decide whether to uh uh you know keep nine units or 50 units of a certain product and for that they have to do forecasting and the understand the demand and so on now uh you know it was a uh recently uh they they started noticing uh some unusual behavior of the models um then when they started to investigating the first thing that they go to is that through the modeling code that's like it learned you know whatever and there's a bit of notebook and from there they start tracing it back all the way and it turned out that what was happening is that the java application code that was there at the which was the source of a lot of this data uh they made some implicit uh decisions um about how to handle products from some geography and not from and how to handle products from a different geography there was some element there but this end-to-end uh debugging process um you know they they they struggled to debug actually what was what was the model that was actually put into production the precise code because the code itself was moving very fast um when they were able to come to the scribble platform itself which whose work ends at the data set generation from here we could go back and say this is exactly what we uh where whatever sources of the data was there because as a matter of routine which we keep track of the metadata we have a linear search all of those kinds of things it was clearly apparent that you cannot have this end-to-end debugging ability with black holes in the middle and in this case they were able to find the they had locked the data set that they had used for the modeling so from there we were able to change it down to the java source and then fix it and it is not you know this happens actually quite frequently so i believe that reproducibility and explainability will be driven not so much by the asks of third parties like regulatory authorities but just because you have the need to understand and debug your own data science systems that you have built if you don't understand what begs it is taking when and why you won't be able to manage the risk wow that's that's really thought provoking the change will come from the internal side as opposed to regulations