Fireside Chat #6: Operationalizing ML -- Patterns and Pain Points from MLOps Practitioners

Shreya Shankar is a computer scientist doing her PhD in databases at UC Berkeley. She was the first ML engineer at Viaduct, did research at Google Brain, and did software engineering at Facebook. In this fireside chat, Shreya joins Hugo Bowne-Anderson, Outerbounds’ Head of Developer Relations, to discuss her team’s recent paper Operationalizing Machine Learning: An Interview Study, and what they discovered about the common practices & challenges across organizations & applications in ML engineering. After attending, you’ll know about - The main tasks that people do in the Production ML lifecycle; - Key properties of ML workflow and infrastructure that dictate how successful deployments will be; - The biggest pain points faced by people deploying ML models to production; - What strategies ML engineers employ to sustain model performance once deployed; - What the biggest opportunities for future MLOps tools and research are; And much more! The fireside chat will be followed by an AMA with Shreya and Hugo at slack.outerbounds.co. 00:00 Prelude 04:42 The fireside chat begins! 07:20 Why it's important to talk now about patterns and pain points from MLOps practitioners 14:10 The main tasks in the production ML lifecycle 19:29 The 3 factors that determine the success of ML projects 27:30 Models to roll back to when things go wrong, shadow models, and challenger models 31:11 "90% of models don't make it to prod" can be a good thing! 33:16 Trade-offs and synergies between Velocity, Validation, and Versioning in machine learning 36:20 What using notebooks actually prioritizes 41:45 Is the premise of data-centric AI flawed? 47:57 The role of subject matter experts and domain expertise in ML 50:33 Software engineering versus machine learning engineering 55:40 What even is a 10x ML engineer? 56:52 The biggest opportunities for MLOps tool builders and research Find out more about how we think about MLOps, OSS, and human-centric data science tools here: https://outerbounds.