ML/AI Maturity Expectations

MLOps.community · Beginner ·🧠 Large Language Models ·1y ago

Key Takeaways

The video discusses the shift in machine learning (ML) from an experimental phase to a production-ready expectation, with a focus on Large Language Models (LLMs) and their potential applications in business, highlighting the differences in ROI calculation, risk perception, and reliability between traditional ML projects and LLM initiatives

Full Transcript

Well, I started working on this stuff at at Google in 2013. Yeah. So, it's been over a decade. Yeah. It's been a long time. So, this is why we could talk about these like adoption curves, right? Like a lot of ML projects back in the day were very experimental. They were back in the day, I'm talking about like even 2018, 2019, people were like, "We would love to find a way to figure out how to use machine learning in our business, but we don't really know like what to do with this. Can you help us?" today it's like we know this is possible and we just got to get this stuff into production. Uh here's here's exactly what we need. And so that means that these types of projects have a much higher expectation of making it to production. Right? So the the like kind of perception and the ROI calculation for this stuff is pretty different. It's like it's like a lower risk. It's like we're confident we can do this and uh we kind of like know what the value is going to be. We know we can reduce fraud. If we reduce fraud by 1%, it's going to give us $10 million a year or, you know, whatever the equivalent is. We get x percent more clickthroughs on the website, blah blah blah, right? And then in the LLM world, it's a little bit different. It's like, we don't really know if we can do this. It's kind of like a skunk works project. We don't even know if this technology can do the thing that we aspire for it to do. And it's the reliability that we need with the forget even about like the enterprise like readiness part of it. It's just literally like we don't even know if the thing we want to do is possible. So there's a lot of those kind of projects because people don't really understand what's fully possible

Original Description

ML used to be an experiment. Now it’s an expectation. Back in 2018, machine learning was a moonshot. Companies were *curious*—"Can we use this in our business somehow?" Fast forward to today: “We *know* ML works. We *know* the ROI. We just need it in production.” In this episode, Michael Del Balso (Co-founder of Tecton) breaks down how the bar has shifted—from exploratory research to results-driven delivery—and how LLMs are resetting that bar *again*. Where classic ML is now expected to be low-risk and high-impact, LLM projects often start out as uncertain, skunkworks-style bets: “Can this tech even do what we want?” “Will it be reliable enough?” “Is this even *possible*?” It’s a fascinating contrast—and a reminder that AI maturity isn’t one-size-fits-all. Full episode here: **Hard Learned Lessons from Over a Decade in AI // Michael Del Balso // MLOps Podcast #321** 🎧 https://home.mlops.community/home/videos/hard-learned-lessons-from-over-a-decade-in-ai
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from MLOps.community · MLOps.community · 0 of 60

← Previous Next →
1 Our 1st MLOps Meetup // Luke Marsden // MLOps Meetup #1
Our 1st MLOps Meetup // Luke Marsden // MLOps Meetup #1
MLOps.community
 2 Remote Collaboration as a Data Scientist
Remote Collaboration as a Data Scientist
MLOps.community
 3 MLOps Manifesto with Luke Marsden from Dotscience
MLOps Manifesto with Luke Marsden from Dotscience
MLOps.community
 4 MLOps lifecycle description
MLOps lifecycle description
MLOps.community
 5 What Does Best in Class AI/ML Governance Look Like in Fin Services? // Charles Radclyffe // MLOps #2
What Does Best in Class AI/ML Governance Look Like in Fin Services? // Charles Radclyffe // MLOps #2
MLOps.community
 6 Life purpose and too many spreadsheets
Life purpose and too many spreadsheets
MLOps.community
 7 Explainability, Black boxes and EU white paper on reproducibility
Explainability, Black boxes and EU white paper on reproducibility
MLOps.community
 8 Hierarchy of Machine Learning Needs // Phil Winder // MLOps Meetup #3
Hierarchy of Machine Learning Needs // Phil Winder // MLOps Meetup #3
MLOps.community
 9 Automatically Retrain Machine Learning Models? Are best practices worth it?
Automatically Retrain Machine Learning Models? Are best practices worth it?
MLOps.community
 10 Building an MLOps Team? Key ideas to keep in mind
Building an MLOps Team? Key ideas to keep in mind
MLOps.community
 11 Hierarchy of MLOps Needs
Hierarchy of MLOps Needs
MLOps.community
 12 Bare necessities for getting an ML model into production
Bare necessities for getting an ML model into production
MLOps.community
 13 MLOps and Monitoring
MLOps and Monitoring
MLOps.community
 14 How Phil Winder got into Data Science and Software Engineering
How Phil Winder got into Data Science and Software Engineering
MLOps.community
 15 Provenance and Reproducibility in Machine Learning; what is it and why you need it?
Provenance and Reproducibility in Machine Learning; what is it and why you need it?
MLOps.community
 16 Friction Between Data Scientists and Software Engineers
Friction Between Data Scientists and Software Engineers
MLOps.community
 17 MLOps Problems in different size companies
MLOps Problems in different size companies
MLOps.community
 18 ML tooling in large companies
ML tooling in large companies
MLOps.community
 19 ML Platforms - The build vs buy question
ML Platforms - The build vs buy question
MLOps.community
 20 ML Services Gateway at SurveyMonkey
ML Services Gateway at SurveyMonkey
MLOps.community
 21 Message buses, Async and sync architecture
Message buses, Async and sync architecture
MLOps.community
 22 MLOps #4: Shubhi Jain - Building an ML Platform @SurveyMonkey
MLOps #4: Shubhi Jain - Building an ML Platform @SurveyMonkey
MLOps.community
 23 Hybrid Data Science Teams @SurveyMonkey
Hybrid Data Science Teams @SurveyMonkey
MLOps.community
 24 How do you handle ML version control at SurveyMonkey
How do you handle ML version control at SurveyMonkey
MLOps.community
 25 Doing ML with Personal Information
Doing ML with Personal Information
MLOps.community
 26 Evolution of the ML feature store @SurveyMonkey
Evolution of the ML feature store @SurveyMonkey
MLOps.community
 27 Developing a Machine Learning Feature Store
Developing a Machine Learning Feature Store
MLOps.community
 28 Auto retrain ML models is not the question
Auto retrain ML models is not the question
MLOps.community
 29 3 key parts to Machine Learning monitoring
3 key parts to Machine Learning monitoring
MLOps.community
 30 MLOps Meetup #6: Mid-Scale Production Feature Engineering with Dr. Venkata Pingali
MLOps Meetup #6: Mid-Scale Production Feature Engineering with Dr. Venkata Pingali
MLOps.community
 31 MLOps meetup #5 High Stakes ML: Active Failures, Latent Factors with Flavio Clesio
MLOps meetup #5 High Stakes ML: Active Failures, Latent Factors with Flavio Clesio
MLOps.community
 32 MLOps: Airflow Pros and Cons
MLOps: Airflow Pros and Cons
MLOps.community
 33 Specific challenges in Machine Learning
Specific challenges in Machine Learning
MLOps.community
 34 Current State Of Machine Learning
Current State Of Machine Learning
MLOps.community
 35 Humans in the Loop are a defining factor in Machine Learning
Humans in the Loop are a defining factor in Machine Learning
MLOps.community
 36 Learning from real life Machine Learning failures
Learning from real life Machine Learning failures
MLOps.community
 37 Survivorship Bias in machine learning tutorials
Survivorship Bias in machine learning tutorials
MLOps.community
 38 Swiss Cheese model in Machine Learning
Swiss Cheese model in Machine Learning
MLOps.community
 39 Resume driven development in Machine learning & software engineering
Resume driven development in Machine learning & software engineering
MLOps.community
 40 Who has the highest standards in ML?
Who has the highest standards in ML?
MLOps.community
 41 Venkata Pingali of Scribble Data Thoughts on the Current State of Machine Learning
Venkata Pingali of Scribble Data Thoughts on the Current State of Machine Learning
MLOps.community
 42 Dependable data and being able to Trust in your Data with Venkata Pengali of Scribble Data
Dependable data and being able to Trust in your Data with Venkata Pengali of Scribble Data
MLOps.community
 43 Speed, Trust, Evolution and Scale in MLOps
Speed, Trust, Evolution and Scale in MLOps
MLOps.community
 44 More difficult transition for data scientists to become ML engineers
More difficult transition for data scientists to become ML engineers
MLOps.community
 45 How many models in prod til I need a dedicated ML platform?
How many models in prod til I need a dedicated ML platform?
MLOps.community
 46 Deeper thinking from data scientists around platform blackholes
Deeper thinking from data scientists around platform blackholes
MLOps.community
 47 Checkpointing, metadata, and confidence in your data
Checkpointing, metadata, and confidence in your data
MLOps.community
 48 Adjacent usecases and multistep feature engineering
Adjacent usecases and multistep feature engineering
MLOps.community
 49 Standardization of Machine Learning tools like in Software Engineering with Venkata Pingali
Standardization of Machine Learning tools like in Software Engineering with Venkata Pingali
MLOps.community
 50 Reproducability flaws in end to end Machine Learning debugging
Reproducability flaws in end to end Machine Learning debugging
MLOps.community
 51 3rd wave of data scientists
3rd wave of data scientists
MLOps.community
 52 MLOps meetup #7 Alex Spanos // TrueLayer 's MLOps Pipeline
MLOps meetup #7 Alex Spanos // TrueLayer 's MLOps Pipeline
MLOps.community
 53 MLOps Meetup #8 Optimizing Your ML Workflow with Kubeflow 1.0
MLOps Meetup #8 Optimizing Your ML Workflow with Kubeflow 1.0
MLOps.community
 54 Are Kubeflow and Airflow complementary?
Are Kubeflow and Airflow complementary?
MLOps.community
 55 Why Kubeflow gained so much traction=open community
Why Kubeflow gained so much traction=open community
MLOps.community
 56 Who decides the dirrection of Kubeflow
Who decides the dirrection of Kubeflow
MLOps.community
 57 What do Kubeflow and Arrikto do and how do they work together?
What do Kubeflow and Arrikto do and how do they work together?
MLOps.community
 58 Versioning your ML steps with Kubeflow
Versioning your ML steps with Kubeflow
MLOps.community
 59 Machine Learning Lifecycles//Perception vs Reality
Machine Learning Lifecycles//Perception vs Reality
MLOps.community
 60 Kubeflow vs SageMaker in Machine Learning
Kubeflow vs SageMaker in Machine Learning
MLOps.community

The video discusses the evolution of ML from an experimental phase to a production-ready expectation, highlighting the differences between traditional ML projects and LLM initiatives, and emphasizing the importance of reliability and ROI calculation in LLM applications

Key Takeaways
  1. Identify the current state of ML adoption in your organization
  2. Assess the potential applications of LLMs in your business
  3. Evaluate the reliability and ROI of LLM initiatives
  4. Design and implement ML pipelines for production-ready ML
  5. Monitor and evaluate the performance of LLMs in production
💡 The shift from experimental to production-ready ML requires a focus on reliability, ROI calculation, and risk perception, particularly in LLM initiatives

Related Reads

📰
OWASP LLM05:2025 : Improper Output Handling — When the Model’s Response Becomes the Attack
Learn about OWASP LLM05:2025, a security risk related to improper output handling in LLMs, and how to mitigate it
Medium · LLM
 📰
AI and UX Opinion — Episode 006
Learn about the current state of AI and UX design, focusing on prompts, and why it matters for designing effective user experiences
Medium · UX Design
 📰
The LLM Thought a Dollar Was Still ₦450: Building a Car Pricing Engine for a Market With No Data
Learn how to build a car pricing engine using LLMs for a market with no data, and understand the challenges of working with outdated currency exchange rates
Dev.to · Chichebe John
 📰
Cross-Modal Knowledge Distillation for precision oncology clinical workflows under multi-jurisdictional compliance
Apply cross-modal knowledge distillation to improve precision oncology clinical workflows while ensuring multi-jurisdictional compliance
Dev.to AI
Up next
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)
Watch →