Versioning your ML steps with Kubeflow
Skills:
ML Pipelines80%
Key Takeaways
Explains versioning Machine Learning steps with Kubeflow and Arrikto
Full Transcript
so as the leading cue flow users have reported versioning of your ml steps enables for faster iteration exploration this enables you to quickly evaluate debug and improve your models let's take a quick look here at oh how this is impacted some reason doesn't want let me go forward there we go this particular model this is an example of a of a model that is you can run in a tutorial this is for the Kegel Titanic example which is a machine learning model that predicts who will survive the Titanic shipwreck and before this cuj with rock and kale it might take you twenty four steps to build this model potentially hundreds of hours in the first iteration hundreds of hours in the second iteration and really involve having a data scientist data engineers ml engineers and DevOps people once you build your code you need to pass it off to an ml engineer that knows about how do I create a docker container and run that in kubernetes as well as be Alderon a pipe lines DSL rock and kale simplify all this take the time down from hundreds of hours to tens of hours and allow the data scientists basically to be able to run these these workflows right from the Jupiter notebook and I believe with that this will kind of show you what the difference of where we're going right before you'd write your ML code you'd write some docker containers write some DSL code compile it upload it and then round your pipeline now you write your code you tag it your cell and then you wait a click of a button and you can run your pipeline
Original Description
How does versioning your Machine Learning steps work with Kubeflow and Arrikto? For our 8th MLOps community meetup Josh Bottum VP of Arrikto and Kubeflow community product manager answers this question for us. This is taken from a longer conversation you can find here: https://youtu.be/jXRbj5xnBy4
Linkedin, Spotify, Volvo, JP Morgan, and many other market leaders are leveraging Kubeflow to simplify the creation and the efficient deployment of Machine Learning models on Kubernetes. This presentation will provide an update on the Kubeflow 1.0 release, and review the Community’s best practices to support Critical User Journeys, which optimize ML workflows.
As a data scientist will often need to build (and save) hundreds of variants of their model, this session will provide a deeper dive into how an integrated storage solution simplifies model-building and increases ML productivity. The presentation will examine how to optimize the daily workflows of data scientists, and eliminate complex and time-consuming manual tasks. The talk will also highlight how efficient Kubeflow operations rely on Kubernetes storage primitives, such as Dynamic Volume Provisioning, Persistent Volumes and StatefulSets. This integrated solution simplifies the configuration, operations and data protection for Kubeflow and generic K8s stateful apps in production-grade, multi-user environments.
In this chat we sit down with Josh Bottum, a Kubeflow Community Product Manager. His Community responsibilities include assisting users to quantify Kubeflow business value, develop critical user journeys (CUJs), triage incoming user issues, prioritize feature delivery, write release announcements and deliver Kubeflow presentations and demonstrations.
Mr. Bottum is also a VP of Arrikto. Arrikto simplifies storage operations for stateful Kubernetes applications by enabling efficient local storage architectures with data durability and portability. Arrikto is a core code contributor to Kubeflow.
Watch on YouTube ↗
(saves to browser)
Sign in to unlock AI tutor explanation · ⚡30
Playlist
Uploads from MLOps.community · MLOps.community · 58 of 60
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
▶
59
60
Our 1st MLOps Meetup // Luke Marsden // MLOps Meetup #1
MLOps.community
Remote Collaboration as a Data Scientist
MLOps.community
MLOps Manifesto with Luke Marsden from Dotscience
MLOps.community
MLOps lifecycle description
MLOps.community
What Does Best in Class AI/ML Governance Look Like in Fin Services? // Charles Radclyffe // MLOps #2
MLOps.community
Life purpose and too many spreadsheets
MLOps.community
Explainability, Black boxes and EU white paper on reproducibility
MLOps.community
Hierarchy of Machine Learning Needs // Phil Winder // MLOps Meetup #3
MLOps.community
Automatically Retrain Machine Learning Models? Are best practices worth it?
MLOps.community
Building an MLOps Team? Key ideas to keep in mind
MLOps.community
Hierarchy of MLOps Needs
MLOps.community
Bare necessities for getting an ML model into production
MLOps.community
MLOps and Monitoring
MLOps.community
How Phil Winder got into Data Science and Software Engineering
MLOps.community
Provenance and Reproducibility in Machine Learning; what is it and why you need it?
MLOps.community
Friction Between Data Scientists and Software Engineers
MLOps.community
MLOps Problems in different size companies
MLOps.community
ML tooling in large companies
MLOps.community
ML Platforms - The build vs buy question
MLOps.community
ML Services Gateway at SurveyMonkey
MLOps.community
Message buses, Async and sync architecture
MLOps.community
MLOps #4: Shubhi Jain - Building an ML Platform @SurveyMonkey
MLOps.community
Hybrid Data Science Teams @SurveyMonkey
MLOps.community
How do you handle ML version control at SurveyMonkey
MLOps.community
Doing ML with Personal Information
MLOps.community
Evolution of the ML feature store @SurveyMonkey
MLOps.community
Developing a Machine Learning Feature Store
MLOps.community
Auto retrain ML models is not the question
MLOps.community
3 key parts to Machine Learning monitoring
MLOps.community
MLOps Meetup #6: Mid-Scale Production Feature Engineering with Dr. Venkata Pingali
MLOps.community
MLOps meetup #5 High Stakes ML: Active Failures, Latent Factors with Flavio Clesio
MLOps.community
MLOps: Airflow Pros and Cons
MLOps.community
Specific challenges in Machine Learning
MLOps.community
Current State Of Machine Learning
MLOps.community
Humans in the Loop are a defining factor in Machine Learning
MLOps.community
Learning from real life Machine Learning failures
MLOps.community
Survivorship Bias in machine learning tutorials
MLOps.community
Swiss Cheese model in Machine Learning
MLOps.community
Resume driven development in Machine learning & software engineering
MLOps.community
Who has the highest standards in ML?
MLOps.community
Venkata Pingali of Scribble Data Thoughts on the Current State of Machine Learning
MLOps.community
Dependable data and being able to Trust in your Data with Venkata Pengali of Scribble Data
MLOps.community
Speed, Trust, Evolution and Scale in MLOps
MLOps.community
More difficult transition for data scientists to become ML engineers
MLOps.community
How many models in prod til I need a dedicated ML platform?
MLOps.community
Deeper thinking from data scientists around platform blackholes
MLOps.community
Checkpointing, metadata, and confidence in your data
MLOps.community
Adjacent usecases and multistep feature engineering
MLOps.community
Standardization of Machine Learning tools like in Software Engineering with Venkata Pingali
MLOps.community
Reproducability flaws in end to end Machine Learning debugging
MLOps.community
3rd wave of data scientists
MLOps.community
MLOps meetup #7 Alex Spanos // TrueLayer 's MLOps Pipeline
MLOps.community
MLOps Meetup #8 Optimizing Your ML Workflow with Kubeflow 1.0
MLOps.community
Are Kubeflow and Airflow complementary?
MLOps.community
Why Kubeflow gained so much traction=open community
MLOps.community
Who decides the dirrection of Kubeflow
MLOps.community
What do Kubeflow and Arrikto do and how do they work together?
MLOps.community
Versioning your ML steps with Kubeflow
MLOps.community
Machine Learning Lifecycles//Perception vs Reality
MLOps.community
Kubeflow vs SageMaker in Machine Learning
MLOps.community
More on: ML Pipelines
View skill →Related Reads
📰
📰
📰
📰
Musk thanks Micron for chips, and builds a $55bn fab to replace it
The Next Web AI
Jensen Huang calls the AI jobs panic ‘complete nonsense’, and takes aim at his peers
The Next Web AI
IMF says Africa has to keep lights on before it can bet on AI
TechCabal
What Does Job Security Even Look Like In 2026? It Starts With Skills
Forbes Innovation
🎓
Tutor Explanation
DeepCamp AI