Praveen Kanamarlapud - Kernel Life Cycle Management | JupyterCon 2020
Skills:
ML Pipelines70%
Key Takeaways
This video covers kernel life cycle management in Jupyter notebooks, including environment management and updates
Full Transcript
Jhal hello hi everyone straight away stop doing discuss life cycle in this mentality of that should need you want to see the question software engineer in subscribe software engineering exam paper me aaye weeks seniors patients in with twitter a name this contest report subscribe now to receive new updates reviews and news report discuss here details let's look at how its traditional setup and how does people who want to thank you to you to you to point to subscribe support to subscribe our quote prevalence specific this challenge Krishna was with different different different uses all this demand ding dong ding consistently particularly you want to see the problem remind to subscribe understand well existing producers are already created so consistency consistency running bluetooth setting turn on bluetooth setting on the different uses a improve word changes on the other hand today and stopped against challenge provides Resources Demand Vidmate Service And Different Different Subscribe Key Request You To Provide You With Valid Can Solve All Problems Subscribe Suvva Reddy Basically Stitching And Software Install Dependency In Boy Student Users Can Donate Consistent With Issues In Operating System Of Central Government For Example Computer Video Subscribe Now Desk A More About Five Words Architecture Of Mode Poochega Davidson Motorcycle One Is Pain In Left Over 231 372 In Princess Cut Ke Juice Start From 6 To 9 Of Jupiter National Survey Subscribe YouTube Channel Ko Aur Subscribe Ke Peechhe Ab Uske Zaroor Aandhi Tarzan Cigarette This Videos National Executive For Fasting In Five And Builders Pacific Subscribe Launched From Subscribe Chapter Note Book Launch A That Edison Give This Aspect Reviews And Appointed Chief Minister Will Hotspot On Deposit Best Army Canteen Type Andhra Vis Chairman Bigg Boss Storage Guru Subscribe Karna Biltu Simple Storage Subscribe and subscribe the Channel Subscribe and subscribe and subscribe the Channel Subscribe Vriddhi Cognitive addiction fog and demand and cigarette clear and then scholarship inside soil song playlist enabled in these events which features addressed Shakya taraf band kar do hua hai iss dog cricket cricket match and isko pause kinnar like I discussed in Mr. Recharges here a little every youngster death in this office on Amazon west dog images and this is just and images scanners effect of at least hui thi and Payal Ambedkar image appeal and users in the developed world cup 2019 se is active center kind absolutely Pandu Chinese street interview typing this all indiscriminately subscribe to the channel and subscribe channel subscribe subscribe toe is definitely friendly of course peppermint advanced pimple fog hi fog meaning which continues for Anupam wedding devotion in the great kar do is also in worse condition last week wish this method that Mr. Cheerwa because do Mr. Midas point delivery system in time Ismail Active Behaving Systems which can live with delivery system is simple that Chandra Medical Management Institute Campus width my videos you can watch this duration permission screen aloe vera different on WhatsApp this image unlock store for this channel dog means the thing time addition for tracking system that if media club stock images of birds system pay t-shirt train dadar turn on the images of information opposition party leader Anil Vij system busy busy busy busy busy busy busy that leave me family doctor in the field of and music comment significance students and asked for family waste oil prohibitions but good jobs created this channel subscribe system changes subscribe this Video give chief media form financial slow down coat of animals pen card loop subscribe our YouTube Channel and subscribe the Channel subscribe The Channel Chief Ministers on karo jhal dhoa lo good department department scholarship management hai that chicken trusted half Vidhu channel management service liye difficult meanwhile pimpl you talk with all the subscribe and subscribe the Channel Please subscribe and subscribe the Channel that Aditya Jain rocks talks to British management service liye this he his izzat loot Lodhi and if channel is of using so from previous center Paun inch Sarpanch Yoga Mirch fixed on launch and indiscriminate withdrawal GPS off A B C and D were appointed simple which is inside from which you subscribe power steering subscribe my channel system change in more bill this a this five eight seats can be obtained New York Rangers defeated by Mouni Roy and determine your who Rameshchand state book for this channel do subscribe and do not forget to subscribe my channel and subscribe to management information within a person please like in written test for not familiar with the third hour channel like and subscribe subscribe do subscribe to this dynasty this properties job me will be available for this system disturbance such cross looting [music] that you all generated in another match tagged with Jupiter and boiled accused register registering is quick and use this channel subscribe don't forget to subscribe to the channel and subscribe subscribe the channel a committee will answer vacancy nikli atak-atak admit card a cigarette in the channel Beedi Respective Channel also launched Ansh pistol and computers requested the vanities users can advocate the can settle richest which all information like what is the name of the power is remedy version reducing find the world is page type with now skand see you are you sir number of competition recipes hello friends now its meter fuse pair peeth under list computers page sadi ko on to make changes to channel like this point to Delhi channel subscribe nahi to subscribe like Share and subscribe the Channel North Sagar Bundi District mere paas maine se update basically chalti manifest information fixed vitamin Ashraf member of competition updated on playlist ek lota vichar in his early discussed in the Air India Management addressed sexual isi enrollment most doctors pe doctor nurse medico minutes for mez thi and mez thi and mez pe channel like this channel subscribe and click on subscribe my channel then thank you president Lord Ganesha listen bilkul na chipke na blind lover in which users can fabric in Chief Minister of images from distance trains which can I do if you feel like 105 Gautam Krishna in this award for this thank you thank you to preserve channel subscribe school me jhal
Original Description
Brief Summary
Notebooks has seen an enthusiastic adaptation over time. With increased adaptation comes new challenges - how to manage different environments for kernels, updating an environment without breaking anything, maintain different versions of kernels for dev and production, share a notebook which will run everywhere. This talk will explain how kernel life cycle management will simplify these challenges
Outline
Jupyter Notebook has seen an enthusiastic adoption among the data science and engineering community, to an extent where it has become a default environment for all the research. Many enterprises use notebooks as a de facto tool for all their data and machine learning development activities, and some of them are even productionalizing the notebooks. Majority of data scientists and engineers at PayPal use notebooks for all development activities. In fact, the entire data and AI/ML platforms are exposed through notebooks.
With increased usage comes new challenges. As thousands of users use notebooks, there is a need for different environments to be used for different use cases and governance around these environments. Here are some of the challenges we faced • It has become a management nightmare to support different python versions and packages for different environments • Sharing a notebook is not simple with different notebooks environments • Hard to maintain different kernels and versions for different environments like development and production
So, we implemented a solution named kernel life cycle management to solve some of these challenges. Praveen and Ayushi will explain • Kernel Life Cycle Management – how life cycle management was introduced for kernels at PayPal by leveraging Kubernetes and Docker • Sharing a notebook – How a shared notebook can be executed for any user • Making changes to kernel environment without breaking scheduled notebooks in production • Kernel as a docker image for isolating the environments
Background knowledge fo
Watch on YouTube ↗
(saves to browser)
Sign in to unlock AI tutor explanation · ⚡30
Playlist
Uploads from JupyterCon · JupyterCon · 58 of 60
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
▶
59
60
Interview Joshua Patterson NVIDIA
JupyterCon
Dave Stuart - Jupyter as an Enterprise “Do It Yourself” (DIY) Analytic Platform | JupyterCon 2020
JupyterCon
Jeffrey Mew - Supercharge your Data Science workflow | JupyterCon 2020
JupyterCon
Michelle Ufford- Supercharging SQL Users with Jupyter Notebooks | JupyterCon 2020
JupyterCon
Alan Yu - What we learned from introducing Jupyter Notebooks to the SQL community | JupyterCon 2020
JupyterCon
Chris Holdgraf- 2i2c: sustaining open source through hosted Jupyter infrastructure | JupyterCon 2020
JupyterCon
Yiwen Li - Intro to Elyra - an AI centric extension for JupyterLab | JupyterCon 2020
JupyterCon
Luciano Resende - What's new on Elyra - A set of AI centric JupyterLab extensions | JupyterCon 2020
JupyterCon
Alan Chin - Explore and Extend AI Pipeline Runtimes with Elyra and JupyterLab | JupyterCon 2020
JupyterCon
Eduardo Blancas- Streamline your Data Science projects with Ploomber | JupyterCon 2020
JupyterCon
Thorin Tabor - Democratizing the accessibility of computational workflows | JupyterCon 2020
JupyterCon
Simon Willison- Using Datasette with Jupyter to publish your data | JupyterCon 2020
JupyterCon
Brendan O'Brien - Using Qri (“query”) to fetch, query, combine and publish datasets.|JupyterCon 2020
JupyterCon
Georgiana Dolocan - Putting the JupyterHub puzzle pieces together | JupyterCon 2020
JupyterCon
Yuvi Panda- Running nonjupyter applications on JupyterHub with jupyter-server-proxy| JupyterCon 2020
JupyterCon
Richard Wagner- The Streetwise Guide to JupyterHub Security | JupyterCon 2020
JupyterCon
TamNguyen- Handling Custom Jupyter Data Sources | JupyterCon 2020
JupyterCon
Immanuel Bayer- ipyannotator - the infinitely hackable annotation framework | JupyterCon 2020
JupyterCon
Rebecca Kelly- A shared Python, R and Q Jupyter Notebook - A Quant Sandbox Dream |JupyterCon 2020
JupyterCon
Itay Dafna - Leap of faith: Transitioning from Excel to Jupyter-based applications | JupyterCon 2020
JupyterCon
Damián Avila - Using the Jupyterverse to power MADS | JupyterCon 2020
JupyterCon
Chiin Rui Tan- From Zero to Hero | JupyterCon 2020
JupyterCon
Firas Moosvi- Teaching an Active Learning class with Jupyter Book| JupyterCon 2020
JupyterCon
Daniel Mietchen- Jupyter in the Wikimedia ecosystem | JupyterCon 2020
JupyterCon
Qiusheng Wu- How Jupyter and geemap enable interactive mapping and analysis | JupyterCon 2020
JupyterCon
Stephanie Juneau- Jupyterenabled astrophysical analysis for researchers and students|JupyterCon 2020
JupyterCon
Denton Gentry- The Care and Feeding of JupyterHub for Climate Solution Models| JupyterCon 2020
JupyterCon
Tingkai Liu- FlyBrainLab: Interactive Computing in the Connectomic/Synaptomic Era | JupyterCon 2020
JupyterCon
Kunal Bhalla- A Notebook Style Guide| JupyterCon 2020
JupyterCon
Julia Wagemann - How to avoid 'Death by Jupyter Notebooks' | JupyterCon 2020
JupyterCon
David Pugh - Best practices for managing Jupyter-based data science | JupyterCon 2020
JupyterCon
Karla Spuldaro - Debugging notebooks and python scripts in JupyterLab | JupyterCon 2020
JupyterCon
Shreyas Dalia - assert browserTest == True # Frontend Testing JupyterLab | JupyterCon 2020
JupyterCon
Chris Holdgraf - The new Jupyter Book stack | JupyterCon 2020
JupyterCon
Hamel Husain - Fastpages - A new, open source Jupyter notebook blogging system | JupyterCon 2020
JupyterCon
Marc Wouts - Jupytext: Jupyter Notebooks as Markdown Documents | JupyterCon 2020
JupyterCon
Sheeba Samuel- ProvBook |JupyterCon 2020
JupyterCon
Philipp Rudiger - To Jupyter and back again | JupyterCon 2020
JupyterCon
Jacob Tomlinson - What is my GPU doing? | JupyterCon 2020
JupyterCon
Afshin Darian - A visual debugger in Jupyter | JupyterCon 2020
JupyterCon
Eric Charles - Jupyter Real Time Collaboration| JupyterCon 2020
JupyterCon
Devin Robison - Optimizing model performance | JupyterCon 2020
JupyterCon
Junhua zhao - PayPal Notebooks: ML & Data Science experience | JupyterCon 2020
JupyterCon
April Wang - Redesigning Notebooks for Better Collaboration | JupyterCon 2020
JupyterCon
Bryan Weber - Distributing and Collecting Jupyter Notebooks for Manual Grading| JupyterCon 2020
JupyterCon
Georgiana Dolocan - The Littlest JupyterHub distribution | JupyterCon 2020
JupyterCon
Tim Metzler - Electronic Examination using Jupyter Notebook | JupyterCon 2020
JupyterCon
Blaine Mooers - Why develop a snippet library for Jupyter in your subject domain? | JupyterCon 2020
JupyterCon
Ryan Abernathey - Cloud Native Repositories for Big Scientific Data | JupyterCon 2020
JupyterCon
Tanya Rai - Introducing Bento: Jupyter Notebooks @ Facebook | JupyterCon 2020
JupyterCon
Kenton McHenry - From Papers to Notebooks | JupyterCon 2020
JupyterCon
Ryan Herr - After model.fit, before you deploy| JupyterCon 2020
JupyterCon
Ana Ruvalcaba - Community building is a sustainability strategy | JupyterCon 2020
JupyterCon
Martin Renou - Xeus: an ecosystem of Jupyter kernels | JupyterCon 2020
JupyterCon
Michael Wilson - Teaching teenagers to understand Dark Energy | JupyterCon 2020
JupyterCon
Davide De Marchi - Voilà dashboards for policy support | JupyterCon 2020
JupyterCon
Marcos Lopez Caniego - ESASky's JupyterLab widget| JupyterCon 2020
JupyterCon
Praveen Kanamarlapud - Kernel Life Cycle Management | JupyterCon 2020
JupyterCon
Aaron Bray - Pulse Physiology Engine | JupyterCon 2020
JupyterCon
Aaron Watters - Using WebGL2 transform/feedback in Jupyter widgets | JupyterCon 2020
JupyterCon
More on: ML Pipelines
View skill →Related Reads
📰
📰
📰
📰
AWS Self-Healing Infrastructure: Catching and Fixing Server Crashes with AI Autopilot
Medium · DevOps
Understanding a Real AWS CodePipeline CI/CD Flow: Source Build Deploy"
Dev.to · Guna SantoshDeep Srivastava
How to Use Terraform with Python for Infrastructure
Dev.to · qing
Interactive containers and exec: get inside, run commands, get out
Dev.to · Javi Palacios
🎓
Tutor Explanation
DeepCamp AI