Denton Gentry- The Care and Feeding of JupyterHub for Climate Solution Models| JupyterCon 2020

JupyterCon · Intermediate ·📊 Data Analytics & Business Intelligence ·5y ago
Brief Summary Imagine a climate solution model, originally constructed as spreadsheet, which grew in success and in scale such the sheer number of files and toil in keeping it all working has become a significant burden. We will discuss the process of re-implementing this model in Python, and the attributes that make JupyterHub well suited for the task. Outline This talk covers the two year adventure of a Python re-implementation of 100 climate solution models originally created in Microsoft Excel. The models show how anthropogenic climate change can be substantially reversed, by reducing and sequestering a trillion tons of CO2-equivalent greenhouse gas emissions through a number of solutions. The Excel versions of these models were used in the publication of the Project Drawdown book in 2017 and The Drawdown Review in 2020. Though Excel was a fine tool for the time, the needs of the project have grown. Excel worked well when there was one Excel file, and when there were five files, and somewhat less well when there were 30, and then 50, and now at 100 files the sheer degree of toil trying to manually update all of them is daunting. Researchers strongly avoid doing things which would require them to open the other 99 files. Nonetheless the model methodology in the Excel files is well structured, with separate sheets for major aspects of the model. The effort to reimplement the model in Python began in 9.2018 at https:github.comProjectDrawdownsolutions and was able to keep the overall structure. Roughly, each sheet in the original spreadsheet has been reimplemented as a Python class performing the same purpose. The Python implementation also brings the modern software infrastructure: extensive tests, distributed versioning, good web support, metrics about the codebase, etc. An outline of the talk: Model and methodology Excel sheet Python class code generation to extract data and settings from Excel files Why Jupyter and JupyterHub? scale up in how many resear
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from JupyterCon · JupyterCon · 27 of 60

1 Interview   Joshua Patterson NVIDIA
Interview Joshua Patterson NVIDIA
JupyterCon
2 Dave Stuart - Jupyter as an Enterprise “Do It Yourself” (DIY) Analytic Platform | JupyterCon 2020
Dave Stuart - Jupyter as an Enterprise “Do It Yourself” (DIY) Analytic Platform | JupyterCon 2020
JupyterCon
3 Jeffrey Mew - Supercharge your Data Science workflow | JupyterCon 2020
Jeffrey Mew - Supercharge your Data Science workflow | JupyterCon 2020
JupyterCon
4 Michelle Ufford- Supercharging SQL Users with Jupyter Notebooks | JupyterCon 2020
Michelle Ufford- Supercharging SQL Users with Jupyter Notebooks | JupyterCon 2020
JupyterCon
5 Alan Yu - What we learned from introducing Jupyter Notebooks to the SQL community  | JupyterCon 2020
Alan Yu - What we learned from introducing Jupyter Notebooks to the SQL community | JupyterCon 2020
JupyterCon
6 Chris Holdgraf- 2i2c: sustaining open source through hosted Jupyter infrastructure | JupyterCon 2020
Chris Holdgraf- 2i2c: sustaining open source through hosted Jupyter infrastructure | JupyterCon 2020
JupyterCon
7 Yiwen Li - Intro to Elyra - an AI centric extension for JupyterLab | JupyterCon 2020
Yiwen Li - Intro to Elyra - an AI centric extension for JupyterLab | JupyterCon 2020
JupyterCon
8 Luciano Resende - What's new on Elyra - A set of AI centric JupyterLab extensions | JupyterCon 2020
Luciano Resende - What's new on Elyra - A set of AI centric JupyterLab extensions | JupyterCon 2020
JupyterCon
9 Alan Chin - Explore and Extend AI Pipeline Runtimes with Elyra and JupyterLab | JupyterCon 2020
Alan Chin - Explore and Extend AI Pipeline Runtimes with Elyra and JupyterLab | JupyterCon 2020
JupyterCon
10 Eduardo Blancas- Streamline your Data Science projects with Ploomber | JupyterCon 2020
Eduardo Blancas- Streamline your Data Science projects with Ploomber | JupyterCon 2020
JupyterCon
11 Thorin Tabor - Democratizing the accessibility of computational workflows | JupyterCon 2020
Thorin Tabor - Democratizing the accessibility of computational workflows | JupyterCon 2020
JupyterCon
12 Simon Willison- Using Datasette with Jupyter to publish your data | JupyterCon 2020
Simon Willison- Using Datasette with Jupyter to publish your data | JupyterCon 2020
JupyterCon
13 Brendan O'Brien - Using Qri (“query”) to fetch, query, combine and publish datasets.|JupyterCon 2020
Brendan O'Brien - Using Qri (“query”) to fetch, query, combine and publish datasets.|JupyterCon 2020
JupyterCon
14 Georgiana Dolocan - Putting the JupyterHub puzzle pieces together | JupyterCon 2020
Georgiana Dolocan - Putting the JupyterHub puzzle pieces together | JupyterCon 2020
JupyterCon
15 Yuvi Panda- Running nonjupyter applications on JupyterHub with jupyter-server-proxy| JupyterCon 2020
Yuvi Panda- Running nonjupyter applications on JupyterHub with jupyter-server-proxy| JupyterCon 2020
JupyterCon
16 Richard Wagner- The Streetwise Guide to JupyterHub Security | JupyterCon 2020
Richard Wagner- The Streetwise Guide to JupyterHub Security | JupyterCon 2020
JupyterCon
17 TamNguyen- Handling Custom Jupyter Data Sources | JupyterCon 2020
TamNguyen- Handling Custom Jupyter Data Sources | JupyterCon 2020
JupyterCon
18 Immanuel Bayer- ipyannotator - the infinitely hackable annotation framework  | JupyterCon 2020
Immanuel Bayer- ipyannotator - the infinitely hackable annotation framework | JupyterCon 2020
JupyterCon
19 Rebecca Kelly- A shared Python, R and Q  Jupyter Notebook - A Quant Sandbox Dream |JupyterCon 2020
Rebecca Kelly- A shared Python, R and Q Jupyter Notebook - A Quant Sandbox Dream |JupyterCon 2020
JupyterCon
20 Itay Dafna - Leap of faith: Transitioning from Excel to Jupyter-based applications | JupyterCon 2020
Itay Dafna - Leap of faith: Transitioning from Excel to Jupyter-based applications | JupyterCon 2020
JupyterCon
21 Damián Avila - Using the Jupyterverse to power MADS | JupyterCon 2020
Damián Avila - Using the Jupyterverse to power MADS | JupyterCon 2020
JupyterCon
22 Chiin Rui Tan- From Zero to Hero | JupyterCon 2020
Chiin Rui Tan- From Zero to Hero | JupyterCon 2020
JupyterCon
23 Firas Moosvi- Teaching an Active Learning class with Jupyter Book| JupyterCon 2020
Firas Moosvi- Teaching an Active Learning class with Jupyter Book| JupyterCon 2020
JupyterCon
24 Daniel Mietchen- Jupyter in the Wikimedia ecosystem | JupyterCon 2020
Daniel Mietchen- Jupyter in the Wikimedia ecosystem | JupyterCon 2020
JupyterCon
25 Qiusheng Wu- How Jupyter and geemap enable interactive mapping and analysis | JupyterCon 2020
Qiusheng Wu- How Jupyter and geemap enable interactive mapping and analysis | JupyterCon 2020
JupyterCon
26 Stephanie Juneau- Jupyterenabled astrophysical analysis for researchers and students|JupyterCon 2020
Stephanie Juneau- Jupyterenabled astrophysical analysis for researchers and students|JupyterCon 2020
JupyterCon
Denton Gentry- The Care and Feeding of JupyterHub for Climate Solution Models| JupyterCon 2020
Denton Gentry- The Care and Feeding of JupyterHub for Climate Solution Models| JupyterCon 2020
JupyterCon
28 Tingkai Liu- FlyBrainLab: Interactive Computing in the Connectomic/Synaptomic Era  | JupyterCon 2020
Tingkai Liu- FlyBrainLab: Interactive Computing in the Connectomic/Synaptomic Era | JupyterCon 2020
JupyterCon
29 Kunal Bhalla- A Notebook Style Guide| JupyterCon 2020
Kunal Bhalla- A Notebook Style Guide| JupyterCon 2020
JupyterCon
30 Julia Wagemann - How to avoid 'Death by Jupyter Notebooks' | JupyterCon 2020
Julia Wagemann - How to avoid 'Death by Jupyter Notebooks' | JupyterCon 2020
JupyterCon
31 David Pugh - Best practices for managing Jupyter-based data science  | JupyterCon 2020
David Pugh - Best practices for managing Jupyter-based data science | JupyterCon 2020
JupyterCon
32 Karla Spuldaro - Debugging notebooks and python scripts in JupyterLab | JupyterCon 2020
Karla Spuldaro - Debugging notebooks and python scripts in JupyterLab | JupyterCon 2020
JupyterCon
33 Shreyas Dalia - assert browserTest == True # Frontend Testing JupyterLab  | JupyterCon 2020
Shreyas Dalia - assert browserTest == True # Frontend Testing JupyterLab | JupyterCon 2020
JupyterCon
34 Chris Holdgraf - The new Jupyter Book stack | JupyterCon 2020
Chris Holdgraf - The new Jupyter Book stack | JupyterCon 2020
JupyterCon
35 Hamel Husain - Fastpages - A new, open source Jupyter notebook blogging system | JupyterCon 2020
Hamel Husain - Fastpages - A new, open source Jupyter notebook blogging system | JupyterCon 2020
JupyterCon
36 Marc Wouts - Jupytext: Jupyter Notebooks as Markdown Documents | JupyterCon 2020
Marc Wouts - Jupytext: Jupyter Notebooks as Markdown Documents | JupyterCon 2020
JupyterCon
37 Sheeba Samuel- ProvBook |JupyterCon 2020
Sheeba Samuel- ProvBook |JupyterCon 2020
JupyterCon
38 Philipp Rudiger - To Jupyter and back again | JupyterCon 2020
Philipp Rudiger - To Jupyter and back again | JupyterCon 2020
JupyterCon
39 Jacob Tomlinson - What is my GPU doing? | JupyterCon 2020
Jacob Tomlinson - What is my GPU doing? | JupyterCon 2020
JupyterCon
40 Afshin Darian - A visual debugger in Jupyter | JupyterCon 2020
Afshin Darian - A visual debugger in Jupyter | JupyterCon 2020
JupyterCon
41 Eric Charles - Jupyter Real Time Collaboration| JupyterCon 2020
Eric Charles - Jupyter Real Time Collaboration| JupyterCon 2020
JupyterCon
42 Devin Robison - Optimizing model performance | JupyterCon 2020
Devin Robison - Optimizing model performance | JupyterCon 2020
JupyterCon
43 Junhua zhao - PayPal Notebooks: ML & Data Science experience | JupyterCon 2020
Junhua zhao - PayPal Notebooks: ML & Data Science experience | JupyterCon 2020
JupyterCon
44 April Wang - Redesigning Notebooks for Better Collaboration | JupyterCon 2020
April Wang - Redesigning Notebooks for Better Collaboration | JupyterCon 2020
JupyterCon
45 Bryan Weber - Distributing and Collecting Jupyter Notebooks for Manual Grading| JupyterCon 2020
Bryan Weber - Distributing and Collecting Jupyter Notebooks for Manual Grading| JupyterCon 2020
JupyterCon
46 Georgiana Dolocan - The Littlest JupyterHub distribution | JupyterCon 2020
Georgiana Dolocan - The Littlest JupyterHub distribution | JupyterCon 2020
JupyterCon
47 Tim Metzler - Electronic Examination using Jupyter Notebook | JupyterCon 2020
Tim Metzler - Electronic Examination using Jupyter Notebook | JupyterCon 2020
JupyterCon
48 Blaine Mooers - Why develop a snippet library for Jupyter in your subject domain? | JupyterCon 2020
Blaine Mooers - Why develop a snippet library for Jupyter in your subject domain? | JupyterCon 2020
JupyterCon
49 Ryan Abernathey - Cloud Native Repositories for Big Scientific Data | JupyterCon 2020
Ryan Abernathey - Cloud Native Repositories for Big Scientific Data | JupyterCon 2020
JupyterCon
50 Tanya Rai - Introducing Bento: Jupyter Notebooks @ Facebook | JupyterCon 2020
Tanya Rai - Introducing Bento: Jupyter Notebooks @ Facebook | JupyterCon 2020
JupyterCon
51 Kenton McHenry - From Papers to Notebooks | JupyterCon 2020
Kenton McHenry - From Papers to Notebooks | JupyterCon 2020
JupyterCon
52 Ryan Herr - After model.fit, before you deploy| JupyterCon 2020
Ryan Herr - After model.fit, before you deploy| JupyterCon 2020
JupyterCon
53 Ana Ruvalcaba - Community building is a sustainability strategy | JupyterCon 2020
Ana Ruvalcaba - Community building is a sustainability strategy | JupyterCon 2020
JupyterCon
54 Martin Renou - Xeus: an ecosystem of Jupyter kernels | JupyterCon 2020
Martin Renou - Xeus: an ecosystem of Jupyter kernels | JupyterCon 2020
JupyterCon
55 Michael Wilson - Teaching teenagers to understand Dark Energy | JupyterCon 2020
Michael Wilson - Teaching teenagers to understand Dark Energy | JupyterCon 2020
JupyterCon
56 Davide De Marchi - Voilà dashboards for policy support | JupyterCon 2020
Davide De Marchi - Voilà dashboards for policy support | JupyterCon 2020
JupyterCon
57 Marcos Lopez Caniego - ESASky's JupyterLab widget| JupyterCon 2020
Marcos Lopez Caniego - ESASky's JupyterLab widget| JupyterCon 2020
JupyterCon
58 Praveen Kanamarlapud - Kernel Life Cycle Management | JupyterCon 2020
Praveen Kanamarlapud - Kernel Life Cycle Management | JupyterCon 2020
JupyterCon
59 Aaron Bray - Pulse Physiology Engine | JupyterCon 2020
Aaron Bray - Pulse Physiology Engine | JupyterCon 2020
JupyterCon
60 Aaron Watters - Using WebGL2 transform/feedback in Jupyter widgets | JupyterCon 2020
Aaron Watters - Using WebGL2 transform/feedback in Jupyter widgets | JupyterCon 2020
JupyterCon

Related AI Lessons

Roblox Data Engineering Interview Questions: Full DE Prep Guide
Prepare for Roblox data engineering interviews with a focus on text-heavy product telemetry and search-related questions
Dev.to · Gowtham Potureddi
Tesla Data Engineering Interview Questions: Full DE Prep Guide
Prepare for Tesla data engineering interviews with this comprehensive guide, covering key concepts and practice questions to help you succeed
Dev.to · Gowtham Potureddi
Exodus Point Data Engineering Interview Questions: Full DE Prep Guide
Prepare for Exodus Point data engineering interviews with this comprehensive guide, covering key concepts and practice questions to help you succeed
Dev.to · Gowtham Potureddi
What I learned scraping Website Contact: schema, gotchas and the tooling that worked
Learn how to scrape Website Contact schema and overcome common obstacles with the right tooling
Dev.to · Can Yılmaz
Up next
May 2026 Databricks Updates: No Code ETL, New GPUs and Death of the Dashboard
Databricks
Watch →