Narwhals: Expanding DataFrame Compatibility Between Libraries | Real Python Podcast #224
How does a Python tool support all types of DataFrames and their various features? Could a lightweight library be used to add compatibility for newer formats like Polars or PyArrow? This week on the show, we speak with Marco Gorelli about his project Narwhals.
👉 Links from the show: https://realpython.com/podcasts/rpp/224/
Narwhals is a project aimed at library maintainers rather than end users. We discuss how the added compatibility benefits users by supporting modern features like lazy evaluation. We cover several projects Marco has been working with to implement Narwhals, including Altair, scikit-lego, and Ibis.
We also discuss how Marco started contributing to open-source projects. Marco has contributed to both pandas and Polars, which helps explain his interest in growing compatibility between libraries. He also offers advice on making your first contribution.
This episode is sponsored by CodeRabbit.
Topics:
- 00:00:00 -- Introduction
- 00:02:02 -- Euro SciPy 2024 and sprints
- 00:04:04 -- How did you get involved in open source?
- 00:07:18 -- Finding a good issue to get started
- 00:09:25 -- Discord and open-source projects
- 00:11:12 -- Who would you describe Narwhals?
- 00:16:47 -- Working on Polars
- 00:19:17 -- Apache Arrow and a data interchange protocol
- 00:22:55 -- Sponsor: CodeRabbit
- 00:23:55 -- Digging into eager vs lazy
- 00:27:04 -- Ibis DataFrame library
- 00:28:57 -- What do libraries need from Narwhals?
- 00:34:57 -- The scikit-lego library
- 00:37:15 -- Video Course Spotlight
- 00:38:45 -- Other libraries interested in Narwhals
- 00:41:56 -- Compatibility policy
- 00:45:18 -- What should an end user expect?
- 00:46:32 -- Have other projects that attempted this?
- 00:47:54 -- Keeping the project light and pure Python
- 00:49:32 -- Contributors and how to get involved
- 00:54:42 -- What are you excited about in the world of Python?
- 00:57:18 -- What do you want to learn next?
- 00:59:05 -- How can people follow your work online?
- 00:
Watch on YouTube ↗
(saves to browser)
Sign in to unlock AI tutor explanation · ⚡30
Playlist
Uploads from Real Python · Real Python · 0 of 60
← Previous
Next →
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
A better Python REPL – bpython vs python interpreter
Real Python
Introducing large-type.com – A Utility Website
Real Python
Reading Hacker News Without Wasting Tons of Time
Real Python
Forward References and Python 3 Type Hints
Real Python
Using Sublime Text as your Git Editor
Real Python
Python Code Linting and Auto-Complete for Sublime Text
Real Python
Make your Python Code More Readable with Custom Exceptions
Real Python
Write Better Tests with Sublime Text's Split Layout Feature
Real Python
How to Use Sublime Text from the Command Line
Real Python
Rename Variables with Multiple Selection in Sublime Text
Real Python
Sublime Text Settings for Writing PEP 8 Python
Real Python
Write Cleaner Python with Sublime Text's Indent Guides
Real Python
Sublime Text Whitespace Settings for Python Development
Real Python
Function Argument Unpacking in Python
Real Python
Python Code Review: Debugging and Refactoring "Conway's Game of Life" + Automated Tests
Real Python
Using "get()" to Return a Default Value from a Python Dict
Real Python
A Python Shorthand for Swapping Two Variables
Real Python
Python Code Review: Refactoring a Web Scraper, PEP 8 Style Guide Compliance, requirements.txt
Real Python
Click & Jump to Test Failures from the Command Line (iTerm2)
Real Python
Setting up Sublime Text for Python Developers
Real Python
Sublime Text + Python Guide Overview
Real Python
Python Code Review: Adding Pytest Tests to an Existing Python Web Scraper
Real Python
Type-Checking Python Programs With Type Hints and mypy
Real Python
A Shorthand for Merging Dictionaries in Python 3.5+
Real Python
Python Code Review Flask Web Security Tutorial + Virtualenvs, requirements.txt
Real Python
My Python Code Looks Ugly and Confusing – Help!
Real Python
Setting Up a Programmer Portfolio/Developer Blog – How To Get Started
Real Python
Do I Need a GitHub/GitLab/Bitbucket Profile as a Developer?
Real Python
Programmer Portfolio – Example and Walkthrough
Real Python
How to Get Your 1st Speaking Gig at a Tech Conference
Real Python
How to Build Your Public Speaking Skills as a Developer
Real Python
The Object-oriented Version of "Spaghetti Code" is "Lasagna Code" ?!
Real Python
Setting up Sublime Text for Python Developers – Lesson #1
Real Python
Cool New Features in Python 3.6
Real Python
"is" vs "==" in Python – What's the Difference? (And When to Use Each)
Real Python
Emulating switch/case Statements in Python with Dictionaries
Real Python
Python Function Argument Unpacking Tutorial (* and ** Operators)
Real Python
What Code Should I Put On My GitHub/GitLab/BitBucket Profile?
Real Python
A Crazy Python Dictionary Expression ?!
Real Python
String Conversion in Python: When to Use __repr__ vs __str__
Real Python
Method Types in Python OOP: @classmethod, @staticmethod, and Instance Methods
Real Python
Optional Arguments in Python With *args and **kwargs
Real Python
Python Context Managers and the "with" Statement (__enter__ & __exit__)
Real Python
Installing Python Packages with pip and virtualenv / venv
Real Python
"For Each" Loops in Python with enumerate() and range()
Real Python
Python Code Review: LibreOffice Automation and the Python Standard Library
Real Python
Managing Python Dependencies With Pip and Virtual Environments – Lesson #1
Real Python
Python Tutorial: List Comprehensions Step-By-Step
Real Python
Leveraging Python's Implicit "return None" Statements
Real Python
What's the meaning of underscores (_ & __) in Python variable names?
Real Python
Python Data Structures: Sets, Frozensets, and Multisets (Bags)
Real Python
Writing automated tests for Python command-line apps and scripts
Real Python
How to find great Python packages on PyPI, the Python Package Repository
Real Python
Immutable vs Mutable Objects in Python
Real Python
PyPI vs Warehouse, the Next-Generation Python Package Repository
Real Python
pep8.org — The Prettiest Way to View the PEP 8 Python Style Guide
Real Python
My Experience at PyCon 2017 in Portland
Real Python
Pylint Tutorial – How to Write Clean Python
Real Python
"Reverse a List in Python" Tutorial: Three Methods & How-to Demos
Real Python
Python Refactoring: "while True" Infinite Loops & The "input" Function
Real Python
More on: PM Basics
View skill →
Related AI Lessons
⚡
⚡
⚡
⚡
Why the Best Mobile App Development Company Is Not Always the Biggest One
Dev.to AI
The Problem with Most Productivity Apps (And How We Tried to Fix It)
Dev.to · Stefan
Building WeRemember in Public — Day 0: Why Another Calendar App?
Dev.to · Sandro Hu
Google Announces Its Chromebook Successor: The Googlebook
Daring Fireball
🎓
Tutor Explanation
DeepCamp AI