Can Pandas Keep Up? Testing Pandas GPU & Polars for Data Analysis & Processing | cuDF Pandas Python

Eduardo Vasquez · Beginner ·📄 Research Papers Explained ·2y ago

About this lesson

In this video, I tested three powerful data processing tools: Pandas, cuDF (Pandas with GPU acceleration), and Polars. I aim to determine which comes out on top regarding speed and memory efficiency. The video covers: 📚 - Reading Data: Which library can load data the fastest? - Filtering Data: How do they compare in filtering operations? - GroupBy Operations: Which one handles Groupby operations more efficiently? - Join Operations: I test the performance of join operations across all three 🔔 Subscribe for More! 🔥 Don't forget to 𝘀𝘂𝗯𝘀𝗰𝗿𝗶𝗯𝗲, 𝐬𝐦𝐚𝐬𝐡 the 𝗹𝗶𝗸𝗲 𝐛𝐮𝐭𝐭𝐨𝐧, and 𝐭𝐮𝐫𝐧 𝐨𝐧 the 𝐧𝐨𝐭𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧 𝐛𝐞𝐥𝐥 for more 𝗲𝘅𝗰𝗶𝘁𝗶𝗻𝗴 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀 and 𝘁𝘂𝘁𝗼𝗿𝗶𝗮𝗹𝘀. 🚀 Timestamps: 0:00 Introduction 0:12 Setup environment 01:31 Pandas 04:40 Pandas with GPU (cuDF Pandas) 07:11 Polars 08:59 Comparisons 09:41Conclusion Links: 💻 Code: https://colab.research.google.com/drive/1uyCXM86LGPB4DmrhOJcKP49G6dm1p5l7?usp=sharing ☕️ Buy me a coffee... or an iced tea: https://www.buymeacoffee.com/eduardov 👔 LinkedIn: https://www.linkedin.com/in/eduardo-vasquez-n/

Original Description

In this video, I tested three powerful data processing tools: Pandas, cuDF (Pandas with GPU acceleration), and Polars. I aim to determine which comes out on top regarding speed and memory efficiency. The video covers: 📚 - Reading Data: Which library can load data the fastest? - Filtering Data: How do they compare in filtering operations? - GroupBy Operations: Which one handles Groupby operations more efficiently? - Join Operations: I test the performance of join operations across all three 🔔 Subscribe for More! 🔥 Don't forget to 𝘀𝘂𝗯𝘀𝗰𝗿𝗶𝗯𝗲, 𝐬𝐦𝐚𝐬𝐡 the 𝗹𝗶𝗸𝗲 𝐛𝐮𝐭𝐭𝐨𝐧, and 𝐭𝐮𝐫𝐧 𝐨𝐧 the 𝐧𝐨𝐭𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧 𝐛𝐞𝐥𝐥 for more 𝗲𝘅𝗰𝗶𝘁𝗶𝗻𝗴 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀 and 𝘁𝘂𝘁𝗼𝗿𝗶𝗮𝗹𝘀. 🚀 Timestamps: 0:00 Introduction 0:12 Setup environment 01:31 Pandas 04:40 Pandas with GPU (cuDF Pandas) 07:11 Polars 08:59 Comparisons 09:41Conclusion Links: 💻 Code: https://colab.research.google.com/drive/1uyCXM86LGPB4DmrhOJcKP49G6dm1p5l7?usp=sharing ☕️ Buy me a coffee... or an iced tea: https://www.buymeacoffee.com/eduardov 👔 LinkedIn: https://www.linkedin.com/in/eduardo-vasquez-n/

Watch on YouTube ↗ (saves to browser)

Sign in to unlock AI tutor explanation · ⚡30

Related Reads

A lightweight workflow for keeping up with AI conference papers

Learn a lightweight workflow to stay updated with AI conference papers and never miss important research again

Dev.to · Daniel

Why CitedEvidence Believes Great Researchers Read Less Than You Think

Great researchers don't read every paper, but rather focus on reading the right ones and applying their knowledge effectively

How to Write a Literature Review That Actually Argues Something

Learn to write a literature review that presents a clear argument, a crucial skill for ML researchers and students

Medium · Machine Learning

I Built a Personal Paper Engine to Stop Losing Research Papers

Build a personal paper engine to organize and annotate research papers efficiently

Dev.to · Ethan

Chapters (6)

Introduction

0:12 Setup environment

1:31 Pandas

4:40 Pandas with GPU (cuDF Pandas)

7:11 Polars

8:59 Comparisons

Welcome to the Next Temperamental Era