Data Engineering with Python and AI/LLMs – Data Loading Tutorial
Master data ingestion for data engineering with Python. Learn to tackle common pipeline failures like schema changes and API limits by adopting the mindset and practices of a senior platform engineer. This course covers essential techniques including extracting data from APIs, automatic schema management, incremental loading, and orchestrating scalable, automated workflows using modern tools.
Course developed by Alexey Grigorev & Adrian Brudaru.
💻 Code: https://github.com/dlt-hub/dlthub-education/tree/main/courses/freecodecamp/de_with_dlt_2025
🏗️ dlthub.com provided a grant to make this c…
Watch on YouTube ↗
(saves to browser)
Chapters (33)
1. Introduction
8:02
2. What is data ingestion
10:04
3. Extracting data: Data Streaming & Batching
14:00
4. Extracting data: Working with RestAPI
29:36
5. Normalizing data
43:41
6. Loading data into DuckDB
48:39
7. Dynamic schema management
56:26
8. What is next?
56:36
1. Introduction
59:29
2. Overview
1:02:08
3. Extracting data with dlt: dlt RestAPI Client
1:08:05
4. dlt Resources
1:10:42
5. How to configure secrets
1:15:12
6. Normalizing data with dlt
1:24:09
7. Data Contracts
1:31:05
8. Alerting schema changes
1:33:56
9. Loading data with dlt
1:33:56
10. Write dispositions
1:37:34
11. Incremental loading
1:43:46
12. Loading data from SQL database to SQL database
1:47:46
13. Backfilling
1:50:42
14. SCD2
1:54:29
15. Performance tuning
2:03:12
16. Loading data to Data Lakes & Lakehouses & Catalogs
2:12:17
17. Loading data to Warehouses/MPPs,Staging
2:18:15
18. Deployment & orchestration
2:18:15
19. Deployment with Git Actions
2:29:04
20. Deployment with Crontab
2:40:05
21. Deployment with Dagster
2:49:47
22. Deployment with Airflow
3:07:00
23. Create pipelines with LLMs: Understanding the challenge
3:10:35
24. Create pipelines with LLMs: Creating prompts and LLM friendly documentation
3:31:38
25. Create pipelines with LLMs: Demo
Playlist
Uploads from freeCodeCamp.org · freeCodeCamp.org · 0 of 60
← Previous
Next →
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
![Python Automation Tutorial – How to Automate Tasks for Beginners [Full Course]](https://i.ytimg.com/vi/s8XjEuplx_U/mqdefault.jpg)
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Deploy 12 apps to AWS, Azure, & Google Cloud
freeCodeCamp.org
Reinforcement Learning Course: Intro to Advanced Actor Critic Methods
freeCodeCamp.org
How does the internet work? (Full Course)
freeCodeCamp.org
Code a Dictionary with React and Material UI - Tutorial
freeCodeCamp.org
Graph Algorithms for Technical Interviews - Full Course
freeCodeCamp.org
HTML Tutorial - Website Crash Course for Beginners
freeCodeCamp.org
Google Cloud Digital Leader Certification Course - Pass the Exam!
freeCodeCamp.org
Advanced OpenGL - Crash Course
freeCodeCamp.org
FastAPI Course for Beginners
freeCodeCamp.org
Google Sheets - Full Course
freeCodeCamp.org
Python Django and Google APIs - Project Tutorial
freeCodeCamp.org
Java Native Interface (28-Hour Course)
freeCodeCamp.org
Learn How to Code - Programming for Beginners Tutorial with Python and C#
freeCodeCamp.org
Android Programming Course - Kotlin, Jetpack Compose UI, Graph Data Structures & Algorithms
freeCodeCamp.org
Code a Discord Chat Bot That Talks Like Your Favorite Character - Tutorial
freeCodeCamp.org
Machine Learning Course for Beginners
freeCodeCamp.org
Selenium Course for Beginners - Web Scraping Bots, Browser Automation, Testing (Tutorial)
freeCodeCamp.org
Figma Course - Web Design Tutorial for Beginners
freeCodeCamp.org
Vue.js Course for Beginners
freeCodeCamp.org
Data Structures and Algorithms in Python - Full Course for Beginners
freeCodeCamp.org
Solidity, Blockchain, and Smart Contract Course – Beginner to Expert Python Tutorial
freeCodeCamp.org
Linux Essentials for Ethical Hackers - Full InfoSec Course
freeCodeCamp.org
Gatsby Static Site Generator Tutorial
freeCodeCamp.org
Django For Everybody - Full Python University Course
freeCodeCamp.org
Python Automation Tutorial – How to Automate Tasks for Beginners [Full Course]
freeCodeCamp.org
R Shiny for Data Science Tutorial – Build Interactive Data-Driven Web Apps
freeCodeCamp.org
Natural Language Processing with spaCy & Python - Course for Beginners
freeCodeCamp.org
HTML & CSS Project Tutorial - Build a Recipes Website
freeCodeCamp.org
Git for Professionals Tutorial - Tools & Concepts for Mastering Version Control with Git
freeCodeCamp.org
TensorFlow for Computer Vision Course - Full Python Tutorial for Beginners
freeCodeCamp.org
HashiCorp Terraform Associate Certification Course - Pass the Exam!
freeCodeCamp.org
AWS VPC Beginner to Pro - Virtual Private Cloud Tutorial
freeCodeCamp.org
Object Oriented Programming with Python - Full Course for Beginners
freeCodeCamp.org
Node.js / Express Course - Build 4 Projects
freeCodeCamp.org
Binary Tree Algorithms for Technical Interviews - Full Course
freeCodeCamp.org
TensorFlow Lite for Edge Devices - Tutorial
freeCodeCamp.org
Angular Tutorial for Beginners - Web Framework with Typescript Course
freeCodeCamp.org
Unreal Engine & C++ Tutorial - Endless Runner Game 🏃🏽♀️
freeCodeCamp.org
Android App Development Tutorial for Beginners - Your First App
freeCodeCamp.org
Django Tutorial - Create a Digital Resume with a Python Backend
freeCodeCamp.org
Learn React by Building an eCommerce Site - Tutorial
freeCodeCamp.org
JavaScript Tutorial - PuzzleCam Game
freeCodeCamp.org
Python API Development - Comprehensive Course for Beginners
freeCodeCamp.org
The 50 Most Popular Linux & Terminal Commands - Full Course for Beginners
freeCodeCamp.org
Web Applications for Everybody Course - Dr. Chuck Teaches HTML, PHP, SQL, CSS, JavaScript, and more!
freeCodeCamp.org
Code a Squid Game JavaScript Game Using Three.js - Tutorial for Beginners
freeCodeCamp.org
Micro-Frontends Course - Beginner to Expert
freeCodeCamp.org
Unreal Engine & C++ Tutorial - 1st Person Shooter Game
freeCodeCamp.org
C++ Programming Tutorial - Build a 3-Band Compressor Audio Plugin (w/ JUCE Framework)
freeCodeCamp.org
Infrastructure as Code - Crash Course
freeCodeCamp.org
AWS Certified Cloud Practitioner Certification Course (CLF-C01) - Pass the Exam!
freeCodeCamp.org
Advanced Git Tutorial - Interactive Rebase, Cherry-Picking, Reflog, Submodules and more
freeCodeCamp.org
Learn ASP.NET Core MVC (.NET 6) - Full Course
freeCodeCamp.org
Feature Toggles - Why and How to Add to Your Software
freeCodeCamp.org
Data Analysis with Python for Excel Users - Full Course
freeCodeCamp.org
NoSQL Database Tutorial – Full Course for Beginners
freeCodeCamp.org
Rust Programming Course for Beginners - Tutorial
freeCodeCamp.org
Razor Pages for ASP.NET Core - Full Course (.NET 6)
freeCodeCamp.org
The Business of Building Apps - App Product Management Course for Developers
freeCodeCamp.org
Bootstrap CSS Framework - Full Course for Beginners
freeCodeCamp.org
DeepCamp AI