Window Aggregations | Stream Processing

Code with Irtiza · Beginner ·🔄 Data Engineering ·3y ago

About this lesson

One of the most common use cases of stream processors is aggregating different values based on time windows. Aggregations range from sum and average to max or min. Windows range from fixed interval windows to sliding windows and even more complex session-based ones. 🥹 If you found this helpful, follow me online here: ✍️ Blog https://irtizahafiz.medium.com 👨‍💻 Website https://irtizahafiz.com/ 📲 Instagram https://www.instagram.com/irtiza.hafiz/ 0:00 What is Windowing? 03:00 States & Memory 04:00 What Time to use for windowing? 06:00 Late Data & Windowing 09:20 Which time to use for your windowing? #streaming #flink #beam #kafka #programming

Original Description

One of the most common use cases of stream processors is aggregating different values based on time windows. Aggregations range from sum and average to max or min. Windows range from fixed interval windows to sliding windows and even more complex session-based ones. 🥹 If you found this helpful, follow me online here: ✍️ Blog https://irtizahafiz.medium.com 👨‍💻 Website https://irtizahafiz.com/ 📲 Instagram https://www.instagram.com/irtiza.hafiz/ 0:00 What is Windowing? 03:00 States & Memory 04:00 What Time to use for windowing? 06:00 Late Data & Windowing 09:20 Which time to use for your windowing? #streaming #flink #beam #kafka #programming
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Related AI Lessons

⚡
How I built the OSS alternatives directory: GitHub ETL, Turso, and the UPSERT trap I hit
Learn how to build a data pipeline for an open-source alternatives directory using GitHub ETL, Turso, and Claude Haiku summaries
Dev.to · MORINAGA
 ⚡
Apache Iceberg in Production: Compaction, Catalogs, and the Pitfalls Nobody Warns You About
Learn how to use Apache Iceberg in production, including compaction, catalogs, and common pitfalls to avoid, to improve data engineering workflows
Dev.to · Gabriel Henrique
 ⚡
Your First Task as a Data Engineer in a New Company? Make the ETL Pipeline Testable
As a new data engineer, make the ETL pipeline testable to ensure data quality and reliability
Towards Data Science
 ⚡
From DataStage and Informatica to Databricks Medallion Architecture: Why Migration Is More Than Code Conversion
Learn how to migrate legacy ETL systems like DataStage to modern architectures like Databricks Medallion, and why it's more than just code conversion
Dev.to · Amit Kumar Singh

Chapters (5)

What is Windowing?
3:00 States & Memory
4:00 What Time to use for windowing?
6:00 Late Data & Windowing
9:20 Which time to use for your windowing?
Up next
A Moment Frozen in Time | Arnav Iyengar | TEDxJenks Youth
TEDx Talks
Watch →