Apache Spark Series -I

How I built the OSS alternatives directory: GitHub ETL, Turso, and the UPSERT trap I hit

Learn how to build a data pipeline for an open-source alternatives directory using GitHub ETL, Turso, and Claude Haiku summaries

Dev.to · MORINAGA

Apache Iceberg in Production: Compaction, Catalogs, and the Pitfalls Nobody Warns You About

Learn how to use Apache Iceberg in production, including compaction, catalogs, and common pitfalls to avoid, to improve data engineering workflows

Dev.to · Gabriel Henrique

Your First Task as a Data Engineer in a New Company? Make the ETL Pipeline Testable

As a new data engineer, make the ETL pipeline testable to ensure data quality and reliability

Towards Data Science

From DataStage and Informatica to Databricks Medallion Architecture: Why Migration Is More Than Code Conversion

Learn how to migrate legacy ETL systems like DataStage to modern architectures like Databricks Medallion, and why it's more than just code conversion

Dev.to · Amit Kumar Singh