Advanced Data Engineering with Snowflake

This is a technical, hands-on course that teaches you how to implement DevOps best practices to build data pipelines, and how to implement observability to maintain and monitor data pipeline health. The course focuses on the most practical Snowflake concepts, features, and tools to get you up and running quickly with these concepts. You'll start by learning about DevOps, DevOps practices, and how DevOps fits into the context of data engineering. You'll incorporate source control, declarative management of database objects, continuous delivery, and use a command-line interface to implement DevOps best practices into a data pipeline. You'll specifically learn how to: - Use Snowflake's git integration to add source control to your data pipeline - Use GitHub for team-wide collaboration on your data pipeline - Use CREATE OR ALTER to declaratively manage database objects - Use GitHub Actions to implement continuous delivery for your pipeline - Use Snowflake CLI to deploy changes into dedicated data environments You'll also learn about observability, and how to implement it to maintain and monitor the health and performance of your data pipeline. You'll specifically learn how to: - Use logs to keep a record of events that occur within your pipeline - Use traces to maintain a detailed journey of events for operations in your pipeline - Use alerts to monitor for specific conditions in your pipeline, and combine them with notifications to encourage action among team members if critical errors occur in the pipeline Throughout the course, you'll follow along with the instructor using a combination of Snowflake, Visual Studio Code, GitHub, and the command line. The course is supplemented with readings containing resources to level up your understanding of specific concepts. You'll come away understanding how to incorporate DevOps best practices into data pipelines, and how to use observability to monitor the health and performance of pipelines.