Apache Airflow: The Hands-On Guide

Course Overview

Master Apache Airflow from A to Z Hands-on videos on Airflow with AWS Kubernetes Docker and more


What youll learn:

Coding Production Grade Data pipelines by Mastering Airflow through Hands-on Examples
How to Follow Best Practices with Apache Airflow
How to Scale Airflow with the Local Celery and Kubernetes Wxecutors
How to Set Up Monitoring with Elasticsearch and Grafana
How to Secure Airflow with authentication crypto and the RBAC UI
Core and Advanced Concepts with Pros and Limitations
Mastering DAGs with timezones unit testing backfill and catchup
Organising the DAG folder and keep things clean

Apache Airflow is a platform created by community to programmatically author schedule and monitor workflows
It is
scalable

dynamic

extensible
and
modulable

Without any doubts mastering Airflow is becoming a
must-have
and an attractive skill for anyone working with data
What you will learn in the course:
Fundamentals of Airflow are explained
such as what is Airflow how the scheduler and the web server works
The Forex Data Pipeline project is incredible way to discover many operators
in Airflow and deal with Slack Spark Hadoop and more
Mastering your DAGs
is a top priority and you will be able to play with
timezones

unit testing your DAGs

how to structure your DAG folder
and much more
Scaling Airflow
through different executors such as the
Local Executor
the
Celery Executor
and the
Kubernetes Executor
will be explained in details You will discover
how to specialise your workers

how to add new workers

what happens when a node crashes

A
Kubernetes cluster of 3 nodes
will be set up with
Rancher

Airflow
and the
Kubernetes Executor
in local to run your data pipelines
Advanced concepts
will be shown through practical examples such as
templatating your DAGs

how to make your DAG dependent of another
what are
Subdags and deadlocks
and more
You will set up a
Kubernetes cluster in the cloud
with
AWS EKS and Rancher
in order to use Airflow along with the
Kubernetes Executor
Monitoring Airflow is extremely important
! Thats why you will know how to do it with
Elasticsearch
and
Grafana

Security
will be also addressed in order to make your Airflow instance compliant with your company
Specifying roles and permissions for your users with RBAC

Prevent from accessing the Airflow UI with authentication and password

data

encryption
and more
In addition:
Many practical exercises
are given along the course so that you will have occasions to apply what you learn
Best practices
are stated when needed to give you the best ways of using Airflow
Quiz
are available to assess your comprehension at the end of each section
Answering fast your questions
is my top-priority and I will do my best for you
I put a lot of effort in order to give you the best content and I hope you will enjoy it as much as I enjoyed doing it
At the end of the course you will more confident than ever to use Airflow
Wish you a great success!
Marc Lamberti