Course details

Building Data Pipelines

Building Data Pipelines


Overview/Description
Expected Duration
Lesson Objectives
Course Number
Expertise Level



Overview/Description

Explore data pipelines and methods of processing them with and without ETL. Creating data pipelines using Apache Airflow is also covered.



Expected Duration (hours)
1.2

Lesson Objectives

Building Data Pipelines

  • describe data pipelines and automation
  • build a traditional ETL pipeline with batch processing
  • build a ETL pipeline with stream processing
  • setup and install Apache Airflow
  • describe the key concepts of Apache Airflow
  • create and instantiate a directed acyclic graph in Airflow
  • use tasks and include arguments in Airflow
  • use dependencies in Airflow
  • build an ETL pipeline with Airflow
  • build an automated pipeline without using ETL
  • test Airflow tasks using the airflow command line utility
  • use Apache Airflow to create a data pipeline
  • Course Number:
    it_dsbdpidj_01_enus

    Expertise Level
    Intermediate