Data Engineering on Microsoft Azure: Data Flow Transformations

Azure    |    Intermediate
  • 11 videos | 1h 23m 25s
  • Includes Assessment
  • Earns a Badge
Rating 4.8 of 30 users Rating 4.8 of 30 users (30)
One of the key components of the Azure Cloud platform is the ability to store and process large amounts of data. Azure Data Flow Transformations can be used to ingest and transform data. In this course, you'll learn about the types of Azure Data Flow transformations that are available. You'll explore how to transform, split, and flatten data, as well as handle duplicate data, using Azure Data Mapping Data Flows. Next, you'll examine the types of expression functions available in Azure Data Flow and how to perform error handling for data rows that would truncate data. Finally, you'll learn how to transform and use derived columns to normalize data values, and how to ingest and transform data using Azure Spark and Scala. This course is one in a collection that prepares learners for the Microsoft Data Engineering on Microsoft Azure (DP-203) exam.

WHAT YOU WILL LEARN

  • Discover the key concepts covered in this course
    Describe the type of available azure data flow transformations
    Transform data using azure data mapping data flows
    Transform and split data using azure data flow
    Transform and flatten data using azure data flow
    Describe the types of expression functions available in azure data flow
  • Configure azure data flow to perform error handling for data rows that would truncate data
    Transform and use derived columns to normalize data values
    Ingest and transform data using azure spark and scala
    Handle duplicate data using azure data flows
    Summarize the key concepts covered in this course

IN THIS COURSE

  • 1m 46s
    In this video, you’ll learn more about your instructor and this course. In this course, you’ll explore the types of Azure Data Flow transformations available. You’ll learn how to transform data, split, handle duplicate data, and flatten data using Azure Data Mapping Data Flows. Next, you'll explore the types of expression functions available in Azure Data Flow, and learn how to perform error handling for data rows that would truncate data. FREE ACCESS
  • 7m 40s
    In this video, you’ll learn more about the transformation blocks available in Azure Data Flow. The first ones you’ll learn about are schema modifiers or transformations to the layout of the data. The aggregate transformation allows you to group data and perform aggregate functions. The aggregate can be stored as a new column or it can be stored in an existing column. FREE ACCESS
  • Locked
    3.  Using Azure Data Mapping Data Flows
    9m 6s
    In this video, you’ll watch a demo. In this demo, you’ll see how to clean incoming data by removing unwanted data using a filter in a data factory data flow pipeline. You’ll use Azure and go to your Resource group. Then you’ll use Blob Storage. You’ll head to the Blob Storage called storage2426 and open it. From there, you’ll go to Containers, and see there are two Containers set up. FREE ACCESS
  • Locked
    4.  Performing an Azure Data Flow Conditional Split
    10m 50s
    In this video, you’ll watch a demo. In this demo, you’ll learn how to use a conditional split transformation in a Data Factory Data Flow. This transformation allows you to split the flow of logic based on a condition. In Azure, you’ll go to your Resource group. You’ll be using a SQL database, which is already set up. You’ll also be using a data factory, which is also already set up. FREE ACCESS
  • Locked
    5.  Using Azure Data Flow Conditional Flatten
    8m 17s
    In this video, you’ll watch a demo. In this demo, you’ll learn how to flatten JSON data using Data Factory. You’ll open Azure and go to your Resource group. The only thing you need for this is a storage account and a data factory. You’ll use the storage account called tempstorage2426. Now, you’ll head to the Containers. Choose the Container set up called demo-data. FREE ACCESS
  • Locked
    6.  Azure Data Flow Expression Functions
    5m 26s
    In this video, you’ll learn about Azure Data Flow. Azure Data Flow allows you to build a series of steps that transform your data. The Expression Builder is a tool within Azure Data Flow that can be used with many of those transformation steps to define logic. Here, you’ll look at some of the functions available in Azure Data Flow's Expression Builder. The Expression Builder has many math functions. FREE ACCESS
  • Locked
    7.  Configuring Azure Data Flow Error Handling
    10m 3s
    In this video, you’ll watch a demo. You’ll learn how to handle a table schema error issue in a Data Factory Data Flow. You’ll start in Azure, and you’ll read a CSV file from a storage account called tempstorage2426, and this will be read by Data factory. You’ll output those records to a SQL database table in the sqldatabase2426 database. You’ll start off by looking at the Storage account and click Containers. FREE ACCESS
  • Locked
    8.  Transforming Azure Data Flow Derived Columns
    10m 16s
    In this video, you’ll watch a demo. You’ll learn how to use derived columns in a Data Factory Data Flow to normalize columns. You’ll use Azure and the Storage account called tempstoarage2426 and a Data factory instance called datafactory2426. You’ll open up your Storage account and go to Containers. You’ll see a Container called demo-data. You’ll open it and see there’s a CSV file. You’ll open that and hit Edit. FREE ACCESS
  • Locked
    9.  Transforming Data Using Azure Spark and Scala
    9m 56s
    In this video, you’ll watch a demo. You’ll learn how to instantiate an HDInsight instance in Azure and run a Scala transformation on a CSV file. You’ll head to Azure and you’ll see a cluster is already set up, the HDInsight cluster. It's called cluster2426, and it’s a Storage account. You’ll learn HDInsight clusters can take a long time to load up. They typically take 20 to 30 minutes. Here, you’ll just learn the steps. FREE ACCESS
  • Locked
    10.  Handling Duplicate Data in Azure Data Flows
    8m 46s
    In this video, you’ll watch a demo. You’ll learn how to filter duplicates in your data by using Data Factory. In Azure, you’ll use two things. You’ll use a Data factory instance that you have in your Resource group datafactory2426. You’ll be using a Storage account with blobs called tempstorage2426. You’ll go to tempstorage2426, click "Containers", and see there’s a Container already created called demo-data. FREE ACCESS
  • Locked
    11.  Course Summary
    1m 19s
    In this video, you’ll summarize what you’ve learned in the course. You’ve examined how to describe the type of available Azure Data Flow Transformations, transform data, split data, flattened data, the types of expression functions, configure error handling, normalizing data values, ingesting data using Azure Spark and Scala, and handling duplicate data. You explored available Azure data flow transformations, transforming data with Azure data mapping data flows, and Azure data flow. FREE ACCESS

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Rating 4.7 of 49 users Rating 4.7 of 49 users (49)
Rating 4.6 of 34 users Rating 4.6 of 34 users (34)
Rating 4.8 of 42 users Rating 4.8 of 42 users (42)

PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Rating 4.7 of 49 users Rating 4.7 of 49 users (49)
Rating 4.5 of 432 users Rating 4.5 of 432 users (432)
Rating 4.6 of 43 users Rating 4.6 of 43 users (43)