Data Engineering on Microsoft Azure: Data Lake Storage

Azure    |    Intermediate
  • 11 videos | 1h 47m 18s
  • Includes Assessment
  • Earns a Badge
Rating 4.6 of 43 users Rating 4.6 of 43 users (43)
Azure Data Lake Storage Gen2 provides features to work with big data analytics using Azure Blob Storage. Azure Blob Storage systems provide performance, management, and security functionality. In this course, you'll learn about the features of the Azure Data Lake Storage Gen2 and when to use this storage type. You'll explore features and methods for securing data for the Azure Data Lake Storage Gen2 service and data at rest. You'll examine methods for processing big data using the Azure Data Lake Storage Gen2 service and monitoring Azure Blob Storage. You'll learn how to manage directories, files, and Access Control Lists in Azure Data Lake Storage Gen2 using the .NET framework, as well as how to perform extract, transform, and load operations using Azure Databricks from Azure Data Lake Storage Gen2. Finally, you'll learn how to access Azure Data Lake Storage Gen2 data using Azure Databricks and Spark. This course is one in a collection that prepares learners for the Microsoft Data Engineering on Microsoft Azure (DP-203) exam.

WHAT YOU WILL LEARN

  • Discover the key concepts covered in this course
    Describe azure data lake storage gen2 features and when to use this storage type
    Create an azure data lake storage gen2 storage account
    Manage directories, files, and access control lists in azure data lake storage gen2 using the .net framework
    Perform extract, transform, and load operations using azure databricks from azure data lake storage gen2
    Transform data using apache spark
  • Transform data by using data factory and data flows
    Integrate pipelines using synapse studio
    Stream data into azure databricks by using event hubs
    Transform data using azure databricks
    Summarize the key concepts covered in this course

IN THIS COURSE

  • 2m
    In this video, you’ll learn more about your instructor and this course. You’ll learn the features of the Azure Data Lake Storage Gen2, and learn when to use this storage type. You’ll also learn the features and methods for securing data for the Azure Data Lake Storage Gen2 service and Securing Data at REST. Then you’ll learn methods for processing Big data using the Azure Data Lake Storage Gen2 service and monitoring Blob storage. FREE ACCESS
  • 7m 52s
    In this video, you’ll learn more about Azure's Data Lake Storage Gen 2. Data Lake Gen 2 is the Azure solution for managing massively scalable data to support big data analytics. It supports many data ingestion methods. It also allows you to use any number of popular analytics tools on the stored data. It’s an ideal choice for big data analytics needs. Data Lake Storage Gen 2 is an extension of Azure Blob Storage. FREE ACCESS
  • Locked
    3.  Creating a Data Lake Storage Gen2 Storage Account
    4m 55s
    In this video, you’ll watch a demo. In this demo, you’ll learn how to create an Azure storage account with a Data Lake Storage Gen2 hierarchical namespace. A hierarchical namespace allows you to store your objects in a collection of directories and subdirectories in the same manner that directories of files work in Windows. First, you’ll go to Azure and head to resource groups. FREE ACCESS
  • Locked
    4.  Managing Azure Data Lake Storage Gen2
    10m 11s
    In this video, you’ll watch a demo. In this demo, you’ll learn how to create a container, a directory, and a subdirectory in data lake. You’ll learn how to set the access control list on a directory to manage access permissions. You'll also learn how to upload a text file as a blob. You’ll do all of this through .NET code. FREE ACCESS
  • Locked
    5.  Extracting Data from Azure Data Lake Storage Gen2
    17m 48s
    In this video, you’ll watch a demo. In this demo, you’ll learn how to perform an ETL or extract, transform, and load using an Azure Databricks notebook. You’ll see the pipeline will extract data from Azure Data Lake Storage, transform it, and then load it into Azure Synapse Analytics. You’ll need several resources for this, which you’ll see are already set up. In Azure, you’ll go to your Resource group. FREE ACCESS
  • Locked
    6.  Transforming Data Using Apache Spark
    12m 54s
    In this video, you’ll watch a demo. In this demo, you’ll learn how to transform data using Spark in a data factory pipeline. First, you’ll create the resources in Azure that you’ll need. In Azure, you’ll hit Create a resource. You’ll need a data factory, so you’ll type in data factory, click Data Factory, and then hit Create. FREE ACCESS
  • Locked
    7.  Transforming Data Using Data Factory
    11m 6s
    In this video, you’ll watch a demo. In this demo, you’ll learn to use data factory to create a data flow between two tables. These two tables are in an Azure SQL database. Onscreen, you’ll see you’re in the SQL database in Azure. You’ll head to the Query editor. You’ll create some tables, and the first table is called Inventory. FREE ACCESS
  • Locked
    8.  Transforming Azure Synapse Pipelines
    17m 56s
    In this video, you’ll watch a demo. In this demo, you’ll learn how to create a pipeline in Azure Synapse Analytics. You’ll look at some of the details, such as monitoring running pipelines and scheduling pipelines. In Azure, the first thing you’ll do is create a Synapse Analytics instance. You’ll click Create a resource. Then, you’ll type in synapse analytics and choose Azure Synapse Analytics. FREE ACCESS
  • Locked
    9.  Streaming Data to Azure Databricks
    10m 13s
    In this video, you’ll watch a demo. In this demo, you’ll learn how to create an event hub in Azure, create a Databricks instance in Azure, and use Databricks notebooks to send and receive messages via that event hub. You’ll see how Databricks can be used to ingest Event Hub messages. FREE ACCESS
  • Locked
    10.  Transforming Data with Databricks
    11m 11s
    In this video, you’ll watch a demo. In this demo, you’ll learn how to connect to an Azure Databricks resource using Data Factory, and then run a Databricks notebook using a Data Factory pipeline. You’ll start in Azure, but you’ll need a few resources. Those are already set up here. You have a data lake, which is a Storage account with a data lake option. You also have Data Factory and a Databricks instance. FREE ACCESS
  • Locked
    11.  Course Summary
    1m 11s
    In this video, you’ll summarize what you’ve learned in the course. In this course, you’ve learned the features of Azure Data Lake Storage Gen2. You also learned how to secure data for the Azure Data Lake Storage Gen2, secure data at rest processing big data, and monitor the Azure Blob Storage service. You explored using and managing Azure Data Lake Storage Gen2 and extracting and accessing data from Azure Data Lake Storage Gen2. FREE ACCESS

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Rating 4.9 of 22 users Rating 4.9 of 22 users (22)
Rating 4.5 of 137 users Rating 4.5 of 137 users (137)
Rating 4.6 of 63 users Rating 4.6 of 63 users (63)