Data Analysis using Spark SQL

Apache Spark 2.3    |    Intermediate
  • 9 videos | 54m 6s
  • Includes Assessment
  • Earns a Badge
Likes 106 Likes 106
Analyze an Apache Spark DataFrame as though it were a relational database table. During this Aspire course, you will discover the different stages involved in optimizing any query or method call on the contents of a Spark DataFrame. Discover how to create views out of a Spark DataFrame's contents and run queries against them; and how to trim and clean a DataFrame. Next, learn how to perform an analysis of data by running different SQL queries; how to configure a DataFrame with an explicitly defined schema; and define what a window is in the context of Spark. Finally, observe how to create and analyze categories of data in a data set by using Windows.

WHAT YOU WILL LEARN

  • recall the different stages involved in optimizing any query or method call on the contents of a Spark DataFrame
    create views out of a Spark DataFrame's contents and run queries against them
    trim and clean a DataFrame before a view is created as a precursor to running SQL queries on it
    perform an analysis of data by running different kinds of SQL queries, including grouping and aggregations
  • recognize how Spark DataFrames infer the schema of data loaded into them and configure a DataFrame with an explicitly defined schema
    define what a window is in the context of Spark DataFrames and when they can be used
    create and analyze categories of data in a dataset using Windows
    analyze data using Spark SQL

IN THIS COURSE

  • 2m 45s
  • 3m
    Upon completion of this video, you will be able to recall the different stages involved in optimizing any query or method call on the contents of a Spark DataFrame. FREE ACCESS
  • Locked
    3.  Introduction to Spark SQL
    7m 41s
    In this video, you will learn how to create views out of a Spark DataFrame's contents and run queries against them. FREE ACCESS
  • Locked
    4.  Preparing Data for Analysis
    7m 14s
    During this video, you will learn how to trim and clean a DataFrame before creating a view as a precursor to running SQL queries on it. FREE ACCESS
  • Locked
    5.  Running SQL Queries
    6m 44s
    In this video, you will learn how to perform an analysis of data by running different kinds of SQL queries, including grouping and aggregations. FREE ACCESS
  • Locked
    6.  Inferred and Explicit Schemas
    7m 56s
    After completing this video, you will be able to recognize how Spark DataFrames infer the schema of data loaded into them and configure a DataFrame with an explicitly defined schema. FREE ACCESS
  • Locked
    7.  Windowing in Spark
    5m 53s
    Learn how to define what a window is in the context of Spark DataFrames and when they can be used. FREE ACCESS
  • Locked
    8.  Applying Window Functions
    8m 3s
    In this video, you will create and analyze categories of data in a dataset using Microsoft Windows. FREE ACCESS
  • Locked
    9.  Exercise: Data Analysis Using Spark SQL
    4m 50s
    During this video, you will learn how to analyze data using SQL with Spark. FREE ACCESS

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Course Introducing Apache Spark for AI Development
Likes 11 Likes 11  
Course Graph Modeling on Apache Spark: Working with Apache Spark GraphFrames
Book Practical Machine Learning with Spark: Uncover Apache Spark's Scalable Performance with High-Quality Algorithms Across NLP, Computer Vision and ML

PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Course Fundamentals & Installation
Likes 167 Likes 167  
Course Apache Kafka: Concepts & Components
Likes 254 Likes 254  
Course Streaming Data Architectures: Processing Streaming Data with Spark
Likes 43 Likes 43