Previous Page

Accessing Data with Spark: Data Analysis Using the Spark DataFrame API

Accessing Data with Spark: Data Analysis Using the Spark DataFrame API


Overview/Description
Expected Duration
Lesson Objectives
Course Number
Expertise Level



Overview/Description

Apache Spark is an open-source cluster-computing framework used for data science and it has become the defacto big data framework. In this Skillsoft Aspire course, you will explore how to analyze real datasets using the DataFrame API methods. Discover how to optimize operations using shared variables and combine data from multiple DataFrames using joins.



Expected Duration (hours)
1.2

Lesson Objectives

Accessing Data with Spark: Data Analysis Using the Spark DataFrame API

  • Course Overview
  • recognize the features that make Spark 2.x versions significantly faster than Spark 1.x
  • specify the reasons for using shared variables in your Spark application and distinguish between the two options available for sharing variables
  • create a Spark DataFrame from the contents of a CSV file and apply some simple transformations on the DataFrame
  • define a transformation to view a random sample of data from a large DataFrame
  • apply grouping and aggregation operations on a DataFrame to analyze categories of data in a dataset
  • use Matplotlib to visualize the contents of a Spark DataFrame
  • perform operations to prepare your dataset for analysis by trimming unnecessary columns and rows containing missing data
  • define and apply a generic transformation on a DataFrame
  • apply complex transformations on a DataFrame to extract meaningful information from a dataset
  • work with broadcast variables and perform a join operation with a DataFrame that has been broadcast
  • use a Spark accumulator as a counter
  • store the contents of a DataFrame in a text file for archiving or sharing
  • define and work with a custom accumulator to count a vector of values
  • perform different join operations on Spark DataFrames to combine data from multiple sources
  • analyze data using the DataFrame API
  • Course Number:
    it_dsadskdj_02_enus

    Expertise Level
    Beginner