Course Details

Previous Page


Spark Monitoring and Tuning


Overview/Description
Target Audience
Prerequisites
Expected Duration
Lesson Objectives
Course Number
Expertise Level



Overview/Description
In this course, you will learn about various ways to monitor Spark applications such as web UIs, metrics, and other monitoring tools. You will also learn about memory tuning.

Target Audience
Programmers and Developers wishing to perform big data development using Apache Spark 2.2

Prerequisites
None

Expected Duration (hours)
1.0

Lesson Objectives

Spark Monitoring and Tuning

  • start the course
  • access the web user interface
  • use the Spark environment configuration parameters
  • use JSON to query monitoring tools for Spark
  • set JVM fractional memory amounts for Spark
  • modify speculation controls for Spark tasks
  • describe data serialization and the role it plays in the performance of Spark applications
  • describe memory management and consumption
  • determine executor memory allocation
  • describe garbage collection tuning
  • set the level of parallelism
  • use the broadcast functionality
  • use query execution plan explainer
  • implement data compression on parquet storage
  • monitor Spark applications
  • Course Number:
    df_apsk_a04_it_enus

    Expertise Level
    Intermediate