Spark Monitoring & Tuning

Apache Spark    |    Intermediate
  • 14 videos | 49m 23s
  • Includes Assessment
  • Earns a Badge
Rating 5.0 of 1 users Rating 5.0 of 1 users (1)
Explore various ways to monitor Spark applications such as web UIs, metrics, and other monitoring tools, and examine memory tuning.

WHAT YOU WILL LEARN

  • Access the web user interface
    Use the spark environment configuration parameters
    Use json to query monitoring tools for spark
    Set jvm fractional memory amounts for spark
    Modify speculation controls for spark tasks
    Describe data serialization and the role it plays in the performance of spark applications
    Describe memory management and consumption
  • Determine executor memory allocation
    Describe garbage collection tuning
    Set the level of parallelism
    Use the broadcast functionality
    Use query execution plan explainer
    Implement data compression on parquet storage
    Monitor spark applications

IN THIS COURSE

  • 5m 24s
    During this video, you will learn how to access the web interface. FREE ACCESS
  • 3m 19s
    Find out how to use the Spark environment configuration parameters. FREE ACCESS
  • Locked
    3.  REST API
    4m 14s
    In this video, find out how to use JSON to query monitoring tools for Spark. FREE ACCESS
  • Locked
    4.  Memory Allocation
    2m 14s
    During this video, you will learn how to set JVM fractional memory amounts for Spark. FREE ACCESS
  • Locked
    5.  Speculation
    2m 34s
    Learn how to modify speculation controls for Spark tasks. FREE ACCESS
  • Locked
    6.  Serialization
    2m 30s
    After completing this video, you will be able to describe data serialization and the role it plays in the performance of Spark applications. FREE ACCESS
  • Locked
    7.  Memory Tuning
    2m 33s
    After completing this video, you will be able to describe memory management and consumption. FREE ACCESS
  • Locked
    8.  Executor Memory
    3m 3s
    In this video, learn how to determine memory allocation for an executor. FREE ACCESS
  • Locked
    9.  Garbage Collection Tuning
    2m 31s
    Upon completion of this video, you will be able to describe how to tune garbage collection. FREE ACCESS
  • Locked
    10.  Parallelism
    2m 5s
    Find out how to set the level of parallelism. FREE ACCESS
  • Locked
    11.  Broadcast Functionality
    4m
    In this video, you will use the broadcasting functionality. FREE ACCESS
  • Locked
    12.  Explain Query Execution
    6m 39s
    Find out how to use the query execution plan explainer. FREE ACCESS
  • Locked
    13.  Data Compression
    4m 29s
    In this video, learn how to implement data compression on parquet storage. FREE ACCESS
  • Locked
    14.  Exercise: Monitor Spark Applications
    3m 50s
    Learn how to monitor Spark applications. FREE ACCESS

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Channel Apache JMeter
Rating 4.0 of 1 users Rating 4.0 of 1 users (1)
Book Practical Machine Learning with Spark: Uncover Apache Spark's Scalable Performance with High-Quality Algorithms Across NLP, Computer Vision and ML
Channel Wintellect Application Tuning

PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Course Overview & Syntax Basics
Rating 4.0 of 54 users Rating 4.0 of 54 users (54)
Course Serverless Applications: Debugging & Monitoring AWS Serverless Implementations
Rating 4.6 of 41 users Rating 4.6 of 41 users (41)
Course Apache Spark SQL
Rating 4.7 of 40 users Rating 4.7 of 40 users (40)