Course Details

Previous Page


Programming and Deploying Apache Spark Applications


Overview/Description
Target Audience
Prerequisites
Expected Duration
Lesson Objectives
Course Number



Overview/Description
Apache Spark is a cluster computing framework for fast processing of Hadoop data. Spark applications can be written in Scala, Java, or Python. In this course, you will learn how to develop Spark applications using Scala, Java, or Python. You will also learn how to test and deploy applications to a cluster, monitor clusters and applications, and schedule resources for clusters and individual applications.

Target Audience
Developers familiar with Scala, Python, or Java who want to learn how to program and deploy Spark applications

Prerequisites
None

Expected Duration (hours)
3.0

Lesson Objectives

Programming and Deploying Apache Spark Applications

  • start the course
  • describe Apache Spark and the main components of a Spark application
  • download and install Apache Spark on Windows 8.1 Pro N
  • download and install Apache Spark on Mac OS X Yosemite
  • download and install Java Development Kit or JDK 8 and build Apache Spark using Simple Build Tool or SBT on Mac OS X Yosemite
  • use the Spark shell for analyzing data interactively
  • link an application to Spark
  • create a SparkContext to initialize Apache Spark
  • introduce Resilient Distributed Datasets or RDDs and create a parallelized collection to generate an RDD
  • load external datasets to create Resilient Distributed Datasets or RDDs
  • distinguish transformations and actions, describe some of the transformations supported by Spark, and use transformations
  • describe some of the actions supported by Spark and use the actions
  • use anonymous function syntax and use static methods in a global singleton to pass functions to Spark
  • work with key-value pairs
  • persist Spark RDDs
  • use broadcast variables in a Spark operation
  • use accumulators in Spark operations
  • use different formats for loading and saving Spark data
  • use basic Spark SQL for data queries in a Spark application
  • use basic Spark GraphX to work with graphs in a Spark application
  • describe how Spark applications run in a cluster
  • deploy a Spark application to a cluster
  • unit test a Spark application
  • describe how to monitor a Spark application or cluster with Web UIs
  • describe options for scheduling resources across applications in a Spark cluster
  • describe how to enable a fair scheduler for fair sharing within an application in a Spark cluster
  • configure fair scheduler pool properties for a Spark context within a cluster
  • practice programming and deploying a Spark application to a cluster
  • Course Number:
    df_apsf_a01_it_enus