Processing Data: Introducing Apache Spark

Apache Spark 3.2    |    Intermediate
  • 13 videos | 1h 44m 10s
  • Includes Assessment
  • Earns a Badge
Apache Spark is a powerful distributed data processing engine that can handle petabytes of data by chunking that data and dividing across a cluster of resources. In this course, explore Spark's structured streaming engine, including components like PySpark shell. Begin by downloading and installing Apache Spark. Then create a Spark cluster and run a job from the PySpark shell. Monitor an application and job runs from the Spark web user interface. Then, set up a streaming environment, reading and manipulating the contents of files that are added to a folder in real-time. Finally, run apps on both Spark standalone and local modes.

WHAT YOU WILL LEARN

  • discover the key concepts covered in this course
    describe how Apache Hadoop and Spark work
    recall the architecture and features of Apache Spark
    recognize the use cases of Spark in general and specifically, its structured streaming engine
    install and configure Apache Spark
    create a Spark cluster with a master and worker
    run a job on the PySpark shell and view its details from the Spark web user interface (UI)
  • execute Spark commands and monitor jobs with the Spark web UI
    configure a Spark cluster using the spark-env.sh file
    set up an environment to stream files, and build an app to process files in real-time
    execute apps on a Spark standalone cluster
    distinguish between Spark standalone and local deployment modes
    summarize the key concepts covered in this course

IN THIS COURSE

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Book Beginning Apache Spark 2: With Resilient Distributed Datasets, Spark SQL, Structured Streaming and Spark Machine Learning Library
Course Data Analysis using Spark SQL
Likes 106 Likes 106  
Course Processing Data: Using Kafka with Cassandra & Confluent

PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Course Processing Data: Integrating Kafka with Python & Using Consumer Groups
Course Data Analysis using Spark SQL
Likes 106 Likes 106  
Course Streaming Data Architectures: Processing Streaming Data with Spark
Likes 43 Likes 43