Processing Data: Introducing Apache Spark
Apache Spark 3.2
| Intermediate
- 13 Videos | 1h 44m 10s
- Earns a Badge
Apache Spark is a powerful distributed data processing engine that can handle petabytes of data by chunking that data and dividing across a cluster of resources. In this course, explore Spark's structured streaming engine, including components like PySpark shell. Begin by downloading and installing Apache Spark. Then create a Spark cluster and run a job from the PySpark shell. Monitor an application and job runs from the Spark web user interface. Then, set up a streaming environment, reading and manipulating the contents of files that are added to a folder in real-time. Finally, run apps on both Spark standalone and local modes.
WHAT YOU WILL LEARN
-
discover the key concepts covered in this coursedescribe how Apache Hadoop and Spark workrecall the architecture and features of Apache Sparkrecognize the use cases of Spark in general and specifically, its structured streaming engineinstall and configure Apache Sparkcreate a Spark cluster with a master and workerrun a job on the PySpark shell and view its details from the Spark web user interface (UI)
-
execute Spark commands and monitor jobs with the Spark web UIconfigure a Spark cluster using the spark-env.sh fileset up an environment to stream files, and build an app to process files in real-timeexecute apps on a Spark standalone clusterdistinguish between Spark standalone and local deployment modessummarize the key concepts covered in this course
IN THIS COURSE
-
1.Course Overview1m 23sUP NEXT
-
2.Apache Spark12m 30s
-
3.Apache Spark Architecture13m 7s
-
4.Structured Streaming in Apache Spark8m 11s
-
5.Downloading and Installing Spark6m 50s
-
6.Deploying a Spark Cluster9m 53s
-
7.Launching a Spark Job11m 9s
-
8.Monitoring Spark Apps with the Web UI7m 31s
-
9.Configuring a Spark Cluster6m 33s
-
10.Building a Spark Streaming App9m 50s
-
11.Running Apps on a Standalone Cluster8m 29s
-
12.Running Apps on Spark Local6m 14s
-
13.Course Summary2m 30s
EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE
Skillsoft is providing you the opportunity to earn a digital badge upon successful completion of this course, which can be shared on any social network or business platform
Digital badges are yours to keep, forever.