Developing a Basic MapReduce Hadoop Application

Apache Hadoop    |    Beginner
  • 10 videos | 1h 13m 19s
  • Includes Assessment
  • Earns a Badge
Rating 3.9 of 10 users Rating 3.9 of 10 users (10)
In this Skillsoft Aspire course, discover how to use Hadoop's MapReduce; provision a Hadoop cluster on the cloud; and build an application with MapReduce to calculate word frequencies in a text document. To start, create a Hadoop cluster on the Google Cloud Platform using its Cloud Dataproc service; then work with the YARN Cluster Manager and HDFS (Hadoop Distributed File System) NameNode web applications that come packaged with Hadoop. Use Maven to create a new Java project for the MapReduce application, and develop a mapper for word frequency application. Create a Reducer for the application that will collect Mapper output and calculate word frequencies in input text files, and identify configurations of MapReduce applications in the Driver program and the project's pom.xml file. Next, build the MapReduce word frequency application with Maven to produce a jar file and prepare for execution from the master node of the Hadoop cluster. Finally, run the application and examine outputs generated to get word frequencies in the input text document. The exercise involves developing a basic MapReduce application.

WHAT YOU WILL LEARN

  • Create and configure a hadoop cluster on the google cloud platform using its cloud dataproc service
    Work with the yarn cluster manager and hdfs namenode web applications that come packaged with hadoop
    Use maven to create a new java project for the mapreduce application
    Develop a mapper for the word frequency application that includes the logic to parse one line of the input file and produce a collection of keys and values as output
    Create a reducer for the application that will collect the mapper output and calculate the word frequencies in the input text file
  • Specify the configurations of the mapreduce applications in the driver program and the project's pom.xml file
    Build the mapreduce word frequency application using maven to produce a jar file and then prepare for execution from the master node of the hadoop cluster
    Run the application and examine the outputs generated to get the word frequencies in the input text document
    Idenfity the apps packaged with hadoop and the purposes they serve and recall the classes/methods used in the map and reduce phases of a mapreduce application

IN THIS COURSE

  • 2m 50s
  • 9m 13s
    In this video, you will learn how to create and configure a Hadoop cluster on the Google Cloud Platform using the Cloud Dataproc service. FREE ACCESS
  • Locked
    3.  Browsing the Hadoop Web Applications
    7m 46s
    In this video, find out how to work with the YARN Cluster Manager and HDFS NameNode web applications that come packaged with Hadoop. FREE ACCESS
  • Locked
    4.  Creating a MapReduce project
    6m 43s
    During this video, you will learn how to use Maven to create a new Java project for a MapReduce application. FREE ACCESS
  • Locked
    5.  Coding the Map Phase
    7m 19s
    Find out how to develop a Mapper for the word frequency application that includes the logic to parse one line of the input file and produce a collection of keys and values as output. FREE ACCESS
  • Locked
    6.  Coding the Reduce Phase
    6m 24s
    In this video, learn how to create a Reducer for the application that will collect the Mapper output and calculate the word frequencies in the input text file. FREE ACCESS
  • Locked
    7.  Defining the Driver Program
    8m 33s
    After completing this video, you will be able to specify the configurations of the MapReduce applications in the Driver program and the project's pom.xml file. FREE ACCESS
  • Locked
    8.  Building the Application
    8m 23s
    In this video, you will learn how to build the MapReduce word frequency application using Maven to produce a jar file. You will then prepare for execution from the master node of the Hadoop cluster. FREE ACCESS
  • Locked
    9.  Executing the MapReduce Application
    8m 56s
    In this video, learn how to run the application and examine the outputs generated to get the word frequencies in the input text document. FREE ACCESS
  • Locked
    10.  Exercise: Developing a Basic MapReduce Application
    7m 12s
    In this video, learn how to identify the apps packaged with Hadoop and the purposes they serve and recall the classes/methods used in the Map and Reduce phases of a MapReduce application. FREE ACCESS

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Course Google Professional Cloud Developer: Cloud Applications & APIs
Rating 4.4 of 78 users Rating 4.4 of 78 users (78)
Book Big Data Using Hadoop and Hive
Course Working with Google Cloud SQL: An Introduction to Cloud SQL
Rating 4.7 of 37 users Rating 4.7 of 37 users (37)

PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Course Working with Files in Hadoop HDFS
Rating 4.5 of 20 users Rating 4.5 of 20 users (20)
Course Advanced Operations Using Hadoop MapReduce
Rating 4.7 of 7 users Rating 4.7 of 7 users (7)
Course Python - Using Pandas to Work with Series & DataFrames
Rating 4.4 of 81 users Rating 4.4 of 81 users (81)