Programming with MapReduce

Apache Hadoop 2.0    |    Beginner
  • 16 Videos | 1h 13m 43s
  • Includes Assessment
  • Earns a Badge
Likes 35 Likes 35
You must have a good understanding of MapReduce to be able to program with it. Here we look at MapReduce in detail, and demonstrate the basics of programming in MapReduce.

WHAT YOU WILL LEARN

  • Understand the components of a MapReduce job and the steps to create them. Explain each job component and it's function in Hadoop MapReduce. Understand the steps in creating map reduce job components.
    Understand a conceptual example of the MapReduce process and how each piece fits into the overall MapReduce algorithm
    Learn how to use Java to write Hadoop MapReduce jobs. Understand which additional JAR to use, and the functionality of the classes within them
    Know how to create and execute Hadoop MapReduce jobs. Illustrate how to compile and run MapReduce programs.
    Understand the programmatic steps in a Hadoop MapReduce job. Know how the JobClient, JobTracker, and TaskTracker work, and their interaction with the Hadoop Distributed File System (HDFS).
    Understand the concept of MapReduce chaining. Describe how MapReduce jobs may have several steps with the last MapReduce output will be used as input for the next MapReduce Job.
    Understanding pre-compile, compile, and run commands. Discover different techniques to package and run MapReduce Jobs.
    Understand how MapReduce stores and reads Big Data. Demonstrate how MapReduce and Hadoop handle data with the HDFS over a distributed processing system.
  • Understand how persistence in the HDFS compares to other file storage systems. Learn the specifics of reading and writing data in the HDFS, and it's redundancy across the cluster.
    Overview of how jobs run in MapReduce. Introduction to the unit testing process. Understand tools and techniques in unit testing.
    Understand how to view MapReduce job status and how to review and understand log files. Learn about how log files are handled by different distributions of Hadoop.
    Understand scenarios where a MapReduce job would need to be terminated. Learn how to use the "-list" and "-kill" commands.
    Overview of JUnit, and unit testing techniques using JUnit. Understand test cases using JUnit. Overview of JUnit configuration scripts.
    Explain Cloudera MRUnit. Compare unit testing with MRUnit and without MRUnit. Understand the unit testing process, and unit testing files.
    Understand how to use a dummy cluster for unit and integration testing. Learn the basics of a mini-HDFS and a mini-MapReduce cluster.
    Understand the basics of the Hadoop LocalJobRunner. Understand that the LocalJobRunner is a mini version of the MapReduce execution engine. Explain how the LocalJobRunner can run in a debugger and can step through code in mappers and reducers.

IN THIS COURSE

  • Playable
    1. 
    Creating Job Components
    4m 24s
    UP NEXT
  • Playable
    2. 
    Understanding MapReduce with a Conceptual Example
    3m 34s
  • Locked
    3. 
    Hadoop MapReduce and Java
    3m 38s
  • Locked
    4. 
    Executing Hadoop MapReduce Jobs
    7m
  • Locked
    5. 
    Job Flow Progress and Monitoring
    3m 57s
  • Locked
    6. 
    Chaining MapReduce Jobs
    3m 15s
  • Locked
    7. 
    Executing Commands
    7m 37s
  • Locked
    8. 
    How Data is Processed, Persisted, and Read on the HDFS
    3m 23s
  • Locked
    9. 
    Understanding Persistence and Types of Big Data
    3m 21s
  • Locked
    10. 
    Understanding High-Level Processing
    3m 46s
  • Locked
    11. 
    Viewing Job Status and Log Files
    5m 36s
  • Locked
    12. 
    Killing MapReduce Jobs
    3m 28s
  • Locked
    13. 
    Using JUnit
    3m 21s
  • Locked
    14. 
    Introduction to MRUnit
    3m 12s
  • Locked
    15. 
    Using a Mini MapReduce (and HDFS) Cluster
    3m 37s
  • Locked
    16. 
    Using the Hadoop LocalJobRunner
    3m 35s

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion of this course, which can be shared on any social network or business platform

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Likes 41