Programming with MapReduce
Apache Hadoop 2.0
| Beginner
- 16 videos | 1h 6m 43s
- Includes Assessment
- Earns a Badge
You must have a good understanding of MapReduce to be able to program with it. Here we look at MapReduce in detail, and demonstrate the basics of programming in MapReduce.
WHAT YOU WILL LEARN
-
Understand the components of a MapReduce job and the steps to create them. Explain each job component and it's function in Hadoop MapReduce. Understand the steps in creating map reduce job components.
Understand a conceptual example of the MapReduce process and how each piece fits into the overall MapReduce algorithm
Learn how to use Java to write Hadoop MapReduce jobs. Understand which additional JAR to use, and the functionality of the classes within them
Know how to create and execute Hadoop MapReduce jobs. Illustrate how to compile and run MapReduce programs.
Understand the programmatic steps in a Hadoop MapReduce job. Know how the JobClient, JobTracker, and TaskTracker work, and their interaction with the Hadoop Distributed File System (HDFS).
Understand the concept of MapReduce chaining. Describe how MapReduce jobs may have several steps with the last MapReduce output will be used as input for the next MapReduce Job.
Understanding pre-compile, compile, and run commands. Discover different techniques to package and run MapReduce Jobs.
Understand how MapReduce stores and reads Big Data. Demonstrate how MapReduce and Hadoop handle data with the HDFS over a distributed processing system. -
Understand how persistence in the HDFS compares to other file storage systems. Learn the specifics of reading and writing data in the HDFS, and it's redundancy across the cluster.
Overview of how jobs run in MapReduce. Introduction to the unit testing process. Understand tools and techniques in unit testing.
Understand how to view MapReduce job status and how to review and understand log files. Learn about how log files are handled by different distributions of Hadoop.
Understand scenarios where a MapReduce job would need to be terminated. Learn how to use the "-list" and "-kill" commands.
Overview of JUnit, and unit testing techniques using JUnit. Understand test cases using JUnit. Overview of JUnit configuration scripts.
Explain Cloudera MRUnit. Compare unit testing with MRUnit and without MRUnit. Understand the unit testing process, and unit testing files.
Understand how to use a dummy cluster for unit and integration testing. Learn the basics of a mini-HDFS and a mini-MapReduce cluster.
Understand the basics of the Hadoop LocalJobRunner. Understand that the LocalJobRunner is a mini version of the MapReduce execution engine. Explain how the LocalJobRunner can run in a debugger and can step through code in mappers and reducers.
IN THIS COURSE
-
4m 24s
-
3m 34s
-
3. Hadoop MapReduce and Java3m 38s
-
4. Executing Hadoop MapReduce Jobs7m
-
5. Job Flow Progress and Monitoring3m 57s
-
6. Chaining MapReduce Jobs3m 15s
-
7. Executing Commands7m 37s
-
8. How Data is Processed, Persisted, and Read on the HDFS3m 23s
-
9. Understanding Persistence and Types of Big Data3m 21s
-
10. Understanding High-Level Processing3m 46s
-
11. Viewing Job Status and Log Files5m 36s
-
12. Killing MapReduce Jobs3m 28s
-
13. Using JUnit3m 21s
-
14. Introduction to MRUnit3m 12s
-
15. Using a Mini MapReduce (and HDFS) Cluster3m 37s
-
16. Using the Hadoop LocalJobRunner3m 35s
EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE
Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.
Digital badges are yours to keep, forever.
