Hadoop 2.6.3运行自带WordCount程序笔记

运行平台：Hadoop 2.6.3

模式：完全分布模式

1、准备统计文本，以一段文字为例：eg.txt

The Project Gutenberg EBook of War and Peace, by Leo Tolstoy

This eBook is for the use of anyone anywhere at no cost and with almost
no restrictions whatsoever.  You may copy it, give it away or re-use it
under the terms of the Project Gutenberg License included with this
eBook or online at www.gutenberg.org

Title: War and Peace

Author: Leo Tolstoy

2、在Shell中上传文本

hadoop fs -put ./eg.txt /

3、进入share/hadoop/mapreduce目录下，启动排序

hadoop jar hadoop-mapreduce-examples-2.6..jar wordcount /eg.txt /out

4、屏幕输出结果如下：

16/03/29 21:30:26 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
16/03/29 21:30:30 INFO input.FileInputFormat: Total input paths to process : 1
16/03/29 21:30:30 INFO mapreduce.JobSubmitter: number of splits:1
16/03/29 21:30:31 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1459233715960_0004
16/03/29 21:30:31 INFO impl.YarnClientImpl: Submitted application application_1459233715960_0004
16/03/29 21:30:31 INFO mapreduce.Job: The url to track the job: http://m1.fredlab.org:8088/proxy/application_1459233715960_0004/
16/03/29 21:30:31 INFO mapreduce.Job: Running job: job_1459233715960_0004
16/03/29 21:30:47 INFO mapreduce.Job: Job job_1459233715960_0004 running in uber mode : false
16/03/29 21:30:47 INFO mapreduce.Job:  map 0% reduce 0%
16/03/29 21:30:57 INFO mapreduce.Job:  map 100% reduce 0%
16/03/29 21:31:09 INFO mapreduce.Job:  map 100% reduce 100%
16/03/29 21:31:10 INFO mapreduce.Job: Job job_1459233715960_0004 completed successfully
16/03/29 21:31:11 INFO mapreduce.Job: Counters: 49
        File System Counters
                FILE: Number of bytes read=547
                FILE: Number of bytes written=213761
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=453
                HDFS: Number of bytes written=361
                HDFS: Number of read operations=6
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=2
        Job Counters
                Launched map tasks=1
                Launched reduce tasks=1
                Data-local map tasks=1
                Total time spent by all maps in occupied slots (ms)=7594
                Total time spent by all reduces in occupied slots (ms)=9087
                Total time spent by all map tasks (ms)=7594
                Total time spent by all reduce tasks (ms)=9087
                Total vcore-milliseconds taken by all map tasks=7594
                Total vcore-milliseconds taken by all reduce tasks=9087
                Total megabyte-milliseconds taken by all map tasks=7776256
                Total megabyte-milliseconds taken by all reduce tasks=9305088
        Map-Reduce Framework
                Map input records=11
                Map output records=62
                Map output bytes=598
                Map output materialized bytes=547
                Input split bytes=98
                Combine input records=62
                Combine output records=45
                Reduce input groups=45
                Reduce shuffle bytes=547
                Reduce input records=45
                Reduce output records=45
                Spilled Records=90
                Shuffled Maps =1
                Failed Shuffles=0
                Merged Map outputs=1
                GC time elapsed (ms)=310
                CPU time spent (ms)=2010
                Physical memory (bytes) snapshot=273182720
                Virtual memory (bytes) snapshot=4122341376
                Total committed heap usage (bytes)=137498624
        Shuffle Errors
                BAD_ID=0
                CONNECTION=0
                IO_ERROR=0
                WRONG_LENGTH=0
                WRONG_MAP=0
                WRONG_REDUCE=0
        File Input Format Counters
                Bytes Read=355
        File Output Format Counters
                Bytes Written=361

5、结果文件位于hadoop集群/out目录下，如果执行成功，则出现_SUCCESS标识文件，并将结果存放于part-r-00000文件中。

Author:	1
EBook	1
Gutenberg	2
Leo	2
License	1
Peace	1
Peace,	1
Project	2
The	1
This	1
Title:	1
Tolstoy	2
War	2
You	1
almost	1
and	3
anyone	1
anywhere	1
at	2
away	1
by	1
copy	1
cost	1
eBook	2
for	1
give	1
included	1
is	1
it	2
it,	1
may	1
no	2
of	3
online	1
or	2
re-use	1
restrictions	1
terms	1
the	3
this	1
under	1
use	1
whatsoever.	1
with	2
www.gutenberg.org	1

可以到http://www.gutenberg.org/上下载更多txt版书籍文本来练习。