hadoop---wordcount命令
[zznu@master file]$ hadoop jar ~/hadoop-2.5.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.2.jar wordcount /inputfile output
16/04/11 22:31:02 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.222.122:18040
16/04/11 22:31:03 INFO input.FileInputFormat: Total input paths to process : 2
16/04/11 22:31:03 INFO mapreduce.JobSubmitter: number of splits:2
16/04/11 22:31:03 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1460438506725_0002
16/04/11 22:31:03 INFO impl.YarnClientImpl: Submitted application application_1460438506725_0002
16/04/11 22:31:04 INFO mapreduce.Job: The url to track the job: http://master:18088/proxy/application_1460438506725_0002/
16/04/11 22:31:04 INFO mapreduce.Job: Running job: job_1460438506725_0002
16/04/11 22:31:13 INFO mapreduce.Job: Job job_1460438506725_0002 running in uber mode : false
16/04/11 22:31:13 INFO mapreduce.Job: map 0% reduce 0%
16/04/11 22:31:26 INFO mapreduce.Job: map 100% reduce 0%
16/04/11 22:31:34 INFO mapreduce.Job: map 100% reduce 100%
16/04/11 22:31:34 INFO mapreduce.Job: Job job_1460438506725_0002 completed successfully
16/04/11 22:31:34 INFO mapreduce.Job: Counters: 49
File System Counters
FILE: Number of bytes read=67
FILE: Number of bytes written=290851
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=237
HDFS: Number of bytes written=25
HDFS: Number of read operations=9
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=2
Launched reduce tasks=1
Data-local map tasks=2
Total time spent by all maps in occupied slots (ms)=21525
Total time spent by all reduces in occupied slots (ms)=6185
Total time spent by all map tasks (ms)=21525
Total time spent by all reduce tasks (ms)=6185
Total vcore-seconds taken by all map tasks=21525
Total vcore-seconds taken by all reduce tasks=6185
Total megabyte-seconds taken by all map tasks=22041600
Total megabyte-seconds taken by all reduce tasks=6333440
Map-Reduce Framework
Map input records=2
Map output records=6
Map output bytes=61
Map output materialized bytes=73
Input split bytes=200
Combine input records=6
Combine output records=5
Reduce input groups=3
Reduce shuffle bytes=73
Reduce input records=5
Reduce output records=3
Spilled Records=10
Shuffled Maps =2
Failed Shuffles=0
Merged Map outputs=2
GC time elapsed (ms)=267
CPU time spent (ms)=2860
Physical memory (bytes) snapshot=515694592
Virtual memory (bytes) snapshot=2516971520
Total committed heap usage (bytes)=257171456
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=37
File Output Format Counters
Bytes Written=25
[zznu@master file]$ hadoop fs -ls output
Found 2 items
-rw-r--r-- 1 zznu supergroup 0 2016-04-11 22:31 output/_SUCCESS
-rw-r--r-- 1 zznu supergroup 25 2016-04-11 22:31 output/part-r-00000
[zznu@master file]$
hadoop---wordcount命令的更多相关文章
- Hadoop Shell命令大全
hadoop支持命令行操作HDFS文件系统,并且支持shell-like命令与HDFS文件系统交互,对于大多数程序猿/媛来说,shell-like命令行操作都是比较熟悉的,其实这也是Hadoop的极大 ...
- hadoop CLASSNAME命令使用注意点
Hadoop中可是使用hadoop CLASSNAME命令.这个CLASSNAME就是你写好的类名.hadoop CLASSNAME命令类似于java classname. 使用hadoop CLAS ...
- 【Hadoop篇】--Hadoop常用命令总结
一.前述 分享一篇hadoop的常用命令的总结,将常用的Hadoop命令总结如下. 二.具体 1.启动hadoop所有进程start-all.sh等价于start-dfs.sh + start-yar ...
- 在执行hadoop fs命令时,出现WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable错误
错误呈现: 解决过程: (参考链接:https://www.cnblogs.com/kevinq/p/5103653.html) 1.输出hadoop的详细日志,并执行hadoop fs命令来查看错误 ...
- hadoop 管理命令dfsadmin
hadoop 管理命令dfsadmin dfsadmin 命令用于管理HDFS集群,这些命令常用于管理员. 1. (Safemode)安全模式 动作 命令 把集群切换到安全模式 bin/hdfs df ...
- Hadoop常用命令总结
一.前述 分享一篇hadoop的常用命令的总结,将常用的Hadoop命令总结如下. 二.具体 1.启动hadoop所有进程start-all.sh等价于start-dfs.sh + start-yar ...
- hadoop 常见 命令
一 hadoop namenode 命令 1 格式化namanode 磁盘 hadoop namenode -format 二 hadoop fs 命令 和 linux 命令 非常类似 ...
- 大数据之路week06--day07(Hadoop常用命令)
一.前述 分享一篇hadoop的常用命令的总结,将常用的Hadoop命令总结如下. 二.具体 1.启动hadoop所有进程start-all.sh等价于start-dfs.sh + start-yar ...
- hadoop目录命令
下面是经常使用到的,以此记录备忘 1.查看hadoop目录 命令: hadoop fs -ls / 2.创建目录 命令:hadoop fs -mkdir /目录名 3.将文件上传hadoop中(也就是 ...
- Hadoop常用命令及基本概念
HADOOP 是什么? 分布式计算开源框架,其核心组件为:HDFS.MAPREDUCE.YARN Hadoop各个功能模块的理解 1. HDFS模块 HDFS负责大数据的存储,通过将大文件分块后进行分 ...
随机推荐
- 64位Java开发平台的选择,如何区分JDK,Tomcat,eclipse的32位与64版本
当你想下载Linux.JDK.Tomcat.eclipse时,你是下载32位版本还是64位版本?64位版本有两种,应该选哪一个? 当你看到这些内容:x86.x64.x86-32.x86-64.ia64 ...
- Kafka单机版安装(CentOS 7环境下)
一.环境操作系统和软件版本介绍 1.环境操作系统为CentOS Linux release 7.2.1511 (Core) 可用cat /etc/redhat-release查询 2.软件版本 Kaf ...
- 【IE6的疯狂之十】父级使用padding后子元素绝对定位的BUG
在前端开发中,经常会用到css的position:absolute来使层浮动,前通过left,top,right等属性来对层进行定位,但ie6对left,top,right等属性的解释和ie7,ie8 ...
- 51nod1092(lcs简单运用/dp)
题目链接:https://www.51nod.com/onlineJudge/questionCode.html#!problemId=1092 题意:中文题诶- 思路: 解法1:最坏的情况就是在原字 ...
- Ddos 攻击
title: Ddos 防御相关 tags: Ddos, 安全, 防御 grammar_cjkRuby: true --- 防御基础 1.1. 攻击流量到底多大 谈到DDoS防御,首先就是要知道到底遭 ...
- PWM
#include "sys.h" #include "beep.h" //tim5 定时器 /* tim4 定时器 定时10分钟 控制LED1 */ void ...
- 一个快速、高效的Levenshtein算法实现——代码实现
在网上看到一篇博客讲解Levenshtein的计算,大部分内容都挺好的,只是在一些细节上不够好,看了很长时间才明白.我对其中的算法描述做了一个简单的修改.原文的链接是:一个快速.高效的Levensht ...
- java变量和数据类型总结
- Request.UrlReferrer为空的问题
Request.UrlReferrer为空的问题 今天在开发时遇到了一个问题,在用Request.UrlReferrer获取上一页面的地址时发现该对象为空(IE下,FF下可以得到对象),于是上网搜 ...
- Selenium+Python之163邮件发送
今晚写了一个163邮箱登录的脚本,由于不停的访问163登录主页导致直接访问163邮箱主页登录需要输入验证码,因为无法获取到验证码,就这导致直接访问主页登录脚本不可行,为了绕过验证码,现在先访问hao1 ...