hadoop---wordcount命令
[zznu@master file]$ hadoop jar ~/hadoop-2.5.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.2.jar wordcount /inputfile output
16/04/11 22:31:02 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.222.122:18040
16/04/11 22:31:03 INFO input.FileInputFormat: Total input paths to process : 2
16/04/11 22:31:03 INFO mapreduce.JobSubmitter: number of splits:2
16/04/11 22:31:03 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1460438506725_0002
16/04/11 22:31:03 INFO impl.YarnClientImpl: Submitted application application_1460438506725_0002
16/04/11 22:31:04 INFO mapreduce.Job: The url to track the job: http://master:18088/proxy/application_1460438506725_0002/
16/04/11 22:31:04 INFO mapreduce.Job: Running job: job_1460438506725_0002
16/04/11 22:31:13 INFO mapreduce.Job: Job job_1460438506725_0002 running in uber mode : false
16/04/11 22:31:13 INFO mapreduce.Job: map 0% reduce 0%
16/04/11 22:31:26 INFO mapreduce.Job: map 100% reduce 0%
16/04/11 22:31:34 INFO mapreduce.Job: map 100% reduce 100%
16/04/11 22:31:34 INFO mapreduce.Job: Job job_1460438506725_0002 completed successfully
16/04/11 22:31:34 INFO mapreduce.Job: Counters: 49
File System Counters
FILE: Number of bytes read=67
FILE: Number of bytes written=290851
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=237
HDFS: Number of bytes written=25
HDFS: Number of read operations=9
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=2
Launched reduce tasks=1
Data-local map tasks=2
Total time spent by all maps in occupied slots (ms)=21525
Total time spent by all reduces in occupied slots (ms)=6185
Total time spent by all map tasks (ms)=21525
Total time spent by all reduce tasks (ms)=6185
Total vcore-seconds taken by all map tasks=21525
Total vcore-seconds taken by all reduce tasks=6185
Total megabyte-seconds taken by all map tasks=22041600
Total megabyte-seconds taken by all reduce tasks=6333440
Map-Reduce Framework
Map input records=2
Map output records=6
Map output bytes=61
Map output materialized bytes=73
Input split bytes=200
Combine input records=6
Combine output records=5
Reduce input groups=3
Reduce shuffle bytes=73
Reduce input records=5
Reduce output records=3
Spilled Records=10
Shuffled Maps =2
Failed Shuffles=0
Merged Map outputs=2
GC time elapsed (ms)=267
CPU time spent (ms)=2860
Physical memory (bytes) snapshot=515694592
Virtual memory (bytes) snapshot=2516971520
Total committed heap usage (bytes)=257171456
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=37
File Output Format Counters
Bytes Written=25
[zznu@master file]$ hadoop fs -ls output
Found 2 items
-rw-r--r-- 1 zznu supergroup 0 2016-04-11 22:31 output/_SUCCESS
-rw-r--r-- 1 zznu supergroup 25 2016-04-11 22:31 output/part-r-00000
[zznu@master file]$
hadoop---wordcount命令的更多相关文章
- Hadoop Shell命令大全
hadoop支持命令行操作HDFS文件系统,并且支持shell-like命令与HDFS文件系统交互,对于大多数程序猿/媛来说,shell-like命令行操作都是比较熟悉的,其实这也是Hadoop的极大 ...
- hadoop CLASSNAME命令使用注意点
Hadoop中可是使用hadoop CLASSNAME命令.这个CLASSNAME就是你写好的类名.hadoop CLASSNAME命令类似于java classname. 使用hadoop CLAS ...
- 【Hadoop篇】--Hadoop常用命令总结
一.前述 分享一篇hadoop的常用命令的总结,将常用的Hadoop命令总结如下. 二.具体 1.启动hadoop所有进程start-all.sh等价于start-dfs.sh + start-yar ...
- 在执行hadoop fs命令时,出现WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable错误
错误呈现: 解决过程: (参考链接:https://www.cnblogs.com/kevinq/p/5103653.html) 1.输出hadoop的详细日志,并执行hadoop fs命令来查看错误 ...
- hadoop 管理命令dfsadmin
hadoop 管理命令dfsadmin dfsadmin 命令用于管理HDFS集群,这些命令常用于管理员. 1. (Safemode)安全模式 动作 命令 把集群切换到安全模式 bin/hdfs df ...
- Hadoop常用命令总结
一.前述 分享一篇hadoop的常用命令的总结,将常用的Hadoop命令总结如下. 二.具体 1.启动hadoop所有进程start-all.sh等价于start-dfs.sh + start-yar ...
- hadoop 常见 命令
一 hadoop namenode 命令 1 格式化namanode 磁盘 hadoop namenode -format 二 hadoop fs 命令 和 linux 命令 非常类似 ...
- 大数据之路week06--day07(Hadoop常用命令)
一.前述 分享一篇hadoop的常用命令的总结,将常用的Hadoop命令总结如下. 二.具体 1.启动hadoop所有进程start-all.sh等价于start-dfs.sh + start-yar ...
- hadoop目录命令
下面是经常使用到的,以此记录备忘 1.查看hadoop目录 命令: hadoop fs -ls / 2.创建目录 命令:hadoop fs -mkdir /目录名 3.将文件上传hadoop中(也就是 ...
- Hadoop常用命令及基本概念
HADOOP 是什么? 分布式计算开源框架,其核心组件为:HDFS.MAPREDUCE.YARN Hadoop各个功能模块的理解 1. HDFS模块 HDFS负责大数据的存储,通过将大文件分块后进行分 ...
随机推荐
- MyEclipse 代码提示设置
打开 Eclipse -> Window -> Perferences -> Java -> Editor -> Content Assist,在右边最下面一栏找到 au ...
- DUIlib使用Fastreport--自定义的数据
报表根据数据源的可以分为拉模式和推模式,拉模式就是在报表中添加数据源组件从数据库中拉取数据,我们上篇报表的简单使用就是拉模式.而推模式就是在程序中构造数据托给报表显示.这篇我们这要说的是推模式. 在程 ...
- spring-事务实现原理
spring事务的实现原理:aop. aop的两种实现方式: 1.动态代理: 事务方法与调用方法不能在同一个类中,否则事务不生效.解决方案,自己注入自己(实质注入的是代理类). 实现方式:jdk动态代 ...
- Hive 执行计划
执行语句 hive> explain select s.id, s.name from student s left outer join student_tmp st on s.name = ...
- appnium框架以及源码研究
android4.0后,google提供了uiautomator来进行自动化方案,appium在高版本android上就是基于这个,4.0下是基于selendroid. appium相当于一个中转站, ...
- clone远程代码及push
clone远程代码1. git bash进入 git文件夹2. 从远程直接clone: git clone root@109.110.100.56:/usr/src/git-2.1.2/data/gi ...
- vs2008编译FileZilla服务端源码
vs2008编译FileZilla服务端源码 FileZilla服务端下载地址:https://download.filezilla-project.org/server/.FileZilla服务端源 ...
- debian安装后sudo命令不能用的解决方法
注:转载他人 且试用过了,我的debian版本是debian8.2 64X debian安装完之后发现sudo命令不能用 找了半天发现是没有安装sudo 得了,进入root安包,炸开他,apt-get ...
- 变更mysql数据库文件目录 Linux
本次需要将mysql默认的数据库文件路径/var/lib/mysql 改为新挂载的目录/data/mysql上,需要做以下修改 1.停止mysql服务 service mysqld stop 2.复制 ...
- ajax 跨域携带COOKIE
这个问题属于Ajax跨域携带Cookie的问题,找了一篇博文的解决方案. 原生ajax请求方式: var xhr = new XMLHttpRequest(); xhr.open("POST ...