HamaWhite 原创，转载请注明出处。欢迎大家增加Giraph
技术交流群： 228591158

1. 近日用Giraph跑大数据的SSSP时，遇到例如以下错误：

org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters - Counters=120 Limit=120 at org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:315) at org.apache.hadoop.mapred.Counters.findCounter(Counters.java:449)
at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:559) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:506) at org.apache.hadoop.mapreduce.TaskInputOutputContext.getCounter(TaskInputOutputContext.java:88) at org.apache.giraph.counters.HadoopCountersBase.getCounter(HadoopCountersBase.java:60)
at org.apache.giraph.counters.GiraphTimers.getSuperstepMs(GiraphTimers.java:117) at org.apache.giraph.master.MasterThread.run(MasterThread.java:127) FATAL graph.GraphMapper (GraphMapper.java:uncaughtException(113)) - uncaughtException: OverrideExceptionHandler
on thread org.apache.giraph.master.MasterThread, msg = org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters - Counters=120 Limit=120, exiting... java.lang.IllegalStateException: org.apache.hadoop.mapred.Counters$CountersExceededException:
Error: Exceeded limits on number of counters - Counters=120 Limit=120 at org.apache.giraph.master.MasterThread.run(MasterThread.java:181) Caused by: org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters -
Counters=120 Limit=120 at org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:315) at org.apache.hadoop.mapred.Counters.findCounter(Counters.java:449) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:559) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:506)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.getCounter(TaskInputOutputContext.java:88) at org.apache.giraph.counters.HadoopCountersBase.getCounter(HadoopCountersBase.java:60) at org.apache.giraph.counters.GiraphTimers.getSuperstepMs(GiraphTimers.java:117)
at org.apache.giraph.master.MasterThread.run(MasterThread.java:127) INFO zk.ZooKeeperManager (ZooKeeperManager.java:run(655)) - run: Shutdown hook started. WARN zk.ZooKeeperManager (ZooKeeperManager.java:run(658)) - onlineZooKeeperServers: Forced a shutdown
hook kill of the ZooKeeper process. INFO zookeeper.ClientCnxn (ClientCnxn.java:run(1159)) - Unable to read additional data from server sessionid 0x145750f19970000, likely server has closed socket, closing socket connection and attempting reconnect INFO zk.ZooKeeperManager
(ZooKeeperManager.java:run(668)) - onlineZooKeeperServers: ZooKeeper process exited with 143 (note that 143 typically means killed).

2. 错误原因分析：该Job的Counters数量限制超过默认的120 。mapred-default.xml配置文件的默认例如以下：

<property>

          <name>mapreduce.job.counters.limit</name>

          <value>120</value>

          <description>Limit on the number of counters allowed per job. </description>

     </property>

3. 改动配置文件mapred-site.xml 。加入例如以下内容：

<property>

          <name>mapreduce.job.counters.limit</name>

          <value>1200</value>

          <description>Limit on the number of counters allowed per job. </description>

     </property>

4. 停止Hadoop服务 。

5. 把改动后的配置文件发送到集群中的每一个节点上。

6. 启动Hadoop服务。

完成！

Giraph执行报错，Error: Exceeded limits on number of counters - Counters=120 Limit=120, exiting...的更多相关文章

  1. 【mysql】 load local data infield 报错 ERROR 1148 (42000): The used command is not allowed with this MySQL version

    mysql> load data local infile '/Users/flint/learn/mysql/pet' into table bx_pet; 执行报错 ERROR 1148 ( ...

  2. php源码安装执行configure报错error: off_t undefined; check your library configuration

    php安装执行configure报错error: off_t undefined; check your library configuration vim /etc/ld.so.conf 添加如下几 ...

  3. zepplin0.7.2报错ERROR, exception: null, result: %text java.lang.NullPointerException的处理

    zepplin0.7.2报错ERROR, exception: null, result: %text java.lang.NullPointerException的处理 问题描述: 使用zeppli ...

  4. open数据库报错ERROR at line 1: ORA-03113: end-of-file on communication channel Process ID: 3880 Session ID: 125 Serial number: 3

    1.今天打开数据时,失败,报错 ERROR at line 1:ORA-03113: end-of-file on communication channelProcess ID: 3880Sessi ...

  5. linux使用wkhtmltopdf报错error while loading shared libraries:

    官网提示 linux需要这些动态库.depends on: zlib, fontconfig, freetype, X11 libs (libX11, libXext, libXrender) 在li ...

  6. 发布报错:Error ITMS-90635 - Invalid Mach-O in bundle - submitting to App store

    发布报错:Error ITMS-90635 - Invalid Mach-O in bundle - submitting to App store 昨晚上传项目到AppStore,报了这个错,纳尼! ...

  7. mysql报错"ERROR 1206 (HY000): The total number of locks exceeds the lock table size"的解决方法

    1. 问题背景         InnoDB是新版MySQL(v5.5及以后)默认的存储引擎,之前版本的默认引擎为MyISAM,因此,低于5.5版本的mysql配置文件.my.cnf中,关于InnoD ...

  8. 运行yum报错Error: Cannot retrieve metalink for reposit

    http://www.netpc.com.cn/593.html 运行yum报错Error: Cannot retrieve metalink for reposit 今天给Centos通过rpm - ...

  9. 转：运行yum报错Error: Cannot retrieve metalink for reposit

    http://www.netpc.com.cn/593.html 运行yum报错Error: Cannot retrieve metalink for repository: epel. Please ...

随机推荐

  1. poj 2846 Repository

    题目大意:给你n个字符串,然后给你m个子串,看这个子串在上面的多少个串中,出现过: 题目链接:http://acm.hdu.edu.cn/showproblem.php?pid=2846 本题可以在字 ...

  2. 蓝牙(CoreBluetooth)-概述

    蓝牙(CoreBluetooth)-概述 通过此框架可以让你的Mac和iOS应用程序与外部蓝牙设备通信 外部设备: 就是需要通过iOS App控制器的其他设备: 例如:心率检测仪.数字温控器 蓝牙通讯 ...

  3. 地址url的split()方法使用；

    stringObject.split(separator,howmany) 参数 描述 separator 必需.字符串或正则表达式,从该参数指定的地方分割 stringObject. howmany ...

  4. url参数

    两个参数情况: String url="http://59.78.93.208:9097/Order?id="+id+"&value="+value; ...

  5. modbus学习

  6. CRF++使用说明

    摘选自 :CRF++: Yet Another CRF toolkit Usage: 1.输入输出文件 He PRP B-NP reckons VBZ B-VP the DT B-NP current ...

  7. js for in

    JavaScript中for..in循环陷阱 大家都知道在JavaScript中提供了两种方式迭代对象:   (1)for 循环:   (2)for..in循环: 使用for循环进行迭代数组对象,想必 ...

  8. Android一键锁屏源码

    APK下载 源程序下载 锁屏流程如下(参考于Android一键锁屏开发全过程[源码][附图]) 源码参考于一键锁屏 源码 一共有2个Java文件: package com.example.onekey ...

  9. 【转】Windows Dump文件获取

    dump文件是进程的内存镜像.可以把程序的执行状态,即当时程序内存空间数据通过调试器保存到dump文件中. 1.利用WinDbg里的adplus来获取dump文件 Adplus.vbs 是一个Visu ...

  10. awk向脚本传递參数（二）

    命令行參数的一个重要限制是它们在BEGIN过程中是不可用的. 也就是说,直到首行输入完毕以后它们才可用.为什么?这是一个easy混乱的部分.从命令行传递的參数就好像文件名称一样被处理.赋值操作知道这个 ...