HamaWhite 原创，转载请注明出处。欢迎大家增加Giraph
技术交流群： 228591158

1. 近日用Giraph跑大数据的SSSP时，遇到例如以下错误：

org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters - Counters=120 Limit=120 at org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:315) at org.apache.hadoop.mapred.Counters.findCounter(Counters.java:449)
at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:559) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:506) at org.apache.hadoop.mapreduce.TaskInputOutputContext.getCounter(TaskInputOutputContext.java:88) at org.apache.giraph.counters.HadoopCountersBase.getCounter(HadoopCountersBase.java:60)
at org.apache.giraph.counters.GiraphTimers.getSuperstepMs(GiraphTimers.java:117) at org.apache.giraph.master.MasterThread.run(MasterThread.java:127) FATAL graph.GraphMapper (GraphMapper.java:uncaughtException(113)) - uncaughtException: OverrideExceptionHandler
on thread org.apache.giraph.master.MasterThread, msg = org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters - Counters=120 Limit=120, exiting... java.lang.IllegalStateException: org.apache.hadoop.mapred.Counters$CountersExceededException:
Error: Exceeded limits on number of counters - Counters=120 Limit=120 at org.apache.giraph.master.MasterThread.run(MasterThread.java:181) Caused by: org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters -
Counters=120 Limit=120 at org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:315) at org.apache.hadoop.mapred.Counters.findCounter(Counters.java:449) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:559) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:506)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.getCounter(TaskInputOutputContext.java:88) at org.apache.giraph.counters.HadoopCountersBase.getCounter(HadoopCountersBase.java:60) at org.apache.giraph.counters.GiraphTimers.getSuperstepMs(GiraphTimers.java:117)
at org.apache.giraph.master.MasterThread.run(MasterThread.java:127) INFO zk.ZooKeeperManager (ZooKeeperManager.java:run(655)) - run: Shutdown hook started. WARN zk.ZooKeeperManager (ZooKeeperManager.java:run(658)) - onlineZooKeeperServers: Forced a shutdown
hook kill of the ZooKeeper process. INFO zookeeper.ClientCnxn (ClientCnxn.java:run(1159)) - Unable to read additional data from server sessionid 0x145750f19970000, likely server has closed socket, closing socket connection and attempting reconnect INFO zk.ZooKeeperManager
(ZooKeeperManager.java:run(668)) - onlineZooKeeperServers: ZooKeeper process exited with 143 (note that 143 typically means killed).

2. 错误原因分析：该Job的Counters数量限制超过默认的120 。mapred-default.xml配置文件的默认例如以下：

<property>

          <name>mapreduce.job.counters.limit</name>

          <value>120</value>

          <description>Limit on the number of counters allowed per job. </description>

     </property>

3. 改动配置文件mapred-site.xml 。加入例如以下内容：

<property>

          <name>mapreduce.job.counters.limit</name>

          <value>1200</value>

          <description>Limit on the number of counters allowed per job. </description>

     </property>

4. 停止Hadoop服务 。

5. 把改动后的配置文件发送到集群中的每一个节点上。

6. 启动Hadoop服务。

完成！

Giraph执行报错，Error: Exceeded limits on number of counters - Counters=120 Limit=120, exiting...的更多相关文章

  1. 【mysql】 load local data infield 报错 ERROR 1148 (42000): The used command is not allowed with this MySQL version

    mysql> load data local infile '/Users/flint/learn/mysql/pet' into table bx_pet; 执行报错 ERROR 1148 ( ...

  2. php源码安装执行configure报错error: off_t undefined; check your library configuration

    php安装执行configure报错error: off_t undefined; check your library configuration vim /etc/ld.so.conf 添加如下几 ...

  3. zepplin0.7.2报错ERROR, exception: null, result: %text java.lang.NullPointerException的处理

    zepplin0.7.2报错ERROR, exception: null, result: %text java.lang.NullPointerException的处理 问题描述: 使用zeppli ...

  4. open数据库报错ERROR at line 1: ORA-03113: end-of-file on communication channel Process ID: 3880 Session ID: 125 Serial number: 3

    1.今天打开数据时,失败,报错 ERROR at line 1:ORA-03113: end-of-file on communication channelProcess ID: 3880Sessi ...

  5. linux使用wkhtmltopdf报错error while loading shared libraries:

    官网提示 linux需要这些动态库.depends on: zlib, fontconfig, freetype, X11 libs (libX11, libXext, libXrender) 在li ...

  6. 发布报错:Error ITMS-90635 - Invalid Mach-O in bundle - submitting to App store

    发布报错:Error ITMS-90635 - Invalid Mach-O in bundle - submitting to App store 昨晚上传项目到AppStore,报了这个错,纳尼! ...

  7. mysql报错"ERROR 1206 (HY000): The total number of locks exceeds the lock table size"的解决方法

    1. 问题背景         InnoDB是新版MySQL(v5.5及以后)默认的存储引擎,之前版本的默认引擎为MyISAM,因此,低于5.5版本的mysql配置文件.my.cnf中,关于InnoD ...

  8. 运行yum报错Error: Cannot retrieve metalink for reposit

    http://www.netpc.com.cn/593.html 运行yum报错Error: Cannot retrieve metalink for reposit 今天给Centos通过rpm - ...

  9. 转：运行yum报错Error: Cannot retrieve metalink for reposit

    http://www.netpc.com.cn/593.html 运行yum报错Error: Cannot retrieve metalink for repository: epel. Please ...

随机推荐

  1. python学习之getpass模块

    getpass模块提供两个功能: getpass.getpass(prompt ='Password:',​​stream = None)    提示用户输入密码而不回显.    使用字符串提示提示用 ...

  2. 李洪强iOS开发之Xcode快捷键

    14个Xcode中常用的快捷键操作   在Xcode 6中有许多快捷键的设定可以使得你的编程工作更为高效,对于在代码文件中快速导航.定位Bug以及新增应用特性都是极有效的. 当然,你戳进这篇文章的目的 ...

  3. vim添加一键编译

    引用来自: http://blog.chinaunix.net/uid-21202106-id-2406761.html; 事先声明,我使用的VIM完全是基于终端的,而不是gvim或vim-x11.因 ...

  4. root-me web server 10-20 writeup

    File upload - double extensions文件上传--双扩展 Gallery v0.02 介绍 Your goal is to hack this photo galery by ...

  5. C++11 中function和bind以及lambda 表达式的用法

    关于std::function 的用法:  其实就可以理解成函数指针 1. 保存自由函数 void printA(int a) { cout<<a<<endl; } std:: ...

  6. STM32F10x_硬件I2C读写EEPROM(标准外设库版本)

    Ⅰ.写在前面 上一篇文章是“STM32F10x_模拟I2C读写EEPROM”,讲述使用IO口模拟I2C总线通信,对EEPROM(AT24Xxx)进行读写操作的过程. 上一篇文章主要内容:I2C协议.模 ...

  7. fiddler获取手机请求

    最近由于项目的关系,一直在测试APP,但是比较令人头痛的问题就是,APP的测试没有办法像web的测试那样,可以直接的通过开发工具看到请求和响应,对于一些东西只能去看日志(比较麻烦),偶然间发现了fid ...

  8. 6：7 题一起MySQL数据库分库备份

    企业Shell面试题6:MySQL数据库分表备份 请实现对MySQL数据库进行分表备份,用脚本实现. 解答: [root@db01 scripts]# cat fenbiao.sh #!/bin/ba ...

  9. MYSQL数据库连接

    <%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding= ...

  10. express搭建权限管理系统

    express搭建权限管理系统 权限管理,是管理系统中的常见组件.通常需要定义资源,把资源调配给用户,通过判断用户是否有权限增删改查来实现. 初衷: 使用express开发过的项目大大小小加在一起也有 ...