HamaWhite 原创，转载请注明出处。欢迎大家增加Giraph
技术交流群： 228591158

1. 近日用Giraph跑大数据的SSSP时，遇到例如以下错误：

org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters - Counters=120 Limit=120 at org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:315) at org.apache.hadoop.mapred.Counters.findCounter(Counters.java:449)
at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:559) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:506) at org.apache.hadoop.mapreduce.TaskInputOutputContext.getCounter(TaskInputOutputContext.java:88) at org.apache.giraph.counters.HadoopCountersBase.getCounter(HadoopCountersBase.java:60)
at org.apache.giraph.counters.GiraphTimers.getSuperstepMs(GiraphTimers.java:117) at org.apache.giraph.master.MasterThread.run(MasterThread.java:127) FATAL graph.GraphMapper (GraphMapper.java:uncaughtException(113)) - uncaughtException: OverrideExceptionHandler
on thread org.apache.giraph.master.MasterThread, msg = org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters - Counters=120 Limit=120, exiting... java.lang.IllegalStateException: org.apache.hadoop.mapred.Counters$CountersExceededException:
Error: Exceeded limits on number of counters - Counters=120 Limit=120 at org.apache.giraph.master.MasterThread.run(MasterThread.java:181) Caused by: org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters -
Counters=120 Limit=120 at org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:315) at org.apache.hadoop.mapred.Counters.findCounter(Counters.java:449) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:559) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:506)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.getCounter(TaskInputOutputContext.java:88) at org.apache.giraph.counters.HadoopCountersBase.getCounter(HadoopCountersBase.java:60) at org.apache.giraph.counters.GiraphTimers.getSuperstepMs(GiraphTimers.java:117)
at org.apache.giraph.master.MasterThread.run(MasterThread.java:127) INFO zk.ZooKeeperManager (ZooKeeperManager.java:run(655)) - run: Shutdown hook started. WARN zk.ZooKeeperManager (ZooKeeperManager.java:run(658)) - onlineZooKeeperServers: Forced a shutdown
hook kill of the ZooKeeper process. INFO zookeeper.ClientCnxn (ClientCnxn.java:run(1159)) - Unable to read additional data from server sessionid 0x145750f19970000, likely server has closed socket, closing socket connection and attempting reconnect INFO zk.ZooKeeperManager
(ZooKeeperManager.java:run(668)) - onlineZooKeeperServers: ZooKeeper process exited with 143 (note that 143 typically means killed).

2. 错误原因分析：该Job的Counters数量限制超过默认的120 。mapred-default.xml配置文件的默认例如以下：

<property>

          <name>mapreduce.job.counters.limit</name>

          <value>120</value>

          <description>Limit on the number of counters allowed per job. </description>

     </property>

3. 改动配置文件mapred-site.xml 。加入例如以下内容：

<property>

          <name>mapreduce.job.counters.limit</name>

          <value>1200</value>

          <description>Limit on the number of counters allowed per job. </description>

     </property>

4. 停止Hadoop服务 。

5. 把改动后的配置文件发送到集群中的每一个节点上。

6. 启动Hadoop服务。

完成！

Giraph执行报错，Error: Exceeded limits on number of counters - Counters=120 Limit=120, exiting...的更多相关文章

  1. 【mysql】 load local data infield 报错 ERROR 1148 (42000): The used command is not allowed with this MySQL version

    mysql> load data local infile '/Users/flint/learn/mysql/pet' into table bx_pet; 执行报错 ERROR 1148 ( ...

  2. php源码安装执行configure报错error: off_t undefined; check your library configuration

    php安装执行configure报错error: off_t undefined; check your library configuration vim /etc/ld.so.conf 添加如下几 ...

  3. zepplin0.7.2报错ERROR, exception: null, result: %text java.lang.NullPointerException的处理

    zepplin0.7.2报错ERROR, exception: null, result: %text java.lang.NullPointerException的处理 问题描述: 使用zeppli ...

  4. open数据库报错ERROR at line 1: ORA-03113: end-of-file on communication channel Process ID: 3880 Session ID: 125 Serial number: 3

    1.今天打开数据时,失败,报错 ERROR at line 1:ORA-03113: end-of-file on communication channelProcess ID: 3880Sessi ...

  5. linux使用wkhtmltopdf报错error while loading shared libraries:

    官网提示 linux需要这些动态库.depends on: zlib, fontconfig, freetype, X11 libs (libX11, libXext, libXrender) 在li ...

  6. 发布报错:Error ITMS-90635 - Invalid Mach-O in bundle - submitting to App store

    发布报错:Error ITMS-90635 - Invalid Mach-O in bundle - submitting to App store 昨晚上传项目到AppStore,报了这个错,纳尼! ...

  7. mysql报错"ERROR 1206 (HY000): The total number of locks exceeds the lock table size"的解决方法

    1. 问题背景         InnoDB是新版MySQL(v5.5及以后)默认的存储引擎,之前版本的默认引擎为MyISAM,因此,低于5.5版本的mysql配置文件.my.cnf中,关于InnoD ...

  8. 运行yum报错Error: Cannot retrieve metalink for reposit

    http://www.netpc.com.cn/593.html 运行yum报错Error: Cannot retrieve metalink for reposit 今天给Centos通过rpm - ...

  9. 转：运行yum报错Error: Cannot retrieve metalink for reposit

    http://www.netpc.com.cn/593.html 运行yum报错Error: Cannot retrieve metalink for repository: epel. Please ...

随机推荐

  1. Mac上Nginx-增加对HLS的支持

    Mac上Nginx-增加对HLS的支持 我们在Mac上搭建直播服务器Nginx说了如何在Mac搭建视频直播服务器Nginx,对RTMP推流和RTMP拉流的支持.接下来说说如何增加对HLS的支持. 在N ...

  2. QT界面 使用QStyledItemDelegate QPainter QStyleOptionViewItem QModelIndex组合实现项的绘制

    QStyledItemDelegate类为来自模型的数据项提供了显示和编辑工具. 当在Qt项视图(例如QTableView)中显示来自模型的数据时,各个项由委托(delegate)绘制.此外,当编辑一 ...

  3. asp.net出现的异常:"由于代码已经过优化或者本机框架位于调用堆栈之上,无法计算表达式的值" 的解决方法

    引用: https://support.microsoft.com/zh-cn/kb/312629 症状 如果您使用的Response.End. Response.Redirect或Server.Tr ...

  4. winform 查看远程服务器上的文件

    解决方案: 1. 在目标服务器上发布webservice,实现文件下载的方法. using System; using System.Collections.Generic; using System ...

  5. 《Netty in action》目录修复版本分享

    最近阅读了Netty in action一书.深感外国友人的书籍编写能力强大.作者由简入深.精简描述了Netty的相关知识,如何使用等等. 本来想翻译一下的.尝试着翻译了一点之后.发现非常痛苦啊.ps ...

  6. TPM--Trusted Platform Module

    trouSerS是IBM的一帮牛人搞的TSS软件栈,提供了与TPM交互的API,从而可以让我们方便地编写应用程序. 地址:https://sourceforge.net/projects/trouse ...

  7. VC编译的时候因为加载符号导致编译过程异常缓慢的处理

    之前调试console程序用了一下Ctrl+C,弹出的异常处理选择框自己点了什么忘记了,结果之后再调试,发现特别慢,最下面一直在走着一系列dll的加载符号.让我特别二呼的是断网的话就没事了,联网就一直 ...

  8. cocos2dx-是男人就坚持20s 练手项目

    前言 前段时间心血来潮看了下app游戏方面的东西 ,对比了下各种技术和市场招聘情况,赶脚cocos2dx在2D游戏方向还算是大有所为,遂找了几个基础教程看看了解了解.并附上一个简单demo作为成果 准 ...

  9. jquery 改变checkbox的值

    似乎没什么用... <script> $(document).ready(function(){ $("#comment").change(function(){ va ...

  10. Django(模板语言-自定义filter和simple_tag)

    Django(模板语言-自定义filter和simple_tag)   filter过滤器的主要形式:变量|函数,意思是将变量交给函数处理,而自定义filter就是自己定义函数,因为用到已有的很少. ...