How to solve java.net.SocketTimeoutException:60000millis problem in HDFS
Many HDFS users encounter the following error when DFSClient ready file from a certain Data Node.
"Java.net.SocketTimeoutException: 60000millis timeout while waiting for channel to be ready for read. ch"
The reason about this error is that the Data Node fails to communicate with DFSClient when DFSClient requests some block from that Data Node, which often occurs when the small cluster conntains too many files and the Data Node with such heavy burden cannot schedules a certain block within the limited time (for instance 60 sec). Then DFSClient waits untill this interval fires, than turn to request the other Data Node containing the same block.
If HDFS is used for in-time service, people wil find the 60 sec timeout is too long and they hope to shorten the interval making the DFSClient switch to other Data Node quickly. When I encounter is problem, I seach the Internet for solution. Depressed, No answeris correct.
Then I debug the source code, finding the timeout value is controlled by the config item "ipc.ping.interval". This item has the default value 60000 (millisec). You can add this config item into you"core-site.xml" file and it will take effert when DFSClient starts.
How to solve java.net.SocketTimeoutException:60000millis problem in HDFS的更多相关文章
- jsoup-提示java.net.SocketTimeoutException:Read timed out
使用Jsoup.connect(url).get()连接某网站时偶尔会出现 java.net.SocketTimeoutException:Read timed out异常. 原因是默认的Socket ...
- java.net.SocketTimeoutException: Read timed out
If you get java.net.SocketTimeoutException: Read timed out exception Try setting own timeout value w ...
- idea 连接redis 出现 Caused by: java.net.SocketTimeoutException: connect timed out
Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException: java.n ...
- 应用程序连接hbase报错:java.net.SocketTimeoutException: callTimeout=60000
背景说明: 今天对生产环境hbase增加了节点,下午的时候一个同事反馈,应用程序后台报错,如下: Tue Feb 26 17:35:35 CST 2019, null, java.net.Socket ...
- Exception in thread "main" java.net.SocketTimeoutException: connect timed ou错误处理
今天做了一个Jsoup解析网站的项目,使用Jsoup.connect(url).get()连接某网站时偶尔会出现java.net.SocketTimeoutException:Read timed o ...
- Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: connect timed out
问题: java连接不上redis. 异常信息: Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.ne ...
- phoenix连接hbase数据库,创建二级索引报错:Error: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions: Tue Mar 06 10:32:02 CST 2018, null, java.net.SocketTimeoutException: callTimeou
v\:* {behavior:url(#default#VML);} o\:* {behavior:url(#default#VML);} w\:* {behavior:url(#default#VM ...
- java.net.SocketTimeoutException: Read timed out 错误解决
这两天项目在测试环境下通过URLConnection 做数据传递时,出现了如下错误 java.net.SocketTimeoutException: Read timed out 经过查找研究,原因是 ...
- 报表使用hive数据源报java.net.SocketTimeoutException: Read timed out
数据库表的数据量大概50W左右,在报表设计器下创建了hive的数据源,连接正常,由于数据量比较大,就用了润乾报表的大数据报表功能,报表设置好后,发布到页面中报错: 数据集ds1中,SQL语句SELEC ...
随机推荐
- Mocha的单元测试实战
Mocha Mocha是一个测试框架,为JS应用添加测试.使用见:mochajs. Mocha结合Nodejs实战 ontstair.js 这里我们使用自定义模块:ontstair.js,代码如下. ...
- git stash让bug来的更猛烈些吧
git stash可以用来暂存当前正在进行的工作,比如想pull最新的代码,又不想加新commit, 或者有一个紧急的bug需要修复,但是这个bug又与你已经在做的工作(还没完成)有关联.这个时候有的 ...
- 微软sqlHelper
//微软的SQLHelper类(含完整中文注释) using System; using System.Data; using System.Xml; using System.Data.SqlCli ...
- Makefile.am编写规则
概念 Makefile.am是比Makefile更高层次的规则只需要指定要生成什么目标,依赖于什么文件,和要安装到什么目录.automake会根据Makefile.am来自动生成Makefile.in ...
- cos距离与欧式距离
1. 欧氏距离(EuclideanDistance) 欧氏距离是最易于理解的一种距离计算方法,源自欧氏空间中两点间的距离公式. (1)二维平面上两点a(x1,y1)与b(x2,y2)间的欧氏距离: ( ...
- 详细介绍Java垃圾回收机制
垃圾收集GC(Garbage Collection)是Java语言的核心技术之一,之前我们曾专门探讨过Java 7新增的垃圾回收器G1的新特性,但在JVM的内部运行机制上看,Java的垃圾回收原理与机 ...
- js实现多行文本超出一定字数显示省略号功能
最近项目中遇到了一个关于超出一定字数用省略号显示的问题,其实这种形式很常见,公司简介.产品介绍啊里面都可能会用到,一行文字显示省略号很容易,多行就得想点办法了.在经过查阅.整理之后,我也算是实现了这个 ...
- Playmaker 基础使用与案例操作
首先是把下载好的插件导入Unity工程中. ▼导入完成后第一个动作就是检查下拉菜单里面是否已经增加了Playmaker的功能,如果在安装后没看到Playmaker的菜单,一般情况下直接点击菜单上的空白 ...
- 1、<img />标签
alt:当图片不显示时的文字说明 title:鼠标悬停在图片上的出现的文字说明
- 【Time系列二】自动关机脚本
今天在弄那个自动关机脚本的时候,遇到最大的麻烦就是怎么像电脑一样显示关机时间,看 了其他大神的博客,明白了原来用的是我没学过的datetime模块和time.strptime模块 ! ! ! 接下来, ...