How to solve java.net.SocketTimeoutException:60000millis problem in HDFS
Many HDFS users encounter the following error when DFSClient ready file from a certain Data Node.
"Java.net.SocketTimeoutException: 60000millis timeout while waiting for channel to be ready for read. ch"
The reason about this error is that the Data Node fails to communicate with DFSClient when DFSClient requests some block from that Data Node, which often occurs when the small cluster conntains too many files and the Data Node with such heavy burden cannot schedules a certain block within the limited time (for instance 60 sec). Then DFSClient waits untill this interval fires, than turn to request the other Data Node containing the same block.
If HDFS is used for in-time service, people wil find the 60 sec timeout is too long and they hope to shorten the interval making the DFSClient switch to other Data Node quickly. When I encounter is problem, I seach the Internet for solution. Depressed, No answeris correct.
Then I debug the source code, finding the timeout value is controlled by the config item "ipc.ping.interval". This item has the default value 60000 (millisec). You can add this config item into you"core-site.xml" file and it will take effert when DFSClient starts.
How to solve java.net.SocketTimeoutException:60000millis problem in HDFS的更多相关文章
- jsoup-提示java.net.SocketTimeoutException:Read timed out
使用Jsoup.connect(url).get()连接某网站时偶尔会出现 java.net.SocketTimeoutException:Read timed out异常. 原因是默认的Socket ...
- java.net.SocketTimeoutException: Read timed out
If you get java.net.SocketTimeoutException: Read timed out exception Try setting own timeout value w ...
- idea 连接redis 出现 Caused by: java.net.SocketTimeoutException: connect timed out
Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException: java.n ...
- 应用程序连接hbase报错:java.net.SocketTimeoutException: callTimeout=60000
背景说明: 今天对生产环境hbase增加了节点,下午的时候一个同事反馈,应用程序后台报错,如下: Tue Feb 26 17:35:35 CST 2019, null, java.net.Socket ...
- Exception in thread "main" java.net.SocketTimeoutException: connect timed ou错误处理
今天做了一个Jsoup解析网站的项目,使用Jsoup.connect(url).get()连接某网站时偶尔会出现java.net.SocketTimeoutException:Read timed o ...
- Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: connect timed out
问题: java连接不上redis. 异常信息: Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.ne ...
- phoenix连接hbase数据库,创建二级索引报错:Error: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions: Tue Mar 06 10:32:02 CST 2018, null, java.net.SocketTimeoutException: callTimeou
v\:* {behavior:url(#default#VML);} o\:* {behavior:url(#default#VML);} w\:* {behavior:url(#default#VM ...
- java.net.SocketTimeoutException: Read timed out 错误解决
这两天项目在测试环境下通过URLConnection 做数据传递时,出现了如下错误 java.net.SocketTimeoutException: Read timed out 经过查找研究,原因是 ...
- 报表使用hive数据源报java.net.SocketTimeoutException: Read timed out
数据库表的数据量大概50W左右,在报表设计器下创建了hive的数据源,连接正常,由于数据量比较大,就用了润乾报表的大数据报表功能,报表设置好后,发布到页面中报错: 数据集ds1中,SQL语句SELEC ...
随机推荐
- Linux环境下安装Redis步骤即问题解决
第一步:将安装包在window平台上解压后拷贝到Linux机器的/usr/soft目录下,并且为文件夹和文件赋予最高权限,chmod+x *: 第二步:进入到redis-3.2.6目录下,执行make ...
- GridControl的单元格中以buttonEdit实现文字和图片按钮并存的效果
话不多说,先上效果图 对于第一列的效果是如何实现的就不多说了,网上有很多例子 重点是第三列的效果实现方法,代码如下 private void GridSet() { DevExpress.XtraEd ...
- hdu1041
#include <iostream> #include <string> using namespace std; const int SIZE = 1001; const ...
- ACM第二题 生理周期
人生来就有三个生理周期,分别为体力.感情和智力周期,它们的周期长度为23天.28天和33天.每一个周期中有一天是高峰.在高峰这天,人会在相应的方面表现出色.例如,智力周期的高峰,人会思维敏捷,精力容易 ...
- oracle_sequence用法
1. About Sequences(关于序列) 序列是数据库对象一种.多个用户可以通过序列生成连续的数字以此来实现主键字段的自动.唯一增长,并且一个序列可为多列.多表同时使用.序列消除了串行化 ...
- Zabbix概念、安装以及快速入门
Zabbix is an enterprise-class open source distributed monitoring solution.[1] Zabbix是一个企业级的.开源的.分布式的 ...
- Webservice WCF WebApi
注明:改编加组合 在.net平台下,有大量的技术让你创建一个HTTP服务,像Web Service,WCF,现在又出了Web API.在.net平台下,你有很多的选择来构建一个HTTP Service ...
- [读书笔记]python3.5实现socket通讯(TCP)
TCP连接: tcp是面向连接的一个协议,意味着,客户端和服务器开发发送数据之前,需要先握手创建一个TCP连接.TCP连接的一端与客户端套接字相互联系,另一端与服务器套接字相联系.当创建该TCP连接的 ...
- 邂逅python
你很美 幸运遇见你 很幸运,能在现在这么多繁杂的语言中选择python.初识python是在2016年的七月.大二学期刚刚结束,留在学校准备学点东西.刚好听同学说下学期要学习python,所以决定要学 ...
- CodeForces 710E Generate a String
$SPFA$,优化,$dp$. 写了一个裸的$SPFA$,然后加了一点优化就过了,不过要$300$多$ms$. $dp$的话跑的就比较快了. $dp[i]$表示输入$i$个字符的最小花费. 首先$dp ...