How to solve java.net.SocketTimeoutException:60000millis problem in HDFS
Many HDFS users encounter the following error when DFSClient ready file from a certain Data Node.
"Java.net.SocketTimeoutException: 60000millis timeout while waiting for channel to be ready for read. ch"
The reason about this error is that the Data Node fails to communicate with DFSClient when DFSClient requests some block from that Data Node, which often occurs when the small cluster conntains too many files and the Data Node with such heavy burden cannot schedules a certain block within the limited time (for instance 60 sec). Then DFSClient waits untill this interval fires, than turn to request the other Data Node containing the same block.
If HDFS is used for in-time service, people wil find the 60 sec timeout is too long and they hope to shorten the interval making the DFSClient switch to other Data Node quickly. When I encounter is problem, I seach the Internet for solution. Depressed, No answeris correct.
Then I debug the source code, finding the timeout value is controlled by the config item "ipc.ping.interval". This item has the default value 60000 (millisec). You can add this config item into you"core-site.xml" file and it will take effert when DFSClient starts.
How to solve java.net.SocketTimeoutException:60000millis problem in HDFS的更多相关文章
- jsoup-提示java.net.SocketTimeoutException:Read timed out
使用Jsoup.connect(url).get()连接某网站时偶尔会出现 java.net.SocketTimeoutException:Read timed out异常. 原因是默认的Socket ...
- java.net.SocketTimeoutException: Read timed out
If you get java.net.SocketTimeoutException: Read timed out exception Try setting own timeout value w ...
- idea 连接redis 出现 Caused by: java.net.SocketTimeoutException: connect timed out
Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException: java.n ...
- 应用程序连接hbase报错:java.net.SocketTimeoutException: callTimeout=60000
背景说明: 今天对生产环境hbase增加了节点,下午的时候一个同事反馈,应用程序后台报错,如下: Tue Feb 26 17:35:35 CST 2019, null, java.net.Socket ...
- Exception in thread "main" java.net.SocketTimeoutException: connect timed ou错误处理
今天做了一个Jsoup解析网站的项目,使用Jsoup.connect(url).get()连接某网站时偶尔会出现java.net.SocketTimeoutException:Read timed o ...
- Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: connect timed out
问题: java连接不上redis. 异常信息: Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.ne ...
- phoenix连接hbase数据库,创建二级索引报错:Error: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions: Tue Mar 06 10:32:02 CST 2018, null, java.net.SocketTimeoutException: callTimeou
v\:* {behavior:url(#default#VML);} o\:* {behavior:url(#default#VML);} w\:* {behavior:url(#default#VM ...
- java.net.SocketTimeoutException: Read timed out 错误解决
这两天项目在测试环境下通过URLConnection 做数据传递时,出现了如下错误 java.net.SocketTimeoutException: Read timed out 经过查找研究,原因是 ...
- 报表使用hive数据源报java.net.SocketTimeoutException: Read timed out
数据库表的数据量大概50W左右,在报表设计器下创建了hive的数据源,连接正常,由于数据量比较大,就用了润乾报表的大数据报表功能,报表设置好后,发布到页面中报错: 数据集ds1中,SQL语句SELEC ...
随机推荐
- python之路 - 基础4
1.函数非固定参数 *args *kwargs def test(*args): print (args) test(1,2,3,4,5) test(*[1,2,3,4,5]) #*args = *[ ...
- [HMLY]11.iOS函数式编程的实现&&响应式编程概念
简介 本篇主要回顾一下--iOS函数式编程 && 响应式编程概念 ,如何一步步实现函数式编程的过程,对阅读Masonry && SnapKit源码有一定的帮助. 作为一 ...
- 第11章:DOM扩展
- Ajax beforeSend和complete 方法与防止重复提交
$.ajax({ beforeSend: function(){ // Handle the beforeSend event }, complete: function(){ // Handle t ...
- ES6 带来了什么
http://www.kuqin.com/shuoit/20150928/348317.html 这篇文章介绍了javascript 的大致历史 http://www.css88.com/archi ...
- C++引用(&)详解
C++引用详解 引用的概念 引用:就是某一变量(目标)的一个别名,对引用的操作与对变量直接操作完全一样. 引用的声明方法:类型标识符 &引用名=目标变量名: 如下:定义引用ra,它是变量a的引 ...
- git linux 多工程部署及git默认端口更改
>> ssh-keygen -t rsa -C "your_email@youremail.com" print Generating public/private ...
- 《C++反汇编与逆向分析技术揭秘》——流程控制语句的识别
if...else...语句 示例: if构成多分支语句 switch 有序线性的switch: 3E82D8位置存放了一个表,标明了要跳转到的地址: 这里的每四字节都标明的是每个case块的首地址: ...
- Python中fileinput模块使用
fileinput模块可以对一个或多个文件中的内容进行迭代.遍历等操作.该模块的input()函数有点类似文件 readlines()方法,区别在于前者是一个迭代对象,需要用for循环迭代,后者是一次 ...
- 图片上传插件用法,JS语法【三】
注意点: 作为文件域(<input type="file">)必须要有name属性,如果没有name属性,上传之后服务器是获取不到图片的.如:正确的写法是<inp ...