转 http://blog.csdn.net/u012303571/article/details/46913471

 
查看 nodemanager 日志发下 如下信息
 
2015-07-16 15:28:58,643 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/data/yarn/nmlocal error, used space above threshold of 90.0%, removing from list of valid directories
2015-07-16 15:28:58,645 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/logs/yarn/nmlogs error, used space above threshold of 90.0%, removing from list of valid directories
2015-07-16 15:28:58,645 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) failed: 1/1 local-dirs are bad: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs are bad: /opt/beh/logs/yarn/nmlogs
2015-07-16 15:28:58,645 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 1/1 local-dirs are bad: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs are bad: /opt/beh/logs/yarn/nmlogs
2015-07-16 15:28:58,648 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService
2015-07-16 15:28:58,648 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: per directory file limit = 8192
2015-07-16 15:28:58,658 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker
2015-07-16 15:28:58,711 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
 
解决方法 
 删除部分没用的数据  让磁盘已用容量下降到90%一下
 
 
2015-07-16 15:29:06,808 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Notifying ContainerManager to unblock new container-requests
2015-07-16 15:38:58,817 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/data/yarn/nmlocal passed disk check, adding to list of valid directories.
2015-07-16 15:38:58,821 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /opt/beh/logs/yarn/nmlogs passed disk check, adding to list of valid directories.
2015-07-16 15:38:58,821 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) turned good: 1/1 local-dirs are good: /opt/beh/data/yarn/nmlocal; 1/1 log-dirs are good: /opt/beh/logs/yarn/nmlogs

Hadoop YARN ERROR 1/1 local-dirs are bad *， 1/1 log-dirs are bad *的更多相关文章

  1. 传统应用迁移到kubernetes（Hadoop YARN）

    spark-on-yarn-with-kubernetes 该例子仅用来说明具体的步骤划分和复杂性,在生产环境应用还有待验证,请谨慎使用. 过程中可能用到的概念和术语初步整理如下: 整个迁移过程分为如 ...

  2. YARN分析系列之二 -- Hadoop YARN各个自模块说明

    先做如下声明,本代码版本是基于 3.1.2 版本. 其实,我们自己在写代码的时候,会有意识地将比较大的功能项独立成包,独立成module, 独立成项目,项目之间的关系既容易阅读理解,又便于管理. 如下 ...

  3. Hadoop YARN：调度性能优化实践(转)

    https://tech.meituan.com/2019/08/01/hadoop-yarn-scheduling-performance-optimization-practice.html 文章 ...

  4. File file:/data1/hadoop/yarn/local/usercache/hp/appcache/application_* does not exi

    AM Container for appattempt_1453292851883_0381_000002 exited with exitCode: -1000For more detailed o ...

  5. Hadoop YARN: 1/1 local-dirs are bad: /var/lib/hadoop-yarn/cache/yarn/nm-local-dir; 1/1 log-dirs are bad: /var/log/hadoop-yarn/containers hdfs硬盘90% yarn unhealthy

    1/1 local-dirs are bad: /var/lib/hadoop-yarn/cache/yarn/nm-local-dir; 1/1 log-dirs are bad: /var/log ...

  6. Hadoop Yarn配置项 yarn.nodemanager.resource.local-dirs探讨

    1. What is the recommended value for "yarn.nodemanager.resource.local-dirs"? We only have ...

  7. hadoop+yarn+hbase+storm+kafka+spark+zookeeper)高可用集群详细配置

    配置 hadoop+yarn+hbase+storm+kafka+spark+zookeeper 高可用集群,同时安装相关组建:JDK,MySQL,Hive,Flume 文章目录 环境介绍 节点介绍 ...

  8. Hadoop YARN配置参数剖析—RM与NM相关参数

    注意,配置这些参数前,应充分理解这几个参数的含义,以防止误配给集群带来的隐患.另外,这些参数均需要在yarn-site.xml中配置. 1.    ResourceManager相关配置参数 (1) ...

  9. hadoop错误org.apache.hadoop.yarn.exceptions.YarnException Unauthorized request to start container

    错误: 14/04/29 02:45:07 INFO mapreduce.Job: Job job_1398704073313_0021 failed with state FAILED due to ...

随机推荐

  1. winform上传文件

    //上传图片 文件 public int addUpPic( String strProCode,String strFileName,String strUpType) { //strFileNam ...

  2. POIXV Permutation

    Description Multiset is a mathematical object similar to a set, but each member of a multiset may ha ...

  3. 分布式缓存-Memcached

    分布式缓存出于如下考虑,首先是缓存本身的水平线性扩展问题,其次是缓存大 并发下的本身的性能问题,再次避免缓存的单点故障问题(多副本和副本一致性).分布式缓存的核心技术包括首先是内存本身的管理问题,包括 ...

  4. 《Mastering Opencv ...读书笔记系列》车牌识别（I）

    http://blog.csdn.net/jinshengtao/article/details/17883075/  <Mastering Opencv ...读书笔记系列>车牌识别(I ...

  5. 四、WCF的配置文件

    注:本文为学习摘抄,原文地址:http://www.cnblogs.com/iamlilinfeng/archive/2012/10/02/2710224.html 一.概述 配置也是WCF编程中的主 ...

  6. C#正则分组实例

    static void Main(string[] args) { string str = "大家家家家家家家明天天天天天天天天玩得得得得得得得开心"; Regex reg = ...

  7. LeetCode OJ 1. Two Sum

    Given an array of integers, return indices of the two numbers such that they add up to a specific ta ...

  8. angularjs表单

    <form ng-app="myApp" ng-controller="validateCtrl" name="myForm" nov ...

  9. MSXML insertBefore(IXMLDOMNode *newChild, VARIANT refChild) 传参

    在xml操作中经常会用到在某一个节点后或前面插入一个节点,MSXML DOM 中使用的函数是insertBefore(IXMLDOMNode *newChild, VARIANT refChild): ...

  10. Openjudge-计算概论（A）-数组顺序逆放

    描述: 将一个数组中的值按逆序重新存放.例如,原来的顺序为8,6,5,4,1.要求改为1,4,5,6,8.输入输入为两行:第一行数组中元素的个数n(1<n<100),第二行是n个整数,每两 ...