报错背景:

CDH集成了Flume服务,准备通过Flume将kafka中的数据放到HDFS中,

启动Flume的时候报错。

报错现象:

// :: INFO hdfs.HDFSDataStream: Serializer = TEXT, UseRawLocalFileSystem = false
// :: INFO hdfs.BucketWriter: Creating hdfs://master:8020/yk/dl/alarm_his/AlarmHis.1557281724769.txt.tmp
// :: WARN hdfs.HDFSEventSink: HDFS IO error
org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/yk/dl/alarm_his":hdfs:supergroup:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:)
at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:)
at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:)
at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:)
at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:)
at org.apache.hadoop.ipc.Server$Handler$.run(Server.java:)
at org.apache.hadoop.ipc.Server$Handler$.run(Server.java:)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:)
at java.lang.reflect.Constructor.newInstance(Constructor.java:)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:)
at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:)
at org.apache.hadoop.hdfs.DistributedFileSystem$.doCall(DistributedFileSystem.java:)
at org.apache.hadoop.hdfs.DistributedFileSystem$.doCall(DistributedFileSystem.java:)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:)
at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:)
at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:)
at org.apache.flume.sink.hdfs.HDFSDataStream.doOpen(HDFSDataStream.java:)
at org.apache.flume.sink.hdfs.HDFSDataStream.open(HDFSDataStream.java:)
at org.apache.flume.sink.hdfs.BucketWriter$.call(BucketWriter.java:)
at org.apache.flume.sink.hdfs.BucketWriter$.call(BucketWriter.java:)
at org.apache.flume.sink.hdfs.BucketWriter$$.run(BucketWriter.java:)
at org.apache.flume.auth.SimpleAuthenticator.execute(SimpleAuthenticator.java:)
at org.apache.flume.sink.hdfs.BucketWriter$.call(BucketWriter.java:)
at java.util.concurrent.FutureTask.run(FutureTask.java:)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:)
at java.lang.Thread.run(Thread.java:)

报错原因:

根据日志信息可以解读出,root用户想要在文件系统里面创建一个目录,失败。

原因是,root用户无法操作hdfs

报错解决:

重启HDFS即可

报错:HDFS IO error org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/yk/dl/alarm_his":hdfs:supergroup:drwxr-xr-x的更多相关文章

  1. kylin cube测试时,报错:org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

    异常: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, i ...

  2. 【Hadoop & Ecilpse】Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=bruce, access=WRITE, inode="/out2/_temporary/0":atguigu:supergroup:drwxr-xr-x

    问题再现: 使用本机 Ecilpse  (Windows环境) 去访问远程 hadoop 集群出现以下异常: 问题原因: 因为远程提交的情况下如果没有 hadoop 的系统环境变量,就会读取当前主机的 ...

  3. 一脸懵逼加从入门到绝望学习hadoop之 org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=Administrator, access=WRITE, inode="/":root:supergroup:drwxr-xr报错

    1:初学hadoop遇到各种错误,这里贴一下,方便以后脑补吧,报错如下: 主要是在window环境下面搞hadoop,而hadoop部署在linux操作系统上面:出现这个错误是权限的问题,操作hado ...

  4. Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=lenovo, access=WRITE, inode="/user/hadoop/spark/people_savemode_test/_temporary/0":hadoop:supergro

    保存文件时权限被拒绝 曾经踩过的坑: 保存结果到hdfs上没有写的权限 通过修改权限将文件写入到指定的目录下 * * * $HADOOP_HOME/bin/hdfs dfs -chmod 777 /u ...

  5. Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=Mypc, access=WRITE, inode="/":fan:supergroup:drwxr-xr-x

    在window上编程提示没有写Hadoop的权限 Exception in thread "main" org.apache.hadoop.security.AccessContr ...

  6. org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security .AccessControlException: Permission denied: user=Administrator, access=WRITE, inode="hadoop": hadoop:supergroup:rwxr-xr-x

    这时windows远程调试hadoop集群出现的这里 做个记录   我用改变系统变量的方法 修正了错误 网上搜索出来大概有三种: 1.在系统的环境变量或java JVM变量里面添加HADOOP_USE ...

  7. hadoop 3.x org.apache.hadoop.security.AccessControlException: Permission denied: user=Administrator, access=WRITE, inode="/":tele:supergroup:drwxr-xr-x

    权限不足,上传文件时应当使用启动hadoop的账户,即在获取FileSystem时就应当指定用户 修改后的代码 public class Demo1 { public static void main ...

  8. 访问HDFS报错:org.apache.hadoop.security.AccessControlException: Permission denied

    import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apac ...

  9. 从 "org.apache.hadoop.security.AccessControlException:Permission denied: user=..." 看Hadoop 的用户登陆认证

    假设远程提交任务给Hadoop 可能会遇到 "org.apache.hadoop.security.AccessControlException:Permission denied: use ...

随机推荐

  1. simple模式下rabbitmq的代码

    simple模式代码 package RabbitMQ import ( "fmt" "github.com/streadway/amqp" "log ...

  2. DVWA-XSS练习

    本周学习内容: 1.学习web应用安全权威指南: 2.学习乌云漏洞: 实验内容: DVWA实验XSS跨站脚攻击 实验步骤: Low 1.打开DVWA,进入DVWA security模块,将难度修改为L ...

  3. C# Ninject使用

    Ninject是一个IOC容器,用来解决程序中组件的耦合问题,它的目的在于做到最少配置.简单来讲就是 为我们选择一个想要的类来处理事务. 百度百科的解释:一个快如闪电.超轻量级的基于.Net平台的依赖 ...

  4. vmware exsi安装部署

    本文章参考:https://blog.csdn.net/fishinhouse/article/details/80980051 1.VMware-ESXi-6.5.0镜像下载 网盘链接:https: ...

  5. python操作toml文件

    # -*- coding: utf-8 -*- # @Time : 2019-11-18 09:31 # @Author : cxa # @File : toml_demo.py # @Softwar ...

  6. 已知 sqrt (2)约等于 1.414,要求不用数学库,求 sqrt (2)精确到小数点后 10 位

    问题:已知 sqrt (2)约等于 1.414,要求不用数学库,求 sqrt (2)精确到小数点后 10 位. 出题人:阿里巴巴出题专家:文景/阿里云 CDN 资深技术专家. 考察点:基础算法的灵活应 ...

  7. 360杯复赛流量分析题 详细writeup

    题目名: 这是捕获的黑客攻击数据包 通过分析流量包,得知黑客先上传了一个文件: 追踪TCP流,可以看到文件内容,是一个木马: 然后通过get请求一个加密key,在响应里能看到key的值. 接下来就是用 ...

  8. sql查询性能调试,用SET STATISTICS IO和SET STATISTICS TIME---解释比较详细

            一个查询需要的CPU.IO资源越多,查询运行的速度就越慢,因此,描述查询性能调节任务的另一种方式是,应该以一种使用更少的CPU.IO资源的方式重写查询命令,如果能够以这样一种方式完成查 ...

  9. Chapter Four

    JSON数据 默认情况下,当开发者新创建一个SpringBoot项目时,会添加Web依赖,在这个依赖中会默认加入jackson-databind作为Json处理器. @RestController 组 ...

  10. [spring-boot] 健康状况监控

    pom文件 <dependency> <groupId>org.springframework.boot</groupId> <artifactId>s ...