保存文件时权限被拒绝 曾经踩过的坑: 保存结果到hdfs上没有写的权限 通过修改权限将文件写入到指定的目录下
*
*
* $HADOOP_HOME/bin/hdfs dfs -chmod 777 /user
*
*
* Exception in thread "main" org.apache.hadoop.security.AccessControlException:
* Permission denied: user=Mypc, access=WRITE,
* inode="/":fan:supergroup:drwxr-xr-x

package cn.spark.study.sql;

import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.sql.DataFrame;
import org.apache.spark.sql.SQLContext;
import org.apache.spark.sql.SaveMode; /**
* SaveModel示例
*
* @author 张运涛
*
*/
public class SaveModeTest { @SuppressWarnings("deprecation")
public static void main(String[] args) {
System.setProperty("hadoop.home.dir", "E:\\hadoop-2.7.1"); SparkConf conf = new SparkConf().setMaster("local").setAppName("SaveModeTest");
JavaSparkContext sc = new JavaSparkContext(conf);
SQLContext sqlContext = new SQLContext(sc); DataFrame peopleDF = sqlContext.read().format("json").load("hdfs://zhang:9000/user/hadoop/spark/people.json");
/**
* 保存文件时权限被拒绝 曾经踩过的坑: 保存结果到hdfs上没有写的权限 通过修改权限将文件写入到指定的目录下
*
*
* $HADOOP_HOME/bin/hdfs dfs -chmod 777 /user
*
*
* Exception in thread "main" org.apache.hadoop.security.AccessControlException:
* Permission denied: user=Mypc, access=WRITE,
* inode="/":fan:supergroup:drwxr-xr-x
*
*/ peopleDF.save("hdfs://zhang:9000/user/people_savemode_test", "json", SaveMode.Append);
} }

Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=lenovo, access=WRITE, inode="/user/hadoop/spark/people_savemode_test/_temporary/0":hadoop:supergro的更多相关文章

  1. Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=Mypc, access=WRITE, inode="/":fan:supergroup:drwxr-xr-x

    在window上编程提示没有写Hadoop的权限 Exception in thread "main" org.apache.hadoop.security.AccessContr ...

  2. kylin cube测试时,报错:org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

    异常: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, i ...

  3. 一脸懵逼加从入门到绝望学习hadoop之 org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=Administrator, access=WRITE, inode="/":root:supergroup:drwxr-xr报错

    1:初学hadoop遇到各种错误,这里贴一下,方便以后脑补吧,报错如下: 主要是在window环境下面搞hadoop,而hadoop部署在linux操作系统上面:出现这个错误是权限的问题,操作hado ...

  4. 报错:HDFS IO error org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/yk/dl/alarm_his":hdfs:supergroup:drwxr-xr-x

    报错背景: CDH集成了Flume服务,准备通过Flume将kafka中的数据放到HDFS中, 启动Flume的时候报错. 报错现象: // :: INFO hdfs.HDFSDataStream: ...

  5. 异常-Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=hdfs, access=WRITE, inode="/hbase":root:supergroup:drwxr-xr-x

    1 详细异常 Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlExce ...

  6. 【Hadoop & Ecilpse】Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=bruce, access=WRITE, inode="/out2/_temporary/0":atguigu:supergroup:drwxr-xr-x

    问题再现: 使用本机 Ecilpse  (Windows环境) 去访问远程 hadoop 集群出现以下异常: 问题原因: 因为远程提交的情况下如果没有 hadoop 的系统环境变量,就会读取当前主机的 ...

  7. hadoop 3.x org.apache.hadoop.security.AccessControlException: Permission denied: user=Administrator, access=WRITE, inode="/":tele:supergroup:drwxr-xr-x

    权限不足,上传文件时应当使用启动hadoop的账户,即在获取FileSystem时就应当指定用户 修改后的代码 public class Demo1 { public static void main ...

  8. org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security .AccessControlException: Permission denied: user=Administrator, access=WRITE, inode="hadoop": hadoop:supergroup:rwxr-xr-x

    这时windows远程调试hadoop集群出现的这里 做个记录   我用改变系统变量的方法 修正了错误 网上搜索出来大概有三种: 1.在系统的环境变量或java JVM变量里面添加HADOOP_USE ...

  9. 访问HDFS报错:org.apache.hadoop.security.AccessControlException: Permission denied

    import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apac ...

随机推荐

  1. py3下怎么用StringIO

    try: from StringIO import StringIO except ImportError: from io import StringIO

  2. C# websocket与html js实现文件发送与接收处理

    C# websocket与html js实现文件发送与接收处理 using System; using System.Collections.Generic; using System.Linq; u ...

  3. Centos 7.x nginx隐藏版本号

    一.打开配置文件 #vim /etc/nginx/nginx.conf 二.增加一行: server_tokens    off; 三.重启nginx #nginx -s reload 四.效果

  4. Introducing Apache Spark Datasets(中英双语)

    文章标题 Introducing Apache Spark Datasets 作者介绍 Michael Armbrust, Wenchen Fan, Reynold Xin and Matei Zah ...

  5. 史上最简单的 SpringCloud 教程 | 终章

    https://blog.csdn.net/forezp/article/details/70148833转载请标明出处:http://blog.csdn.net/forezp/article/det ...

  6. oracle 函数 输入值,查询数据,返回相应处理结果

    create or replace function FUN_SEARCH_ORDERBY ( INSTACEID in varchar2,TYE IN varchar2) return varcha ...

  7. amqp笔记

    1.exchange message的生产者可以将消息发送给exchange,然后由exchange路由到不同的queue中. exchange有4种类型: direct exchange:msg只会 ...

  8. Protobuf3 序列化

    在message_lite.h中定义了SerializeToString ,SerializeToArray ,SerializeToCodedStream ,SerializeToZeroCopyS ...

  9. [转]SOA架构设计经验分享—架构、职责、数据一致性

    阅读目录: 1.背景介绍 2.SOA的架构层次 2.1.应用服务(原子服务) 2.2.组合服务 2.3.业务服务(编排服务) 3.SOA化的重构 3.1.保留服务空间,为了将来服务的组合 4.运用DD ...

  10. 怎么让Windows2012和Windows2008多用户同时远程

    具体方法请参照百度经验:http://jingyan.baidu.com/article/cd4c2979f19765756e6e60ec.html.经过实践证明,是没有问题的.