hadoop报错WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/06/14 10:44:58 WARN common.Util: Path /opt/hadoopdata/hdfs/name should be specified as a URI in configuration files. Please update hdfs configuration.
19/06/14 10:44:58 WARN common.Util: Path /opt/hadoopdata/hdfs/name should be specified as a URI in configuration files. Please update hdfs configuration.
19/06/14 10:44:58 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
1 解决
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
查看
[root@hadoop1 conf]# sed -i '$a export HADOOP_ROOT_LOGGER=DEBUG,console' /etc/profile
[root@hadoop1 conf]# source /etc/profile
[hadoop@hadoop1 sbin]$ hadoop fs -ls /
19/06/14 11:04:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 2 items
drwxrwx--- - hadoop supergroup 0 2019-05-31 16:26 /tmp
drwxr-xr-x - hadoop supergroup 0 2019-05-31 16:20 /user
##查看文件是否有系统一致
[root@hadoop1 native]# file /opt/hadoop/lib/native/libhadoop.so.1.0.0
libhadoop.so.1.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
[hadoop@hadoop2 sbin]$ uname -i
x86_64
给hadoop执行操作了debug，还是没看到详细日志
一开始执行网上的这些操作步骤
vim /opt/hadoop/etc/hadoop/hadoop-env.sh
vim /etc/profile
export HADOOP_HOME=/opt/hadoop/
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib:$HADOOP_COMMON_LIB_NATIVE_DIR"
source /etc/profile
vim ~/.bashrc
source ~/.bashrc
完成之后还是报错
于是开始执行
[root@hadoop1 build]# strings /lib64/libc.so.6 | grep GLIBC
GLIBC_2.2.5
GLIBC_2.2.6
GLIBC_2.3
GLIBC_2.3.2
GLIBC_2.3.3
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.5
GLIBC_2.6
GLIBC_2.7
GLIBC_2.8
GLIBC_2.9
GLIBC_2.10
GLIBC_2.11
GLIBC_2.12
GLIBC_PRIVATE
应该是缺少了2.14的支持
http://ftp.gnu.org/gnu/glibc/glibc-2.14.tar.gz
tar -zxvf glibc-2.14.tar.gz
cd glibc-2.14 && mkdir build && cd build
../configure --prefix=/opt/glibc-2.14
##如果有报错
configure: error: in `/opt/glibc-2.14/build':
configure: error: no acceptable C compiler found in $PATH
--执行yum install -y gcc gcc-c++ make cmake
make -j4
make install
[root@hadoop1 build]# mkdir /opt/glibc-2.14/etc/
[root@hadoop1 build]# cp /etc/ld.so.c* /opt/glibc-2.14/etc/
cp: omitting directory `/etc/ld.so.conf.d'
[root@hadoop1 build]# ln -sf /opt/glibc-2.14/lib/libc-2.14.so /lib64/libc.so.6
[root@hadoop1 build]# strings /lib64/libc.so.6 | grep GLIBC
GLIBC_2.2.5
GLIBC_2.2.6
GLIBC_2.3
GLIBC_2.3.2
GLIBC_2.3.3
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.5
GLIBC_2.6
GLIBC_2.7
GLIBC_2.8
GLIBC_2.9
GLIBC_2.10
GLIBC_2.11
GLIBC_2.12
GLIBC_2.13
GLIBC_2.14
GLIBC_PRIVATE
[hadoop@hadoop1 sbin]$ hadoop fs -ls / ###不在报错
Found 2 items
drwxrwx--- - hadoop supergroup 0 2019-05-31 16:26 /tmp
drwxr-xr-x - hadoop supergroup 0 2019-05-31 16:20 /user
2 hdfs文件修<!-- Put site-specific property overrides in this file. --
<configuration>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>hadoop2:9001</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/opt/hadoopdata/hdfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/opt/hadoopdata/hdfs/data</value>
</property>
<property>
<name>dfs.namenode.checkpoint.dir</name>
<value>/opt/hadoopdata/hdfs/snn</value>
</property>
<property>
<name>dfs.namenode.checkpoint.period</name>
<value>3600</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
====ha之后新增加的内容
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.nameservices</name>
<value>ns1</value>
</property>
<property>
<name>dfs.ha.namenodes.ns1</name>
<value>nn1,nn2</value>
</property>
<property>
<name>dfs.namenode.rpc-address.ns1.nn1</name>
<value>hadoop1:8020</value>
</property>
<property>
<name>dfs.namenode.rpc-address.ns1.nn2</name>
<value>hadoop2:8020</value>
</property>
<property>
<name>dfs.namenode.servicerpc-address.ns1.nn1</name>
<value>hadoop1:8040</value>
</property>
<property>
<name>dfs.namenode.servicerpc-address.ns1.nn2</name>
<value>hadoop2:8040</value>
</property> <property>
<name>dfs.namenode.http-address.ns1.nn1</name>
<value>hadoop1:50070</value>
</property>
<property>
<name>dfs.namenode.http-address.ns1.nn2</name>
<value>hadoop2:50070</value>
</property> <property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://hadoop1:8485;hadoop2:8485;hadoop3:8485/ns1</value>
</property>
<property>
<name>dfs.client.failover.proxy.provider.ns1</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/opt/hadoopdata/hdfs/journal</value>
</property>
<property>
<name>dfs.ha.fencing.methods</name>
<value>sshfence</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/home/hadoop/.ssh/id_rsa</value>
</property>
</configuration>
修改并应用到其他节点
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>hadoop2:9001</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:///opt/hadoopdata/hdfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:////opt/hadoopdata/hdfs/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
启动
[hadoop@hadoop1 hadoop]$ /opt/hadoop/sbin/start-dfs.sh
Starting namenodes on [hadoop1 hadoop2]
hadoop2: starting namenode, logging to /opt/hadoop-2.9.1/logs/hadoop-hadoop-namenode-hadoop2.out
hadoop1: starting namenode, logging to /opt/hadoop-2.9.1/logs/hadoop-hadoop-namenode-hadoop1.out
hadoop2: starting datanode, logging to /opt/hadoop-2.9.1/logs/hadoop-hadoop-datanode-hadoop2.out
hadoop1: starting datanode, logging to /opt/hadoop-2.9.1/logs/hadoop-hadoop-datanode-hadoop1.out
hadoop3: starting datanode, logging to /opt/hadoop-2.9.1/logs/hadoop-hadoop-datanode-hadoop3.out
Starting journal nodes [hadoop1 hadoop2 hadoop3]
hadoop1: starting journalnode, logging to /opt/hadoop-2.9.1/logs/hadoop-hadoop-journalnode-hadoop1.out
hadoop2: starting journalnode, logging to /opt/hadoop-2.9.1/logs/hadoop-hadoop-journalnode-hadoop2.out
hadoop3: starting journalnode, logging to /opt/hadoop-2.9.1/logs/hadoop-hadoop-journalnode-hadoop3.out
Starting ZK Failover Controllers on NN hosts [hadoop1 hadoop2]
hadoop1: starting zkfc, logging to /opt/hadoop-2.9.1/logs/hadoop-hadoop-zkfc-hadoop1.out
hadoop2: starting zkfc, logging to /opt/hadoop-2.9.1/logs/hadoop-hadoop-zkfc-hadoop2.out
hadoop报错WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable的更多相关文章
- Hadoop集群“WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable”解决办法
Hadoop集群部署完成后,经常会提示 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platfo ...
- Hadoop问题解决：WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
在配置好hadoop的环境之后,命令启动./start-all.sh发现经常出现这样的一个警告: WARN util.NativeCodeLoader: Unable to load native-h ...
- hadoop "startdfs.sh" WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
这个waring 信息是可以忽略的.下面是解决方案 在hadoop-env.sh中添加 export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.pat ...
- 17/11/24 05:08:44 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2017-11-24 21:20:25 1:什么叫失望,什么叫绝望.总之是一脸懵逼的继续...... 之前部署的hadoop都是hadoop-2.4.1.tar.gz,这几天换成了hadoop-2.6 ...
- hadoop2.4 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
在Ubuntu上安装完hadoop2.4以后,使用以下命令: hadoop fs -ls // :: WARN util.NativeCodeLoader: Unable to load native ...
- Hadoop安装—— WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platf
今天在安装hadoop完成测试创建用户目录失败在网上找到了原因记录一下原文地址 http://blog.csdn.net/l1028386804/article/details/51538611 配置 ...
- Hadoop - 彻底解决警告：WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
目录 1 - 在日志配置文件中忽略警告 - 有效 2 - 指定本地库的路径 - 无效 3 - 不使用 Hadoop 本地库 - 无效 4 - 替换 Hadoop 本地库 - 有效 5 - 根据源码,编 ...
- [hadoop] WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
hadoop 启动后,有警告信息: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform ...
- HADOOP:WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable终于解决了
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin- ...
随机推荐
- hier - 文件系统描述
DESCRIPTION 描述 一个典型的Linux系统具有以下几个目录: / 根目录,是所有目录树开始的地方. /bin 此目录下包括了单用户方式及系统启动或修复所用到的所有执行程序. /boot 包 ...
- vi/vim常用命令总结
vim是vi的升级模式,完全兼容vi 解决vim打开中文乱码问题 编辑/etc/vim/vimrc配置文件,添加下面的内容: ''' set fileencodings=utf-8,ucs-bom,g ...
- php产品细节图多图上传示例代码 无刷新
前台文件代码 upload.html <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" " ...
- robotframework调用外部python多次运行拿到的都是同一个值
外部python是一个爬虫,爬取的内容的定义没有放入函数中.导致一次爬取多次使用的情况出现. 第一版函数如下: 改版后:
- python基础练习题6
01:求1+2!+3!+....+20!的和 s=0 t=1 for n in range(1,21): t*=n s+=t print('1!+2!+3!+.....+20!=%d'%s) 02:利 ...
- 动态树(Link-Cut-Tree)简单总结(指针版本)
Link-Cut-Tree(动态树 LCT) 1.定义 1. 偏爱子节点: 一颗子树内最后访问的点若在该子树根节点 X 的某个儿子节点 P 的子树中,则称 P 为 X 的偏爱子节点. 偏爱边:连向偏爱 ...
- SQL 使用分区方法
- 配置MongoDB的Windows服务
[1] 创建directorys和files Create a configuration file and a directory path for MongoDB log output (logp ...
- 更好的构建 Node 服务的工具
更好的构建 Node 服务的工具 无论前端项目在打包后都发送给后台, 有时候自己想看看效果在运行 npm run build 后只是看到一个 build 文件夹,但是直接打开是无法浏览,因此需要开启一 ...
- Android系统关机或重启的几种实现方式
前阵子工作上遇到一些关于Android系统关机或重启的系统修改,于是,做了一些尝试,也搜集了一下资料,现在整理一下,做一些总结,方便学习或者日后工作的需要. 默认的SDK并没有提供应用开发者直接的An ...