启动HDFS伪分布式环境时报权限错误

问题描述

操作系统：Ubuntu18.04 LTS

HDFS版本：hadoop-3.2.3

普通用户登录，参照官方文档在单机上安装伪分布式环境时，启动HDFS报权限错误。

具体报错信息如下：

$ ./sbin/start-dfs.sh
Starting namenodes on [localhost]
pdsh@zhangsan-ZHAOYANG-K21-80: localhost: rcmd: socket: Permission denied
Starting datanodes
pdsh@zhangsan-ZHAOYANG-K21-80: localhost: rcmd: socket: Permission denied
Starting secondary namenodes [zhangsan-ZHAOYANG-K21-80]
pdsh@zhangsan-ZHAOYANG-K21-80: zhangsan-ZHAOYANG-K21-80: rcmd: socket: Permission denied

已经设置过本机SSH免密码登录：

$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ chmod 0600 ~/.ssh/authorized_keys

原因分析及解决

虽然已经设置了本机ssh免密登录，但是由于pdsh默认使用的是rsh，而并非ssh，因此报错。

解决办法：

$ vim ~/.bashrc
export PDSH_RCMD_TYPE=ssh

$ source ~/.bashrc

再次执行HDFS启动时即可成功。

$ ./sbin/start-dfs.sh
Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [zhangsan-ZHAOYANG-K21-80]

【参考】

https://stackoverflow.com/questions/42756555/permission-denied-error-while-running-start-dfs-sh Permission Denied error while running start-dfs.sh

https://www.xiexianbin.cn/linux/commands/rcmd/index.html rcmd

https://developer.aliyun.com/article/401197 并行分布式运维工具pdsh

https://www.cnblogs.com/chen2ha/p/14787728.html Linux海王之pdsh

https://blog.csdn.net/qq_37865420/article/details/117263959 pdsh使用

https://blog.csdn.net/love666666shen/article/details/74350358 hadoop伪分布式下 无法启动datanode的原因及解决办法