hadoop(十)hdfs上传删除文件（完全分布式七)|12

集群测试

上传小文件到集群，随便选择一个小文件上传到hdfs的根目录

[shaozhiqi@hadoop102 hadoop-3.1.2]$ bin/hdfs dfs -put wcinput/wc.input /
[shaozhiqi@hadoop102 hadoop-3.1.2]$

验证是否上传成功，验证ok,Replication的3代表三个副本数

image.png

上传一个大文件如我们的hadoop安装包

[shaozhiqi@hadoop102 hadoop-3.1.2]$ bin/hdfs dfs -put /opt/software/hadoop-3.1.2.tar.gz /

image.png

查看每个文件在hdfs的副本情况和存储块block，默认每个block大小为128M
关于块block的说明：当hdfs上收到的文件大于128M,则该文件会被切分存储为多个块，多个块以块为单位存储在不用的datanode上。但是值的注意的是如果某个文件大小没有达到128M,则按文件大小存储，不会占用整个128M的空间。

点击：Wc.Input

image.png

点击Hadoop-3.1.2.tar.gz

image.png

每个block最大存储128M，多出就会再多一个block去存储

文件在hdfs的实际路劲

[shaozhiqi@hadoop102 subdir0]$ ll
total 327200
-rw-rw-r--. 1 shaozhiqi shaozhiqi 50 Jun 27 21:18 blk_1073741825
-rw-rw-r--. 1 shaozhiqi shaozhiqi 11 Jun 27 21:18 blk_1073741825_1001.meta
-rw-rw-r--. 1 shaozhiqi shaozhiqi 134217728 Jun 27 21:25 blk_1073741826
-rw-rw-r--. 1 shaozhiqi shaozhiqi 1048583 Jun 27 21:25 blk_1073741826_1002.meta
-rw-rw-r--. 1 shaozhiqi shaozhiqi 134217728 Jun 27 21:26 blk_1073741827
-rw-rw-r--. 1 shaozhiqi shaozhiqi 1048583 Jun 27 21:26 blk_1073741827_1003.meta
-rw-rw-r--. 1 shaozhiqi shaozhiqi 63998133 Jun 27 21:26 blk_1073741828
-rw-rw-r--. 1 shaozhiqi shaozhiqi 499995 Jun 27 21:26 blk_1073741828_1004.meta
[shaozhiqi@hadoop102 subdir0]$ pwd
/opt/module/hadoop-3.1.2/data/tmp/dfs/data/current/BP-1884196904-192.168.1.102-1561640654781/current/finalized/subdir0/subdir0
[shaozhiqi@hadoop102 subdir0]$ cat blk_1073741825 #可以确定整个就是我们的wc.input
hadoop yarn
hadoop mapreduce
shaozhiqi
shaozhiqi
[shaozhiqi@hadoop102 subdir0]$

删除我们刚刚上传到hdfs的文件

[shaozhiqi@hadoop102 hadoop-3.1.2]$ bin/hdfs dfs -rm -r /wc.input
Deleted /wc.input
[shaozhiqi@hadoop102 hadoop-3.1.2]$ bin/hdfs dfs -rm -r /hadoop-3.1.2.tar.gz
Deleted /hadoop-3.1.2.tar.gz
[shaozhiqi@hadoop102 hadoop-3.1.2]$

验证是否删除，验证ok

image.png