HDFS重启集群导致数据损坏，使用fsck命令修复过程

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　作者：尹正杰

版权声明：原创作品，谢绝转载！否则将追究法律责任。

　　我们先看一组输出

[root@flume112 ~]# hdfs fsck /user/kuaikan/report_new////

Connecting to namenode via http://calculation111.aggrx:50070

FSCK started by root (auth:SIMPLE) from /10.1.2.112 for path /user/kuaikan/report_new//// at Sat May  :: CST

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................Status: HEALTHY

 Total size:     B (Total open files size:  B)

 Total dirs:

 Total files:

 Total symlinks:                 (Files currently being written: )

 Total blocks (validated):       (avg. block size  B) (Total open file blocks (not validated): )　　　　#注意，这个26没有进行验证，说明有26个存在问题！

 Minimally replicated blocks:    (100.0 %)

 Over-replicated blocks:         (0.0 %)

 Under-replicated blocks:        (0.0 %)

 Mis-replicated blocks:          (0.0 %)

 Default replication factor:

 Average block replication:     3.0

 Corrupt blocks:

 Missing replicas:               (0.0 %)

 Number of data-nodes:

 Number of racks:

FSCK ended at Sat May  :: CST  in  milliseconds

The filesystem under path '/user/kuaikan/report_new/2019/05/23/' is HEALTHY

[root@flume112 ~]#

　　官方关于fsck命令有对应的解释,我们查看更详细的信息，官方链接：https://hadoop.apache.org/docs/r2.9.1/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#fsck。

[root@flume112 ~]# hdfs fsck /user/kuaikan/report_new////  -openforwrite

Connecting to namenode via http://calculation111.aggrx:50070

FSCK started by root (auth:SIMPLE) from /10.1.2.112 for path /user/kuaikan/report_new//// at Sat May  :: CST

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

................................................/user/kuaikan/report_new////report.log.----45.10---160.1558554302405.txt  bytes,  block(s), OPENFORWRITE: ../user/kuaikan/report_new////report.log.----45.10---248.1558554301696.txt  bytes,  block(s), OPENFORWRITE: ....../user/kuaikan/report_new////report.log.----45.10---71.1558554301576.txt  bytes,  block(s), OPENFORWRITE: .........................................

/user/kuaikan/report_new////report.log.----55.10---208.1558554901562.txt  bytes,  block(s), OPENFORWRITE: .../user/kuaikan/report_new////report.log.----55.10---248.1558554901949.txt  bytes,  block(s), OPENFORWRITE: ...../user/kuaikan/report_new////report.log.----55.10---233.1558554901647.txt  bytes,  block(s), OPENFORWRITE: .........................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

................................................................/user/kuaikan/report_new////report.log.----30.10---208.1558575001741.txt  bytes,  block(s), OPENFORWRITE: ...................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

......................./user/kuaikan/report_new////report.log.----55.10---93.1558587301545.txt  bytes,  block(s), OPENFORWRITE: ........../user/kuaikan/report_new////report.log.----55.10---88.1558587301803.txt  bytes,  block(s), OPENFORWRITE: .........................../user/kuaikan/report_new////report.log.----01.10---88.1558587709627.txt  bytes,  block(s), OPENFORWRITE: .....................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

.........................................................................../user/kuaikan/report_new////report.log.----45.10---35.1558601101874.txt  bytes,  block(s), OPENFORWRITE: /user/kuaikan/report_new////report.log.----45.10---40.1558601102383.txt  bytes,  block(s), OPENFORWRITE: /user/kuaikan/report_new////report.log.----45.10---46.1558601101876.txt  bytes,  block(s), OPENFORWRITE: /user/kuaikan/report_new////report.log.----45.10---58.1558601102255.txt  bytes,  block(s), OPENFORWRITE: /user/kuaikan/report_new////report.log.----45.10---93.1558601102225.txt  bytes,  block(s), OPENFORWRITE: /user/kuaikan/report_new////report.log.----45.10---38.1558601101874.txt  bytes,  block(s), OPENFORWRITE: /user/kuaikan/report_new////report.log.----45.10---208.1558601101557.txt  bytes,  block(s), OPENFORWRITE: /user/kuaikan/report_new////report.log.----45.10---160.1558601101782.txt  bytes,  block(s), OPENFORWRITE: /user/kuaikan/report_new////report.log.----45.10---33.1558601102283.txt  bytes,  block(s), OPENFORWRITE: /user/kuaikan/report_new////report.log.----45.10---150.1558601101668.txt  bytes,  block(s), OPENFORWRITE: /user/kuaikan/report_new////report.log.----45.10---248.1558601101427.txt  bytes,  block(s), OPENFORWRITE: ./user/kuaikan/report_new////report.log.----45.10---226.1558601101904.txt  bytes,  block(s), OPENFORWRITE: ./user/kuaikan/report_new////report.log.----45.10---96.1558601101582.txt  bytes,  block(s), OPENFORWRITE: /user/kuaikan/report_new////report.log.----45.10---88.1558601101302.txt  bytes,  block(s), OPENFORWRITE: /user/kuaikan/report_new////report.log.----45.10---233.1558601101531.txt  bytes,  block(s), OPENFORWRITE: /user/kuaikan/report_new////report.log.----45.10---71.1558601101720.txt  bytes,  block(s), OPENFORWRITE: .......

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

....................................................................................................

..............................................................Status: HEALTHY

 Total size:     B

 Total dirs:

 Total files:

 Total symlinks:

 Total blocks (validated):       (avg. block size  B)

 Minimally replicated blocks:    (99.50153 %)

 Over-replicated blocks:         (0.0 %)

 Under-replicated blocks:        (0.0 %)

 Mis-replicated blocks:          (0.0 %)

 Default replication factor:

 Average block replication:     2.985046

 Corrupt blocks:

 Missing replicas:               (0.0 %)

 Number of data-nodes:

 Number of racks:

FSCK ended at Sat May  :: CST  in  milliseconds

The filesystem under path '/user/kuaikan/report_new/2019/05/23/' is HEALTHY

[root@flume112 ~]#

[root@flume112 ~]# hdfs fsck /user/kuaikan/report_new/2019/05/23/ -openforwrite

找到损坏的文件后，我们手动进行回复，注意，我们输入的必须是一个文件，并不是一个目录哟！

[root@calculation111 ~]# hdfs debug recoverLease -path /user/kuaikan/report_new////report.log.----45.10---71.1558554301576.txt  -retries

recoverLease returned false.

Retrying in  ms...

Retry #

recoverLease SUCCEEDED on /user/kuaikan/report_new////report.log.----45.10---71.1558554301576.txt

[root@calculation111 ~]#

HDFS重启集群导致数据损坏，使用fsck命令修复过程的更多相关文章

  1. Hadoop基础-HDFS集群中大数据开发常用的命令总结

    Hadoop基础-HDFS集群中大数据开发常用的命令总结 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 本盘博客仅仅列出了我们在实际生成环境中常用的hdfs命令,如果想要了解更多, ...

  2. Hadoop集群-HDFS集群中大数据运维常用的命令总结

    Hadoop集群-HDFS集群中大数据运维常用的命令总结 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 本篇博客会简单涉及到滚动编辑,融合镜像文件,目录的空间配额等运维操作简介.话 ...

  3. Hadoop（八）Java程序访问HDFS集群中数据块与查看文件系统

    前言 我们知道HDFS集群中,所有的文件都是存放在DN的数据块中的.那我们该怎么去查看数据块的相关属性的呢?这就是我今天分享的内容了 一.HDFS中数据块概述 1.1.HDFS集群中数据块存放位置 我 ...

  4. 本地日志数据实时接入到hadoop集群的数据接入方案

    1. 概述 本手册主要介绍了,一个将传统数据接入到Hadoop集群的数据接入方案和实施方法.供数据接入和集群运维人员参考. 1.1.  整体方案 Flume作为日志收集工具,监控一个文件目录或者一个文 ...

  5. elasticsearch5.0集群大数据量迁移方法及注意事项

    当es集群的数据量较小的情况下elasticdump这个工具比较方便,但是当数据量达到一定级别比如上百G的时候,elasticdump速度就很慢了,此时我们可以使用快照的方法进行备份 elasticd ...

  6. MongoDB DBA 实践5-----复制集集群的数据同步和故障转移

    (1)复制集集群的数据同步 1>主节点数据库test,在其中goods集合中加入一个文档. 2>在副节点中查看 注意:SECONDARY是不允许读写的,要使用rs.slaveOk()获得读 ...

  7. 软件架构自学笔记----分享“去哪儿 Hadoop 集群 Federation 数据拷贝优化”

    去哪儿 Hadoop 集群 Federation 数据拷贝优化 背景 去哪儿 Hadoop 集群随着去哪儿网的发展一直在优化改进,基本保证了业务数据存储量和计算量爆发式增长下的存储服务质量.然而,随着 ...

  8. Redis 切片集群的数据倾斜分析

    Redis 中如何应对数据倾斜 什么是数据倾斜 数据量倾斜 bigkey导致倾斜 Slot分配不均衡导致倾斜 Hash Tag导致倾斜 数据访问倾斜 如何发现 Hot Key Hot Key 如何解决 ...

  9. KingbbaseES V8R6集群维护案例之---集群之间数据迁移

    案例说明: 生产环境是集群环境,测试环境是集群,现需要将生产环境的数据迁移到测试集群中运行,本文档详细介绍了从集群环境迁移数据的操作步骤,可以作为生产环境迁移数据的参考. 适用版本: Kingbase ...

随机推荐

  1. k8s记录-不同集群服务互联

    1.外部访问内部服务 添加nodePort 修改type:ClusteIP为type:NodePort 2.内部服务访问外部服务 kubectl create -f test.yaml apiVers ...

  2. [转]使用apt安装nodejs10

    使用apt安装nodejs10 链接地址:https://blog.csdn.net/sunhaobo1996/article/details/80340513

  3. [LeetCode] 489. Robot Room Cleaner 扫地机器人

    Given a robot cleaner in a room modeled as a grid. Each cell in the grid can be empty or blocked. Th ...

  4. k8s pv,pvc无法删除问题

    一般删除步骤为:先删pod再删pvc最后删pv 但是遇到pv始终处于“Terminating”状态,而且delete不掉.如下图: 解决方法: 直接删除k8s中的记录: 1 kubectl patch ...

  5. 【嵌入式硬件Esp32】安装MQTT服务器(Windows) 并连接测试

    对于不知道MQTT的肯定会问MQTT是干什么的....... 现在我有一个项目需求, 看到这个项目第一想法肯定需要一个服务器,所有的wifi设备和手机都去连接这个服务器,然后服务器进行信息的中转,类似 ...

  6. 【视频开发】【CUDA开发】ffmpeg Nvidia硬件加速总结

    原文链接:https://developer.nvidia.com/ffmpeg GPU-accelerated video processing integrated into the most p ...

  7. Netty 面试题解析

    1.Netty 是什么? Netty 是一款基于 NIO(Nonblocking I/O,非阻塞IO)开发的网络通信框架,对比于 BIO(Blocking I/O,阻塞IO),他的并发性能得到了很大提 ...

  8. 用Python写一个滑动验证码

    1.准备阶段 滑动验证码我们可以直接用GEETEST的滑动验证码. 打开网址:https://www.geetest.com/ ,找到技术文档中的行为验证,打开部署文档,点击Python,下载ZIP包 ...

  9. TensorFlow学习笔记（1）—— 基本概念与框架

    入门框架时的常见问题 学习框架的原因? 方便.易用 学习框架的哪些知识点? 掌握一个项目的基本流程,就知道需要学习哪些知识点了 迅速学习框架的方法 根据项目每块流程的需要针对性的学 可以看官方的入门教 ...

  10. hdu1016 Prime Ring Problem【素数环问题（经典dfs）】

    Prime Ring Problem Time Limit: 4000/2000 MS (Java/Others)    Memory Limit: 65536/32768 K (Java/Other ...