Linux_LVM Couldn't find device with uuid

Linux LVM commands result in

• Couldn't find device with uuid

• Couldn't find all physical volumes for volume group

---

Help!  Commands like 'lvs', 'lvdisplay', 'vgdisplay', and 'pvscan' result in an error like the following:

Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find all physical volumes for volume group metabackupvg.
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find all physical volumes for volume group metabackupvg.
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find all physical volumes for volume group metabackupvg.
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find all physical volumes for volume group metabackupvg.

Volume group "metabackupvg" not found

[root@dwetlprod2 ~]# pvscan
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  PV /dev/emcpoweraa1   VG srcdata1vg      lvm2 [50.00 GB / 0    free]
  PV /dev/emcpowerz1    VG srcdata2vg      lvm2 [50.00 GB / 0    free]
  PV /dev/emcpowerj1    VG metalogvg       lvm2 [3.00 GB / 0    free]
  PV /dev/emcpowerl1    VG metalogvg       lvm2 [3.00 GB / 0    free]
  PV /dev/emcpowern1    VG metalogvg       lvm2 [4.99 GB / 0    free]
  PV /dev/emcpowerad1   VG metalogvg       lvm2 [14.00 GB / 0    free]
  PV /dev/emcpowery1    VG auditbackupvg   lvm2 [229.99 GB / 0    free]
  PV /dev/emcpoweru1    VG metabackupvg    lvm2 [45.00 GB / 0    free]
  PV unknown device     VG metabackupvg    lvm2 [69.99 GB / 0    free]
  PV /dev/emcpoweraf1   VG metabackupvg    lvm2 [124.99 GB / 0    free]
  PV /dev/emcpowerv1    VG auditlogvg      lvm2 [3.00 GB / 0    free]
  PV /dev/emcpowerw1    VG auditlogvg      lvm2 [3.00 GB / 0    free]
  PV /dev/emcpowerx1    VG auditlogvg      lvm2 [3.00 GB / 0    free]
  PV /dev/emcpowert1    VG auditlogvg      lvm2 [89.99 GB / 0    free]
  PV /dev/emcpowerb1    VG auditdatavg     lvm2 [25.00 GB / 0    free]
  PV /dev/emcpowerc1    VG auditdatavg     lvm2 [25.00 GB / 0    free]
  PV /dev/emcpowerd1    VG auditdatavg     lvm2 [25.00 GB / 0    free]
  PV /dev/emcpowere1    VG auditdatavg     lvm2 [25.00 GB / 0    free]
  PV /dev/emcpowerk1    VG auditdatavg     lvm2 [25.00 GB / 0    free]
  PV /dev/emcpowerm1    VG auditdatavg     lvm2 [25.00 GB / 0    free]
  PV /dev/emcpowero1    VG auditdatavg     lvm2 [25.00 GB / 0    free]
  PV /dev/emcpowerq1    VG auditdatavg     lvm2 [25.00 GB / 0    free]
  PV /dev/emcpowerp1    VG auditdatavg     lvm2 [25.00 GB / 0    free]
  PV /dev/emcpowerr1    VG auditdatavg     lvm2 [25.00 GB / 0    free]
  PV /dev/emcpowerab1   VG auditdatavg     lvm2 [25.00 GB / 0    free]
  PV /dev/emcpowerf1    VG metadatavg      lvm2 [11.00 GB / 0    free]
  PV /dev/emcpowerg1    VG metadatavg      lvm2 [11.00 GB / 0    free]
  PV /dev/emcpowerh1    VG metadatavg      lvm2 [11.00 GB / 0    free]
  PV /dev/emcpoweri1    VG metadatavg      lvm2 [11.00 GB / 0    free]
  PV /dev/emcpowers1    VG metadatavg      lvm2 [20.00 GB / 0    free]
  PV /dev/emcpowera1    VG metadatavg      lvm2 [20.00 GB / 0    free]
  PV /dev/emcpowerae1   VG metadatavg      lvm2 [14.00 GB / 0    free]
  Total: 32 [1.04 TB] / in use: 32 [1.04 TB] / in no VG: 0 [0   ]

Solution

---

This scenario indicates that a disk is missing from the volume group.
 This is clear since pvscan scans all disks for physical volumes, and
it's telling us that it cannot find a device.  In my case, I deleted a
LUN from the EMC that I knew was no longer being used, but I forgot to
update the LVM configuration.  To correct the issue, I removed the lost
physical volume from the volume group using the --removemissing argument of the vgreduce command.

Tip: Run vgreduce with the --test and --verbose options to validate what will be executed and confirm this is what you want to do.

[root@dwetlprod2 ~]# vgreduce --removemissing --verbose metabackupvg
    Finding volume group "metabackupvg"
    Wiping cache of LVM-capable devices
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find all physical volumes for volume group metabackupvg.
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find all physical volumes for volume group metabackupvg.
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find all physical volumes for volume group metabackupvg.
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find all physical volumes for volume group metabackupvg.
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
  Couldn't find device with uuid 'R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP'.
    Archiving volume group "metabackupvg" metadata (seqno 73).
    metabackupvg/backup2lv has missing extents: removing (including dependencies)
    Removing LV backup2lv from VG metabackupvg
    Removing PV with UUID R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP from VG metabackupvg
    Creating volume group backup "/etc/lvm/backup/metabackupvg" (seqno 74).
  Wrote out consistent volume group metabackupvg
[root@dwetlprod2 ~]#

If a device needs to be added back, a solution for recovery is to cut
another 70G LUN on the EMC, present it to the host and add it back into
the volume group with the same UUID.  Read my post on adding a new lun.

[root@dwetlprod2 ~]# pvcreate --uuid R5zG2s-g4yi-ytEO-8Xvk-2B0s-S36G-9nWhXP /dev/emcpowerah
  Physical volume "/dev/emcpowerah" successfully created
[root@dwetlprod2 ~]# vgcfgrestore metabackupvg
  Restored volume group metabackupvg
[root@dwetlprod2 ~]# vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "srcdata1vg" using metadata type lvm2
  Found volume group "srcdata2vg" using metadata type lvm2
  Found volume group "metalogvg" using metadata type lvm2
  Found volume group "auditbackupvg" using metadata type lvm2
  Found volume group "auditlogvg" using metadata type lvm2
  Found volume group "metabackupvg" using metadata type lvm2
  Found volume group "auditdatavg" using metadata type lvm2
  Found volume group "metadatavg" using metadata type lvm2

Run a file system check on the device then mount it.  If your device
(LUN) was mistakenly deleted, or in the case of physical media the disk
has gone kaput, then you'll need to restore data from backups.

You may have noticed that I didn't even fdisk the replacement device.
 Don't forget to do this step if you will be restoring data and you're
restoring to a partition.