Search This Blog

Monday, May 23, 2011

In vgdisplay Cur LV != Open LV


On one of our systems I see that the current number of logical volumes in vg00 does not equal the number of logical volumes currently open. The host does not seem to have any unusual problems because of this.

There are 5 pairs of files in the /dev/vg_ctmapp_repl_01 directory, which matches the 5 "open" LVs. However, there are 6 reported as "Cur LV".

Unfortunately, due to additions/deletions the minor numbers are all over the map ranging from 0x000001 to 0x00001c.

How can I find out which ones are missing? Are there other things which could cause the Cur LV to not match Open LV?

root> vgdisplay /dev/vg_ctmapp_repl_01
--- Volume groups ---
VG Name                     /dev/vg_ctmapp_repl_01
VG Write Access             read/write
VG Status                   available
Max LV                      255
Cur LV                      6
Open LV                     5
Max PV                      255
Cur PV                      4
Act PV                      4
Max PE per PV               15800
VGDA                        8
PE Size (Mbytes)            32
Total PE                    1732
Alloc PE                    1600
Free PE                     132
Total PVG                   0
Total Spare PVs             0
Total Spare PVs in use      0

root> ls -l /dev/vg_ctmapp_repl_01 | sort +5
total 0
crw-r-----   1 root       root        64 0x1f0000 Oct 18  2009 group
brw-r-----   1 root       sys         64 0x1f0001 Oct 18  2009 lv_ctm_em
crw-r-----   1 root       sys         64 0x1f0001 Oct 18  2009 rlv_ctm_em
brw-r-----   1 root       sys         64 0x1f0002 Oct 18  2009 lv_ctmslap
crw-r-----   1 root       sys         64 0x1f0002 Oct 18  2009 rlv_ctmslap
brw-r-----   1 root       sys         64 0x1f0003 Oct 18  2009 lv_ctmslep
crw-r-----   1 root       sys         64 0x1f0003 Oct 18  2009 rlv_ctmslep
brw-r-----   1 root       sys         64 0x1f0004 Oct 18  2009 lv_ctmslnp
crw-r-----   1 root       sys         64 0x1f0004 Oct 18  2009 rlv_ctmslnp
brw-r-----   1 root       sys         64 0x1f0005 Oct 18  2009 lv_ctmsvxp
crw-r-----   1 root       sys         64 0x1f0005 Oct 18  2009 rlv_ctmsvxp

root> strings /etc/lvmtab
/dev/vg00
/dev/dsk/c2t0d0
/dev/dsk/c2t1d0
/dev/vg_common_01
/dev/dsk/c9t15d5
/dev/dsk/c8t15d5
/dev/dsk/c9t15d6
/dev/dsk/c8t15d6
/dev/vg_lms_common_01
/dev/dsk/c7t1d1
/dev/dsk/c6t1d1
/dev/vg_ctmapp_repl_01
/dev/dsk/c6t3d6
/dev/dsk/c7t3d6
/dev/dsk/c7t3d7
/dev/dsk/c6t3d7
/dev/dsk/c6t4d0
/dev/dsk/c7t4d0
/dev/dsk/c7t4d1
/dev/dsk/c6t4d1

root> pvdisplay -v /dev/dsk/c6t4d1|more
Device file path "/dev/dsk/c6t4d1" is an alternate path
to the Physical Volume. Using Primary Link "/dev/dsk/c7t4d1".
--- Physical volumes ---
PV Name                     /dev/dsk/c7t4d1
PV Name                     /dev/dsk/c6t4d1     Alternate Link
VG Name                     /dev/vg_ctmapp_repl_01
PV Status                   available
Allocatable                 yes
VGDA                        2
Cur LV                      0
PE Size (Mbytes)            32
Total PE                    433
Free PE                     132
Allocated PE                301
Stale PE                    0
IO Timeout (Seconds)        default
Autoswitch                  On
Proactive Polling           On


   --- Physical extents ---
   PE    Status   LV                      LE
   00000 current  ???                     00019
   00001 current  ???                     00020
   00002 current  ???                     00021
   00003 current  ???                     00022
   00004 current  ???                     00023
   00005 current  ???                     00024
   00006 current  ???                     00025
   00007 current  ???                     00026
   00008 current  ???                     00027
   00009 current  ???                     00028
   00010 current  ???                     00029
   00011 current  ???                     00030
   00012 current  ???                     00031
   00013 current  ???                     00032
   00014 current  ???                     00033
   00015 current  ???                     00034
   00016 current  ???                     00035
   00017 current  ???                     00036
   00018 current  ???                     00037
   00019 current  ???                     00038
   00020 current  ???                     00039
   00021 current  ???                     00040
   00022 current  ???                     00041

# mknod /dev/vg_ctmapp_repl_01/lv_xyz b 64 0x1f0006
# mknod /dev/vg_ctmapp_repl_01/rlv_xyz c 64 0x1f0006

dr023-root> ls -l /dev/vg_ctmapp_repl_01 | sort +5
total 0
crw-r-----   1 root       root        64 0x1f0000 Oct 18  2009 group
brw-r-----   1 root       sys         64 0x1f0001 Oct 18  2009 lv_ctm_em
crw-r-----   1 root       sys         64 0x1f0001 Oct 18  2009 rlv_ctm_em
brw-r-----   1 root       sys         64 0x1f0002 Oct 18  2009 lv_ctmslap
crw-r-----   1 root       sys         64 0x1f0002 Oct 18  2009 rlv_ctmslap
brw-r-----   1 root       sys         64 0x1f0003 Oct 18  2009 lv_ctmslep
crw-r-----   1 root       sys         64 0x1f0003 Oct 18  2009 rlv_ctmslep
brw-r-----   1 root       sys         64 0x1f0004 Oct 18  2009 lv_ctmslnp
crw-r-----   1 root       sys         64 0x1f0004 Oct 18  2009 rlv_ctmslnp
brw-r-----   1 root       sys         64 0x1f0005 Oct 18  2009 lv_ctmsvxp
crw-r-----   1 root       sys         64 0x1f0005 Oct 18  2009 rlv_ctmsvxp
brw-r--r--   1 root       sys         64 0x1f0006 May 22 12:26 lv_xyz
crw-r--r--   1 root       sys         64 0x1f0006 May 22 12:26 rlv_xyz

root> vgdisplay /dev/vg_ctmapp_repl_01
--- Volume groups ---
VG Name                     /dev/vg_ctmapp_repl_01
VG Write Access             read/write
VG Status                   available
Max LV                      255
Cur LV                      6
Open LV                     6
Max PV                      255
Cur PV                      4
Act PV                      4
Max PE per PV               15800
VGDA                        8
PE Size (Mbytes)            32
Total PE                    1732
Alloc PE                    1600
Free PE                     132
Total PVG                   0
Total Spare PVs             0
Total Spare PVs in use      0

# fsck -F vxfs /dev/vg_ctmapp_repl_01/lv_xyz
# mount -o largefiles,noatime,log /dev/vg_ctmapp_repl_01/lv_xyz /mnt

Solution Used:
My guess is the device file for the one (1) "mysterious" LVs are was deleted, that's why Cur LV does not match Open LV. The minor numbers are increment (eg: 0x000001, 0x000002, 0x000003 and so on). So it goes from 0->9,a->f and 10->11 (ie 18 all together for your case). Now all you have to do is look in /dev/vg00 and find out which minor number is missing.
So for instance 6 and 7 are missing. Before that for all the disks (PVs) in vg00, run this.
# pvdisplay -v /dev/rdsk/cXtYdZ | more
==> I think in the extents where the missing LV is you should see "???" in the output.
If you see it proceed ... we have to recreate the device file until the correct minor number is found for the pvdisplay to NOT show "???".
Now you manually create the "missing LV" (call it lvol99)
# mknod /dev/vg00/lvol99 b 64 0x000006
# mknod /dev/vg00/rlvol99 c 64 0x000006
Run the "pvdisplay" again and if you still see "???", remove the "lvol99" file and recreate it with the next minor number. Repeat till you get the correct one.
As you can see it can get quite messy. ANy additional input from you will help.

No comments:

Post a Comment