How To: set udev rule for setting the disk permission on ASM disks when using multipath on Linux 6.x

在RHEL6.4上安装11gR2的RAC时，使用了MULTIPATH来聚合绑定多路径的磁盘，并且修改磁盘的权限，赋予grid:asmadmin用户和组。

此时，在安装时可以发现磁盘，日志如下

INFO: ... discoveryString = /dev/mapper/*
INFO: Executing [/tmp/OraInstall2014-11-24_04-55-14PM/ext/bin/kfod, nohdr=true, verbose=true, disks=all, status=true, op=disks, asm_diskstring='/dev/mapper/*']
 with environment variables {XFILESEARCHPATH=/usr/dt/app-defaults/%L/Dt, LANG=en_US.UTF-8, DISPLAY=:1.0, ORACLE_BASE=/oracle, PWD=/home/grid, _=/software/grid/install/.oui, ORACLE_SID=+ASM1, NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat, USER=grid, QTINC=/usr/lib64/qt-3.3/include, CLASSPATH=, G_BROKEN_FILENAMES=1, HOME=/home/grid, HOSTNAME=ssodb01, LESSOPEN=|/usr/bin/lesspipe.sh %s, LD_LIBRARY_PATH=/tmp/OraInstall2014-11-24_04-55-14PM/ext/bin/../lib, ORACLE_HOME=/tmp/OraInstall2014-11-24_04-55-14PM/ext/bin, LOGNAME=grid, SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass, SHELL=/bin/bash, CVS_RSH=ssh, SHLVL=2, HISTCONTROL=ignoredups, MAIL=/var/spool/mail/grid, QTDIR=/usr/lib64/qt-3.3, JAVA_HOME=, QTLIB=/usr/lib64/qt-3.3/lib, LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:, LC_ALL=, HISTSIZE=1000, TERM=xterm, PATH=/usr/bin:/usr/ccs/bin:/oracle/product/11g/grid/bin:/usr/sbin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/grid/bin:/software/grid/install}
INFO: Starting Output Reader Threads for process /tmp/OraInstall2014-11-24_04-55-14PM/ext/bin/kfod
INFO: Parsing 102400 CANDIDATE /dev/mapper/mpathb grid     asmadmin
INFO: The process /tmp/OraInstall2014-11-24_04-55-14PM/ext/bin/kfod exited with code 0
INFO: Waiting for output processor threads to exit.
INFO: Parsing 102400 CANDIDATE /dev/mapper/mpathc grid     asmadmin
INFO: Parsing 102400 CANDIDATE /dev/mapper/mpathd grid     asmadmin
INFO: Parsing 102400 CANDIDATE /dev/mapper/mpathe grid     asmadmin
INFO: Parsing 102400 CANDIDATE /dev/mapper/mpathf grid     asmadmin
INFO: Parsing 1024 CANDIDATE /dev/mapper/mpathg grid     asmadmin
INFO: Parsing 1024 CANDIDATE /dev/mapper/mpathh grid     asmadmin
INFO: Parsing 1024 CANDIDATE /dev/mapper/mpathi grid     asmadmin
INFO: Parsing 1024 CANDIDATE /dev/mapper/mpathj grid     asmadmin
INFO: Parsing 1024 CANDIDATE /dev/mapper/mpathk grid     asmadmin
INFO: Parsing 1024 CANDIDATE /dev/mapper/mpathl grid     asmadmin
INFO: Parsing 1024 CANDIDATE /dev/mapper/mpathm grid     asmadmin
INFO: Parsing 1024 CANDIDATE /dev/mapper/mpathn grid     asmadmin
INFO: Parsing 1024 CANDIDATE /dev/mapper/mpatho grid     asmadmin

再进一步使用时，遇到错误ins-30508 invalid asm disks

INFO: Starting Output Reader Threads for process /tmp/OraInstall2014-11-24_04-55-14PM/ext/bin/kfod
INFO: Parsing KFOD-00311: Error scanning device /dev/mapper/mpathi
INFO: Parsing ORA-15025: could not open disk "/dev/mapper/mpathi"
INFO: Parsing Linux-x86_64 Error: 13: Permission denied
INFO: The process /tmp/OraInstall2014-11-24_04-55-14PM/ext/bin/kfod exited with code 0
INFO: Parsing Additional information: 42
INFO: Waiting for output processor threads to exit.
INFO: Parsing Additional information: 22608487
INFO: Parsing Additional information: 1598903119
INFO: Parsing KFOD-00311: Error scanning device /dev/mapper/mpathh
INFO: Parsing ORA-15025: could not open disk "/dev/mapper/mpathh"
INFO: Parsing Linux-x86_64 Error: 13: Permission denied
INFO: Parsing Additional information: 42
INFO: Parsing Additional information: 22608487
INFO: Parsing Additional information: 1598903119
INFO: Parsing KFOD-00311: Error scanning device /dev/mapper/mpathg
INFO: Parsing ORA-15025: could not open disk "/dev/mapper/mpathg"
INFO: Parsing Linux-x86_64 Error: 13: Permission denied
INFO: Parsing Additional information: 42
INFO: Parsing Additional information: 22608487
INFO: Parsing Additional information: 1598903119
INFO: Output processor threads exited.
SEVERE: [FATAL] [INS-30508] Invalid ASM disks.
   CAUSE: The disks [/dev/mapper/mpathg, /dev/mapper/mpathh, /dev/mapper/mpathi] were not valid.
   ACTION: Please choose or enter valid ASM disks.
INFO: Advice is ABORT

原来识别到的磁盘此时的权限已经发生变化。此时就需要在multipath层面对磁盘进行显示的权限属组配置。

查看Redhat 6的文档，有这么一段话，来源https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/DM_Multipath/config_file_defaults.html

As of the Red Hat Enterprise Linux 6.0 release, the mode, uid, and gid parameters have been deprecated.
Permissions for device-mapper devices (including mulitpath mappings) are set by means of udev rules.
There is a template file in /usr/share/doc/device-mapper-version called 12-dm-permissions.rules
which you can use and place in the /etc/udev/rules.d directory for it to take effect.

1、使用dmsetup ls查看确认已配置的multipath的磁盘设备

[root@ssodb01 ~]# dmsetup ls | grep mpath| sort
mpathb  (253:4)
mpathc  (253:2)
mpathd  (253:3)
mpathe  (253:5)
mpathf  (253:6)
mpathg  (253:8)
mpathh  (253:9)
mpathi  (253:7)
mpathj  (253:15)
mpathk  (253:10)
mpathl  (253:11)
mpathm  (253:13)
mpathn  (253:14)
mpatho  (253:12)

2、编辑文件/etc/udev/rules.d/12-dm-permissions.rules，显示配置磁盘的属组为grid:asmadmin。据文档描述，该文档的模板存在于/usr/share/doc/device-mapper-{version}/12-dm-permissions.rules。增加如下片段

ENV{DM_NAME}=="mpathb",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"
ENV{DM_NAME}=="mpathc",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"
ENV{DM_NAME}=="mpathd",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"
ENV{DM_NAME}=="mpathe",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"
ENV{DM_NAME}=="mpathf",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"
ENV{DM_NAME}=="mpathg",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"
ENV{DM_NAME}=="mpathh",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"
ENV{DM_NAME}=="mpathi",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"
ENV{DM_NAME}=="mpathj",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"
ENV{DM_NAME}=="mpathk",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"
ENV{DM_NAME}=="mpathl",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"
ENV{DM_NAME}=="mpathm",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"
ENV{DM_NAME}=="mpathn",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"
ENV{DM_NAME}=="mpatho",OWNER:="grid",GROUP:="asmadmin",MODE:="660",SYMLINK+="mapper/$env{DM_NAME}"

3、重启服务或刷新重载，Any way you like。

4、Restart the GI Installation Session。