ASMlib操作系统包安装与配置asm disk磁盘

1.加入6块硬盘，每块100g。不管是热加还是冷加。不管是加硬盘，用san存储划lun，或者再加上多路径，都是能够这么做的。

在操作系统层，能识别这种lun。以下的sdb就是一个刚划分的300g的lun（也就是硬盘。或者裸块设备）。

[root@pri ~]# cd /dev

[root@pri dev]# ls -l | grep sd

brw-r----- 1 root disk 8, 0 Apr 11 12:21 sda

brw-r----- 1 root disk 8, 1 Apr 11 12:21 sda1

brw-r----- 1 root disk 8, 2 Apr 11 12:21 sda2

brw-r----- 1 root disk 8, 16 Apr 11 12:21 sdb

分区

我们对硬盘进行分区，兴许把分区分别作为asm磁盘。事实上就一块机械硬盘，根本不是必需分区再做asm。

仅仅有多块硬盘的时候。能够用asm做镜像与条带化等。假设存储已经做了raid，给我划分的lun我也不是必需再做分区，直接能够加为asm了。

当然，比方有/dev/sda。/dev/sdb两块机械硬盘，我能够分别划分出/dev/sda1、/dev/sda2、/dev/sdb1、/dev/sdb2，然后摘取/dev/sda1与/dev/sdb1出来联合做一个asm disk1.

[root@pri dev]# fdisk /dev/sdb

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel. Changes will remain in memory only,

until you decide to write them. After that, of course, the previous

content won't be recoverable.

The number of cylinders for this disk is set to 39162.

There is nothing wrong with that, but this is larger than 1024,

and could in certain setups cause problems with:

1) software that runs at boot time (e.g., old versions of LILO)

2) booting and partitioning software from other OSs

   (e.g., DOS FDISK, OS/2 FDISK)

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n

Command action

   e extended

   p primary partition (1-4)

p

Partition number (1-4): 1

First cylinder (1-39162, default 1): 1

Last cylinder or +size or +sizeM or +sizeK (1-39162, default 39162): 60

Command (m for help): n

Command action

   e extended

   p primary partition (1-4)

2

Invalid partition number for type `2'

Command action

   e extended

   p primary partition (1-4)

p

Partition number (1-4): 2

First cylinder (61-39162, default 61): 61

Last cylinder or +size or +sizeM or +sizeK (61-39162, default 39162): 120

Command (m for help): n

Command action

   e extended

   p primary partition (1-4)

3

Invalid partition number for type `3'

Command action

   e extended

   p primary partition (1-4)

p

Partition number (1-4): 3

First cylinder (121-39162, default 121): 121

Last cylinder or +size or +sizeM or +sizeK (121-39162, default 39162): 180

Command (m for help): n

Command action

   e extended

   p primary partition (1-4)

p

Selected partition 4

First cylinder (181-39162, default 181): 181

Last cylinder or +size or +sizeM or +sizeK (181-39162, default 39162): 240

Command (m for help): n

You must delete some partition and add an extended partition first

Command (m for help): m

Command action

   a toggle a bootable flag

   b edit bsd disklabel

   c toggle the dos compatibility flag

   d delete a partition

   l list known partition types

   m print this menu

   n add a new partition

   o create a new empty DOS partition table

   p print the partition table

   q quit without saving changes

   s create a new empty Sun disklabel

   t change a partition's system id

   u change display/entry units

   v verify the partition table

   w write table to disk and exit

   x extra functionality (experts only)

Command (m for help): d

Partition number (1-4): 4

Command (m for help): n

Command action

   e extended

   p primary partition (1-4)

e

Selected partition 4

First cylinder (181-39162, default 181):

Using default value 181

Last cylinder or +size or +sizeM or +sizeK (181-39162, default 39162): 240

Command (m for help): n

First cylinder (181-240, default 181):

Using default value 181

Last cylinder or +size or +sizeM or +sizeK (181-240, default 240): 210

Command (m for help): n

First cylinder (211-240, default 211):

Using default value 211

Last cylinder or +size or +sizeM or +sizeK (211-240, default 240):

Using default value 240

Command (m for help): m

Command action

   a toggle a bootable flag

   b edit bsd disklabel

   c toggle the dos compatibility flag

   d delete a partition

   l list known partition types

   m print this menu

   n add a new partition

   o create a new empty DOS partition table

   p print the partition table

   q quit without saving changes

   s create a new empty Sun disklabel

   t change a partition's system id

   u change display/entry units

   v verify the partition table

   w write table to disk and exit

   x extra functionality (experts only)

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

[root@pri dev]# pwd

/dev

[root@pri dev]# ls | grep sd

sda

sda1

sda2

sdb

sdb1

sdb2

sdb3

sdb4

sdb5

sdb6

我们将sdb这块裸设备，分区称sdb1-sdb6，但这些sdb1-6也并不含文件系统。也就是说，还没被文件系统格式化。

从官方站点下载oracle程序包

pri:orcl2 > uname -a

Linux pri 2.6.18-308.el5 #1 SMP Fri Jan 27 17:17:51 EST 2012 x86_64 x86_64 x86_64 GNU/Linux

http://www.oracle.com/technetwork/topics/linux/asmlib

oracleasmlib-2.0 - ASM库

oracleasm-support-2.0 - utilities needed to administer ASMLib 管理ASMlib所需的有用程序

oracleasm - a kernel module for the Oracle ASM library

ASM库的内核模块

下载了这5个，一定要注意，看上面5个rpm包，都写着是64位的。

结果最后仅仅能装上这三个。

-rw-r--r-- 1 root root 25460 Apr 11 18:34 oracleasm-2.6.18-308.24.1.el5-2.0.5-1.el5.x86_64.rpm

-rw-r--r-- 1 root root 90225 Apr 11 18:36 oracleasm-support-2.1.8-1.el5.x86_64.rpm

-rw-r--r-- 1 root root 14176 Apr 11 18:36 oracleasmlib-2.0.4-1.el5.x86_64.rpm

对于dji单数据库用户来装，就不用分grid用户与oracle用户了。

据说不建asmadmin等等的组，那么就能够用普通的oracle用户也拥有asmdba权限。

加入每一个节点的/etc/security/limits.conf

oracle soft nproc 2047

oracle hard nproc 16384

oracle soft nofile 1024

oracle hard nofile 65536

oracle soft core unlimited

oracle soft core unlimited

加入每一个节点的/etc/pam.d/login文件

session required pam_limits.so

对于Bourne、Bash或Korn shell，将下面内容加到每一个节点的/etc/profile文件里。

if [ $USER = "oracle"] || [ $USER = "grid" ];then

    if [ $SHELL = "/bin/ksh" ];then

        ulimit -u 16384

        ulimit -n 65536

    else

        ulimit -u 16384 -n 65536

    fi

    umask 022

fi

配置ASMLib，不加-i就是显示当前配置，然后运行oracleasm init就等同于之前-i的指令。

/usr/sbin/oracleasm configure -i

配置之前：

[root@pri asmlib]# /usr/sbin/oracleasm configure

ORACLEASM_ENABLED=false

ORACLEASM_UID=

ORACLEASM_GID=

ORACLEASM_SCANBOOT=true

ORACLEASM_SCANORDER=""

ORACLEASM_SCANEXCLUDE=""

ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"

配置：

[root@pri asmlib]# /usr/sbin/oracleasm configure -i

Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library

driver. The following questions will determine whether the driver is

loaded on boot and what permissions it will have. The current values

will be shown in brackets ('[]'). Hitting <ENTER> without typing an

answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: oracle

Default group to own the driver interface []: dba

Start Oracle ASM library driver on boot (y/n) [n]: y

Scan for Oracle ASM disks on boot (y/n) [y]: y

Writing Oracle ASM library driver configuration: done

配置之后：

[root@pri asmlib]# /usr/sbin/oracleasm configure

ORACLEASM_ENABLED=true

ORACLEASM_UID=oracle

ORACLEASM_GID=dba

ORACLEASM_SCANBOOT=true

ORACLEASM_SCANORDER=""

ORACLEASM_SCANEXCLUDE=""

ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"

为asm加入asm磁盘

此时/dev/sdb1仅仅是操作系统认识的分区，我要将其设为让ASM也认识。

/usr/sbin/oracleasm createdisk data1 /dev/sdb1

每一个节点以root输入。ASM磁盘对每一个节点可用。

/usr/sbin/oracleasm scandisks

/usr/sbin/oracleasm listdisks（检验一下）

[root@pri asmlib]# /usr/sbin/oracleasm createdisk data1 /dev/sdb1

Writing disk header: done

Instantiating disk: failed

Clearing disk header: done

[root@pri asmlib]#

[root@pri asmlib]# /usr/sbin/oracleasm createdisk data1 /dev/sdb2

Writing disk header: done

Instantiating disk: failed

Clearing disk header: done

[root@pri asmlib]# /usr/sbin/oracleasm createdisk data1 /dev/sdb3

Writing disk header: done

Instantiating disk: failed

Clearing disk header: done

[root@pri asmlib]# /usr/sbin/oracleasm createdisk data1 /dev/sdb4

Writing disk header: failed

Unable to label device "/dev/sdb4"

[root@pri asmlib]# cd /dev

[root@pri dev]# ls | grep sd

sda

sda1

sda2

sdb

sdb1

sdb2

sdb3

sdb4

sdb5

sdb6

不知道为什么，我尽管有sdb4,可是这样会失败，是不是扩产分区就不能用来做asm盘的。

每一个节点以root输入，ASM磁盘对每一个节点可用。

[root@pri dev]# /usr/sbin/oracleasm scandisks

Reloading disk partitions: done

Cleaning any stale ASM disks...

Scanning system for ASM disks...

以下这个必须配置上，不然不能装GI。这个要跟vip在同一个网段上，而且不能是本机ip，也不能使已经使用的。

/etc/hosts  

10.60.17.150 pri-scan

假设出现[ins-40724] no locally defined network interface matches the scan subnet.报错。重新启动下主机就好。

后来又发现有报错，在例如以下这个界面。好像要再再/etc/hosts中，弄一个pri-vip。随便弄一个跟公网不同一个网段，但地址不一样的ip才干通过。

[root@pri ~]# /usr/sbin/oracleasm createdisk data1 /dev/sdb1

Writing disk header: done

Instantiating disk: failed

Clearing disk header: done

[root@pri ~]# vi /var/log/oracleasm

Creating /dev/oracleasm mount point: /dev/oracleasm

Loading module "oracleasm": failed

Unable to load module "oracleasm"

Cleaning any stale ASM disks...

Loading module "oracleasm": failed

Unable to load module "oracleasm"

最后发现。官网上给下载的是oracle linux的asm包。

那redhat 的怎么办呢？原来redhat的光碟自带有iso文件。

[root@erpp1 Server]# ls | grep oracleasm

oracleasm-2.6.18-274.el5-2.0.5-1.el5.x86_64.rpm

oracleasm-2.6.18-274.el5debug-2.0.5-1.el5.x86_64.rpm

oracleasm-2.6.18-274.el5xen-2.0.5-1.el5.x86_64.rpm

oracleasm-support-2.1.7-1.el5.x86_64.rpm

我就是下载了例如以下两个，rpm -ivh 。其余两个安装不了。

oracleasm-support-2.1.7-1.el5.x86_64.rpm

oracleasm-2.6.18-274.el5-2.0.5-1.el5.x86_64.rpm

[root@pri mnt]# cd /mnt

[root@pri mnt]# mkdir cdrom

[root@pri mnt]# mount /dev/cdrom /mnt/cdrom

mount: block device /dev/cdrom is write-protected, mounting read-only

发现还是不行。

将之前下载的64bit的oracleasmlib-2.0.4-1.el5.x86_64.rpm也上传上去，安装上。

也不行。尝试重新启动一下。

[root@erpp1 ~]# /usr/sbin/oracleasm createdisk data1 /dev/sdc1

Writing disk header: done

Instantiating disk: done

成功了。可能刚才直接重新启动也能够，不用加一个rpm包，也可能要。

反正重新启动是必须的。

[root@erpp1 ~]# /usr/sbin/oracleasm createdisk data1 /dev/sdc1

Writing disk header: done

Instantiating disk: done

[root@erpp1 ~]# /usr/sbin/oracleasm createdisk data2 /dev/sdb1

Writing disk header: done

Instantiating disk: done

[root@erpp1 ~]# /usr/sbin/oracleasm createdisk data3 /dev/sdd1

Writing disk header: done

Instantiating disk: done

[root@erpp1 ~]# /usr/sbin/oracleasm createdisk data4 /dev/sde1

Writing disk header: done

Instantiating disk: done

[root@erpp1 ~]# /usr/sbin/oracleasm scandisks

Reloading disk partitions: done

Cleaning any stale ASM disks...

Scanning system for ASM disks...

[root@erpp1 ~]# 

[root@erpp1 ~]# /usr/sbin/oracleasm listdisks  或者service oracleasm listdisks

DATA1

DATA2

DATA3

DATA4

有了这4个ASM盘后，開始装GI。

假设要删除 ASM磁盘可通过下面命令： 

[root@node1 ~]#/etc/init.d/oracleasm deletedisk ASMDISK5

[root@erpdb mapper]# /usr/sbin/oracleasm createdisk data1 /dev/mapper/mpath0p7

Writing disk header: done

Instantiating disk: done