linux 10201 ASM RAC 安装+升级到10205

准备环境的时 ，要4个对外IP，2个对内IP

不超过2T,，一般都用OCFS

高端存储适合用ASM

linux10G安装的时候，安装的机器时间要小于等于（如果是等于要严格等于）第二个机器的时间（只有linux 10G会有这个问题），不然会报错，拷贝空值什么的。

出现此错误，说明是时间问题，

hosts里面要注意的地方：

1）127.0.0.1 那一行不能删除。

2）主机名要小写，不能大写。

3）不能加下划线。

1.安装系统，把所有的开发包全部安装上

关掉防火墙和SELinux

yum -y install compat-libstdc++* 
yum -y install elfutils-libelf-devel* 
yum -y install elfutils-libelf-devel-static* 
yum -y install gcc* 
yum -y install gcc-c++* 
yum -y install glibc* 
yum -y install glibc-common* 
yum -y install glibc-devel* 
yum -y install glibc-headers* 
yum -y install kernel-headers* 
yum -y install ksh* 
yum -y install libaio* 
yum -y install libaio-devel* 
yum -y install libgcc* 
yum -y install libgomp* 
yum -y install libstdc++* 
yum -y install libstdc++-devel* 
yum -y install make* 
yum -y install sysstat* 
yum -y install unixODBC* 
yum -y install unixODBC-devel* 
yum -y install libXp*

2.配置核心参数

vi /etc/sysctl.conf

kernel.core_uses_pid = 1
fs.file-max = 65536
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 1048576
net.core.rmem_max = 1048576
net.core.wmem_default = 262144
net.core.wmem_max = 262144
kernel.shmmni = 4096
kernel.sem = 500 64000 100 128
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1

修改完执行此命令 ： sysctl -p

vi  /etc/profile 
if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi
之后，执行:$ulimit验证一下.

vi  /etc/csh.login
if ( $USER == "oracle" ) then
limit maxproc 16384
limit descriptors 65536
umask 022
endif

vi /etc/pam.d/login
session    required     pam_limits.so

vi /etc/security/limits.conf
oracle  soft     nofile 655360
oracle  hard     nofile 635360
oracle  soft    nproc   10240
oracle  hard    nproc   16384

3.配置网络
127.0.0.1     linux1 localhost.localdomain localhost
需要将其删除成如下（注意上面的linux1被删除）： 
127.0.0.1     localhost.localdomain localhost
如果 RAC 节点名出现在回送地址中，您在 RAC 安装期间将接收到以下错误信息： 
ORA-00603:ORACLE server session terminated by fatal error
或 
ORA-29702:error occurred in Cluster Group Service operation

在RAC1中，需要配置两个网段（RAC2也一样）

192.168.56.101 是公有网段

10.10.10.11 是私有网段

在RAC1中，配置两个文件可以达到修改IP的目的（RAC2中也如此类似操作）

vi  /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
BOOTPROTO=none
HWADDR=08:00:27:65:d4:be
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
IPADDR=192.168.56.101
NETMASK=255.255.255.0
GATEWAY=192.168.56.1

vi  /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth1
BOOTPROTO=static
ONBOOT=yes
HWADDR=08:00:27:5f:b7:85
IPADDR=10.10.10.11
NETMASK=255.255.255.0

之后，重启IP，使之生效。

service network restart

vi /etc/hosts

127.0.0.1  localhost.localdomain localhost
::1              localhost6.localdomain6  localhost6

#rac1
192.168.56.101 rac1
192.168.56.111 rac1vip
10.10.10.11    rac1priv
#rac2
192.168.56.102 rac2
192.168.56.222 rac2vip
10.10.10.22    rac2priv

4.创建用户，用户组

/usr/sbin/groupadd -g 1000 oinstall
/usr/sbin/groupadd -g 1200 dba
/usr/sbin/useradd -u 555 -g oinstall -G dba oracle

两边的用户id，组id 要一样

mkdir -p   /u01/app/oracle/product/10.2/db_1

mkdir -p   /u01/app/oracle/product/10.2/crs
chown -R oracle:oinstall /u01
chmod -R 775 /u01

vi /home/oracle/.bash_profile

export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/10.2/db_1
export ORA_CRS_HOME=$ORACLE_BASE/product/10.2/crs
export ORACLE_TERM=xterm
export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data
LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export LD_LIBRARY_PATH
export PATH=$PATH:$ORACLE_HOME/bin:$ORA_CRS_HOME/bin:$PATH

5 . 配置互信（一定要是oracle用户，两个节点都要做。）
/usr/bin/ssh-keygen -t rsa
/usr/bin/ssh-keygen -t dsa
把4个文件合成一个文件

注意权限要是是644

chmod 644 authorized_keys

在节点RAC2进行的操作

[oracle@RAC2 ~]$ ls -a
.   .bash_history  .bash_profile  .emacs  .mozilla  .viminfo
..  .bash_logout   .bashrc        .kde    .ssh      .Xauthority

[oracle@RAC2 ~]$ cd .ssh

[oracle@RAC2 .ssh]$ ls
id_dsa  id_dsa.pub  id_rsa  id_rsa.pub

[oracle@RAC2 .ssh]$ scp id_dsa.pub rac1:/home/oracle/.ssh/id_dsa.pub2
oracle@rac1's password: 
id_dsa.pub                                         100%  601     0.6KB/s   00:00

[oracle@RAC2 .ssh]$ ls
id_dsa  id_dsa.pub  id_rsa  id_rsa.pub  known_hosts

[oracle@RAC2 .ssh]$ scp id_rsa.pub rac1:/home/oracle/.ssh/id_rsa.pub2
oracle@rac1's password: 
id_rsa.pub

再到RAC1节点
cat id_dsa.pub  id_dsa.pub2  id_rsa.pub  id_rsa.pub2 > authorized_keys

scp authorized_keys  rac2:/home/oracle/.ssh

之后要测试一下（如果都不需要密码，得到时间后。就说明ssh 对等性配置成功。）

[oracle@rac1 ~]$ ssh rac1 date

[oracle@rac1 ~]$ ssh rac2 date
Sat Sep 14 16:03:58 CST 2013

[oracle@rac1 ~]$ ssh rac1priv date
Sat Sep 14 16:04:59 CST 2013

[oracle@rac1 ~]$ ssh rac2priv date
The authenticity of host 'rac2priv (10.10.10.22)' can't be established.
RSA key fingerprint is db:ad:62:77:5a:b5:d6:8f:c6:55:a9:64:61:9b:39:bc.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'rac2priv,10.10.10.22' (RSA) to the list of known hosts.
Sat Sep 14 16:05:09 CST 2013

[oracle@rac2 ~]$ ssh rac1 date
Sat Sep 14 16:06:54 CST 2013

[oracle@rac2 ~]$ ssh rac2 date
Sat Sep 14 16:09:30 CST 2013

[oracle@rac2 ~]$ ssh rac1priv date
Sat Sep 14 16:09:55 CST 2013

[oracle@rac2 ~]$ ssh rac2priv date
The authenticity of host 'rac2priv (10.10.10.22)' can't be established.
RSA key fingerprint is db:ad:62:77:5a:b5:d6:8f:c6:55:a9:64:61:9b:39:bc.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'rac2priv,10.10.10.22' (RSA) to the list of known hosts.
Sat Sep 14 16:10:03 CST 2013

6.创建共享裸设备

首先在rac3和rac4添加一块共享存储盘。

[root@rac3 ~]# fdisk -l

Disk /dev/sda: 8589 MB, 8589934592 bytes

255 heads, 63 sectors/track, 1044 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      BlocksId  System

/dev/sda1*           1          13      10439183  Linux

/dev/sda2              14        10448281507+  8e  Linux LVM

Disk /dev/sdb: 16.8 GB, 16862150656 bytes

255 heads, 63 sectors/track, 2050 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb doesn't contain a valid partition table

 

 

[root@rac4 ~]# fdisk -l

 

Disk /dev/sda: 8589 MB, 8589934592 bytes

255 heads, 63 sectors/track, 1044 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End   Blocks   IdSystem

/dev/sda1   *1          13      10439183  Linux

/dev/sda2              14        10448281507+  8e  Linux LVM

 

Disk /dev/sdb: 16.8 GB, 16862150656 bytes

255 heads, 63 sectors/track, 2050 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

Disk /dev/sdb doesn't contain a valid partition table

 

 

[root@rac3 ~]# fdisk /dev/sdb

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel. Changes will remain in memory only,

until you decide to write them. After that, of course, the previous

content won't be recoverable.

 

 

The number of cylinders for this disk is set to 2050.

There is nothing wrong with that, but this is larger than 1024,

and could in certain setups cause problems with:

1) software that runs at boot time (e.g., old versions of LILO)

2) booting and partitioning software from other OSs

   (e.g., DOS FDISK, OS/2 FDISK)

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

 

Command (m for help): d

No partition is defined yet!

 

Command (m for help): n

Command action

   eextended

   pprimary partition (1-4)

p

Partition number (1-4): 1

First cylinder (1-2050, default 1):

Using default value 1

Last cylinder or +size or +sizeM or +sizeK (1-2050, default 2050): 300m

 

Command (m for help): n

Command action

   eextended

   pprimary partition (1-4)

p

Partition number (1-4): 2

First cylinder (301-2050, default 301):

Using default value 301

Last cylinder or +size or +sizeM or +sizeK (301-2050, default 2050): 600m   //注意在这里一定要注意600这个值，这里的分区取值是600-前面

                                                                                                                   // 的300  ，所以是300，而不是600。

Command (m for help): n

Command action

   eextended

   pprimary partition (1-4)

p

Partition number (1-4): 3

First cylinder (601-2050, default 601):

Using default value 601

Last cylinder or +size or +sizeM or +sizeK (601-2050, default 2050):

Using default value 2050

 

Command (m for help): w

The partition table has been altered!

 

Calling ioctl() to re-read partition table.

Syncing disks.

[root@rac3 raw]# fdisk -l

 

Disk /dev/sda: 8589 MB, 8589934592 bytes

255 heads, 63 sectors/track, 1044 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End      BlocksId  System

/dev/sda1   *1          13      10439183  Linux

/dev/sda2              14        10448281507+  8e  Linux LVM

 

Disk /dev/sdb: 16.8 GB, 16862150656 bytes

255 heads, 63 sectors/track, 2050 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End      BlocksId  System

/dev/sdb1               1         3002409718+  83  Linux

/dev/sdb2             301         6002409750   83  Linux

/dev/sdb3             601        205011647125   83  Linux

 

 

 

[root@rac4 raw]# fdisk /dev/sdb

 

The number of cylinders for this disk is set to 2050.

There is nothing wrong with that, but this is larger than 1024,

and could in certain setups cause problems with:

1) software that runs at boot time (e.g., old versions of LILO)

2) booting and partitioning software from other OSs

   (e.g., DOS FDISK, OS/2 FDISK)

 

Command (m for help): w

The partition table has been altered!

 

Calling ioctl() to re-read partition table.

Syncing disks.

[root@rac4 raw]# fdisk -l

 

Disk /dev/sda: 8589 MB, 8589934592 bytes

255 heads, 63 sectors/track, 1044 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End      BlocksId  System

/dev/sda1   *1          13      10439183  Linux

/dev/sda2              14        10448281507+  8e  Linux LVM

 

Disk /dev/sdb: 16.8 GB, 16862150656 bytes

255 heads, 63 sectors/track, 2050 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End      BlocksId  System

/dev/sdb1               1         3002409718+  83  Linux

/dev/sdb2             301         6002409750   83  Linux

/dev/sdb3             601        205011647125   83  Linux

 

[root@rac3 ~]# cd /etc/udev/rules.d

[root@rac3 rules.d]# ls

05-udev-early.rules  60-pcmcia.rules         90-alsa.rules

40-multipath.rules   60-raw.rules            90-dm.rules

50-udev.rules        60-wacom.rules          90-hal.rules

51-hotplug.rules     61-uinput-stddev.rules  95-pam-console.rules

60-libsane.rules     61-uinput-wacom.rules   98-kexec.rules

60-net.rules         85-pcscd_ccid.rules     bluetooth.rules

 

 

 

[root@rac3 rules.d]# vi 60-raw.rules

[root@rac3 rules.d]# cat 60-raw.rules

# Enter raw device bindings here.

#

# An example would be:

#   ACTION=="add", KERNEL=="sda", RUN+="/bin/raw /dev/raw/raw1 %N"

# to bind /dev/raw/raw1 to /dev/sda, or

#   ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="1", RUN+="/bin/raw /dev/raw/raw2 %M %m"

# to bind /dev/raw/raw2 to the device with major 8, minor 1.

 

ACTION=="add", KERNEL=="sdb1", RUN+="/bin/raw /dev/raw/raw1 %N"

ACTION=="add", KERNEL=="sdb2", RUN+="/bin/raw /dev/raw/raw3 %N"

ACTION=="add", KERNEL=="sdb3", RUN+="/bin/raw /dev/raw/raw4 %N"

KERNEL=="raw[1-9]", OWNER="oracle", GROUP="oinstall", MODE="0660"

 

 

[root@rac3 rules.d]# start_udev

Starting udev:                                             [  OK  ]

[root@rac3 rules.d]# ls -l /dev/raw

total 0

crw-rw---- 1 oracle oinstall 162, 1 Sep 17 10:21 raw1

crw-rw---- 1 oracle oinstall 162, 3 Sep 17 10:21 raw3

crw-rw---- 1 oracle oinstall 162, 4 Sep 17 10:21 raw4

 

[root@rac4 rules.d]# fdisk /dev/sdb

 

The number of cylinders for this disk is set to 2050.

There is nothing wrong with that, but this is larger than 1024,

and could in certain setups cause problems with:

1) software that runs at boot time (e.g., old versions of LILO)

2) booting and partitioning software from other OSs

   (e.g., DOS FDISK, OS/2 FDISK)

 

Command (m for help): w

The partition table has been altered!

 

Calling ioctl() to re-read partition table.

Syncing disks.

 

[root@rac4 dev]# cd /etc/udev/

[root@rac4 udev]# ls

makedev.drules.d  scripts  udev.conf

[root@rac4 udev]# cd rules.d

[root@rac4 rules.d]# ls

05-udev-early.rules  60-net.rules            61-uinput-wacom.rules  95-pam-console.rules

40-multipath.rules   60-pcmcia.rules         85-pcscd_ccid.rules    98-kexec.rules

50-udev.rules        60-raw.rules            90-alsa.rules          bluetooth.rules

51-hotplug.rules     60-wacom.rules          90-dm.rules

60-libsane.rules     61-uinput-stddev.rules  90-hal.rules

[root@rac4 rules.d]# vi 60-raw.rules

[root@rac4 rules.d]# more 60-raw.rules

# Enter raw device bindings here.

#

# An example would be:

#   ACTION=="add", KERNEL=="sda", RUN+="/bin/raw /dev/raw/raw1 %N"

# to bind /dev/raw/raw1 to /dev/sda, or

#   ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="1", RUN+="/bin/raw /dev/raw/raw2 %M %m"

# to bind /dev/raw/raw2 to the device with major 8, minor 1.

 

ACTION=="add", KERNEL=="sdb1", RUN+="/bin/raw /dev/raw/raw1 %N"

ACTION=="add", KERNEL=="sdb2", RUN+="/bin/raw /dev/raw/raw3 %N"

ACTION=="add", KERNEL=="sdb3", RUN+="/bin/raw /dev/raw/raw4 %N"

KERNEL=="raw[1-9]", OWNER="oracle", GROUP="oinstall", MODE="0660"   //生产环境中一般都要加上这一句

 

[root@rac4 rules.d]# start_udev

Starting udev:[  OK  ]

[root@rac4 rules.d]# cd /dev/raw

[root@rac4 raw]# ls

raw1  raw3raw4

[root@rac4 raw]# ll

total 0

crw-rw---- 1 oracle oinstall 162, 1 Sep 17 10:32 raw1

crw-rw---- 1 oracle oinstall 162, 3 Sep 17 10:32 raw3

crw-rw---- 1 oracle oinstall 162, 4 Sep 17 10:32 raw4

 

7，安装集群软件 

注意：出现如下的时候，一定要注意改变一下目录的路径

改成如下所示（不能放在db_1，要放在crs）：

 

当出现如下所示时

要改成如下所示：

 

当出现如下所示时

要改成如下所示

 

注意如下OCR和VOTE的存放方式：

 

到下面所示，会有提示执行两个脚本，两个节点都要执行这两个脚本

执行的顺序是：RAC3执行第一个脚本，RAC4执行第个脚本，RAC3执行第二个脚本，RAC4执行第二个脚本

 

出现如下，为正常：

 

执行上面的脚本，先别急着点OK，必须先执行以下步骤

用root用户

cd /u01/app/oracle/product/10.2/crs/bin

vi vipca加一行（两个节点都要做 ）

unset LD_ASSUME_KERNEL

 

 

./oifcfg getif    //查看是否有信息

./oifcfg setif -global eth0/192.168.56.0:public

./oifcfg setif -global eth1/10.10.10.0:cluster_interconnect

./vipca     //在第一个节点上做

如果在第一个节点（第二个节点没事）出现下面所示，说明raw没有清除干净。（一定要用dd命令才能删除，不能用rm命令）

ddif=/dev/zero  of=/dev/raw/raw1 bs=8192 count=100

8，安装数据库软件

9，升级集群软件和数据库软件

注意：这两个软件包是是同一个。

先升级集群软件，再升级数据库软件。

注意下面的一定要看清楚是集群软件还是数据库软件。

这里下面的是集群软件。我们选择先升级集群软件。

到这一步，一定要注意看清提示上说的，按照上面的步骤来做。

 

一定要看清楚里面说什么，不能退出。里面会提示执行脚本。

第一个节点执行完这两个脚本，然后再到第二个节点执行两个脚本。

 

10，升级完成，配监听（配完监听才能建库）

11,建库

dbca建库过程中， 遇到如下，注意，选 Select All

注意这里下面选择的是ASM方式。

注意下面这里的选择是pfile

注意下面这里的选择