Changing Controller Numbers in Solaris

If you need to change the controller numbers (c#) that a disk has assigned to it, whether it is for cluster, Solstice DiskSuite[TM]/Solaris[TM] Volume Manager, or just for ease of management. With Solaris[TM] 8 and above, there is a new directory, /dev/cfg, that allows us to move them easily.

Steps to Follow

You will have to quiesce the data to all of the devices that you are going to move. This cannot be done while the system is trying to access these paths. This works for SAN, and internal disks.

Looking at the output of format and cfgadm we can see that our disks are on c3 and c4.

root@ralph / # echo|format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c0t0d0
/pci@1f,0/pci@1,1/ide@3/dad@0,0
1. c3t50020F230000C21Ed0
/pci@1f,0/pci@1/SUNW,qlc@3/fp@0,0/ssd@w50020f230000c21e,0
2. c3t50020F230000C21Ed1
/pci@1f,0/pci@1/SUNW,qlc@3/fp@0,0/ssd@w50020f230000c21e,1
3. c3t50020F230000C21Ed2
/pci@1f,0/pci@1/SUNW,qlc@3/fp@0,0/ssd@w50020f230000c21e,2
4. c3t50020F230000C21Ed3
/pci@1f,0/pci@1/SUNW,qlc@3/fp@0,0/ssd@w50020f230000c21e,3
5. c4t50020F230000C21Ed0
/pci@1f,0/pci@1/SUNW,qlc@3,1/fp@0,0/ssd@w50020f230000c21e,0
6. c4t50020F230000C21Ed1
/pci@1f,0/pci@1/SUNW,qlc@3,1/fp@0,0/ssd@w50020f230000c21e,1
7. c4t50020F230000C21Ed2
/pci@1f,0/pci@1/SUNW,qlc@3,1/fp@0,0/ssd@w50020f230000c21e,2
8. c4t50020F230000C21Ed3
/pci@1f,0/pci@1/SUNW,qlc@3,1/fp@0,0/ssd@w50020f230000c21e,3
Specify disk (enter its number):

root@ralph / # cfgadm -o show_FCP_dev -al
Ap_Id Type Receptacle Occupant Condition
c3 fc-fabric connected configured unknown
c3::50020f230000c21e,0 disk connected configured unknown
c3::50020f230000c21e,1 disk connected configured unknown
c3::50020f230000c21e,2 disk connected configured unknown
c3::50020f230000c21e,3 disk connected configured unknown
c4 fc-fabric connected configured unknown
c4::50020f230000c21e,0 disk connected configured unknown
c4::50020f230000c21e,1 disk connected configured unknown
c4::50020f230000c21e,2 disk connected configured unknown
c4::50020f230000c21e,3 disk connected configured unknown

Looking at /dev/cfg, you can see that there are links to the device paths with a controller number designation.

root@ralph / # cd /dev/cfg
root@ralph /dev/cfg # ls
./ ../ c0@ c3@ c4@
root@ralph /dev/cfg # ls -l
total 16
drwxr-xr-x 2 root root 512 Apr 20 12:50 ./
drwxr-xr-x 17 root sys 3584 May 5 17:47 ../
lrwxrwxrwx 1 root root 41 Apr 6 14:18 c0 -> ../../devices/pci@1f,0/pci@1,1/ide@3:scsi
lrwxrwxrwx 1 root root 49 Apr 13 15:21 c3 -> ../../devices/pci@1f,0/pci@1/SUNW,qlc@3/fp@0,0:fc
lrwxrwxrwx 1 root root 51 Apr 13 15:21 c4 -> ../../devices/pci@1f,0/pci@1/SUNW,qlc@3,1/fp@0,0:fc

All you have to do at this point, is remove any extra links, for clean up. And then move the controller, in this case c4, to the new controller number.

root@ralph /dev/cfg # mv c4 c2

Now the '../../devices/pci@1f,0/pci@1/SUNW,qlc@3,1/fp@0,0:fc' link has been moved from c4 to c2. When we recreate the link Solaris will look at this and create all of the links on c2. In a sense 'forcing' it to the controller that we want.

root@ralph /dev/cfg # ls
./ ../ c0@ c2@ c3@
root@ralph /dev/cfg # ls -l
total 16
drwxr-xr-x 2 root root 512 May 9 10:17 ./
drwxr-xr-x 17 root sys 3584 May 9 10:18 ../
lrwxrwxrwx 1 root root 41 Apr 6 14:18 c0 -> ../../devices/pci@1f,0/pci@1,1/ide@3:scsi
lrwxrwxrwx 1 root root 51 Apr 13 15:21 c2 -> ../../devices/pci@1f,0/pci@1/SUNW,qlc@3,1/fp@0,0:fc
lrwxrwxrwx 1 root root 49 Apr 13 15:21 c3 -> ../../devices/pci@1f,0/pci@1/SUNW,qlc@3/fp@0,0:fc

Next remove all of the old device links in /dev/rdsk and /dev/dsk

root@ralph /dev/cfg # cd /dev/rdsk
root@ralph /dev/rdsk # rm -r c4*
root@ralph /dev/rdsk # cd /dev/dsk
root@ralph /dev/dsk # rm -r c4*

With direct attached disks and san disks that are already configured with cfgadm, just run devfsadm -C to clean up dangling /dev links.

root@ralph /dev/dsk # devfsadm -C

Now format and cfgadm show the c4 devices on c2.

root@ralph /dev/dsk # echo|format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c0t0d0
/pci@1f,0/pci@1,1/ide@3/dad@0,0
1. c2t50020F230000C21Ed0
/pci@1f,0/pci@1/SUNW,qlc@3,1/fp@0,0/ssd@w50020f230000c21e,0
2. c2t50020F230000C21Ed1
/pci@1f,0/pci@1/SUNW,qlc@3,1/fp@0,0/ssd@w50020f230000c21e,1
3. c2t50020F230000C21Ed2
/pci@1f,0/pci@1/SUNW,qlc@3,1/fp@0,0/ssd@w50020f230000c21e,2
4. c2t50020F230000C21Ed3
/pci@1f,0/pci@1/SUNW,qlc@3,1/fp@0,0/ssd@w50020f230000c21e,3
5. c3t50020F230000C21Ed0
/pci@1f,0/pci@1/SUNW,qlc@3/fp@0,0/ssd@w50020f230000c21e,0
6. c3t50020F230000C21Ed1
/pci@1f,0/pci@1/SUNW,qlc@3/fp@0,0/ssd@w50020f230000c21e,1
7. c3t50020F230000C21Ed2
/pci@1f,0/pci@1/SUNW,qlc@3/fp@0,0/ssd@w50020f230000c21e,2
8. c3t50020F230000C21Ed3
/pci@1f,0/pci@1/SUNW,qlc@3/fp@0,0/ssd@w50020f230000c21e,3
Specify disk (enter its number):

You can do this to any controller except the device that you are booted off of. Doing this procedure on your boot controller can cause the system to panic.