[转]How to build a data storage and VM Server using comodity hardware and free software

Source: http://learnandremember.blogspot.jp/2010_01_01_archive.html

Requisites:

1) RAID protection for data
We'll use ZFS as filesystem. RAIDZ is somewhat akin to RAID5 and should provide better performance than other types of software RAID5 with several advantages that we'll discuss later.

2) UPS power outage protection
UPSes are expensive, and we're keeping our costs down. ZFS RAIDZ works eleminates the RAID5 loophole. This won't replace an UPS for all it is needed for, but we'll greatly improve the odds of not data corruption for files directly stored on the ZFS filesystem in the event of sudden power loss.

3) Controlled costs
ZFS is available on Linux trough xxxxx or natively in Open Solaris. Both cost zilch and can be used professionaly without costs.
Because cost is a factor, storage host and VM host should be the same hardware so direct access to hard disks by VM should be supported.

4) Ease of use and management and multiple OS virtualization ability
Even if setup is not a walk in the park, usage and maintnance should be easy. Also the virtualization environment should allow for windows, linux, solaris and as many of other OSes as possbible. This means Vmware, Xen Server, or xxxxx.

At the time i bought the parts to build the computer i did not buy a motherboard + cpu combination that allowed for direct hardware access using vmware as Intel Directed I/O is needed. Alas, one must go around this. In fact going around is cheaper.

Hardware used:
- Asus P5Q3 motherboard
- Intel Quad Core Q9550 cpu
- 8gb DDR3
- 4x 1TB HDD
- 1x 80gb old HDD (original plan was to use 16gb USB pen, but was unable to create a datastore on a USB pen.)
- Power supply, silent box, fanless cpu cooler.

The plan:

1) Install VMWARE 4.0 ESXi on the bare metal.

1.1) Install VMWARE 4.0 ESXi to the 80 GB HDD

2) Install Open Solaris 2009.06

OpenSolaris is being prefered to linux zfs support because of performance. I have read everywere about how much peformance is lost so we'll stick to linux. However, on the Open Solaris side, there is no paracirtualization support in VMWARE ESXi and as far as i can tell, no support for VMNET3 virtual NIC, so i don't really know wich one is better, but have bet on Open Solaris anyway.

Moving on.

2.1) Create VM to install Open Solaris into
First create a new virtual machine. 1 GB RAM should be enough and i don't see the advantage of more than 1 cpu, performance wise. No need for a large hard drive, too, 5gb should be enough as the only job for OpenSolaris will be to run the ZFS filesystem, and to offer an ISCSI target for ESXi to connect to and eventually offer shares for access to files stored.

2.2) Installation itself
No special requirements for installing. Just next, next, next, answering the several queries should be fairly easy.

2.3) Installing VMWARE Tools on OpenSolaris 2009.06
There are no vmware tools available for Open Solaris. However the package available for Solaris 10 should do fine, with some modifications.

unzip the vmware tools:

gunzip vmware-solaris-tools.tar.gz

untar the vmware tools:

tar -xf vmware-solaris-tools.tar

now, execute the installer: 

./vmware-install.pl

Just press enter on every query.

you will get an error:

Unable to copy the source file
/user/lib/vmware-tools/configurator/XOrg/7.4/vmware_drv.so to the destination
file /usr/X11/lib/modules/drivers/vmware_drv.so

what you need to do is the following:

mkdir /usr/lib/vmware-tools/configurator/XOrg/7.4
cp /usr/lib/vmware-tools/configurator/XOrg/7.1/* /usr/lib/vmware-tools/configurator/XOrg/7.4

then re-execute the installer:

./vmware-install.pl

Again, just press enter on any query.

Now, VMWARE Tools should be installed. This will add improvements to performance.

Next, restart Open Solaris.

Voilá, open solaris is now installed, with VMware Tools loaded. Nice.

=======

补充：安装 vmware tools 时，如上提到的 7.4 中缺少相关文件，从7.6中提取。