mfs 使用心得

CentOS的安装方法：

To install MooseFS from officially supported repository on EL7, follow the steps below:

Add the key:

# curl "http://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS

Add an appropriate repository entry:

# curl "http://ppa.moosefs.com/MooseFS-3-el7.repo" > /etc/yum.repos.d/MooseFS.repo

then install appropriate MooseFS components:

For Master Servers:

# yum install moosefs-master moosefs-cgi moosefs-cgiserv moosefs-cli
For Chunkservers:
# yum install moosefs-chunkserver
For Metaloggers:
# yum install moosefs-metalogger
For Clients:
# yum install moosefs-client

Note: If you like to install MooseFS on EL6 distributions, just change MooseFS-3-el7.repo in repository URL to MooseFS-3-el6.repo

元数据服务器 Master server 安装

需要下载的源码包及依赖包：
moosefs-3.0.97.tar.gz
libpcap-1.4.0-1.el6.rft.x86_64.rpm
libpcap-devel-1.4.0-1.el6.rft.x86_64.rpm

yum install -y gcc make rpm-build fuse-devel zlib-devel

yum install -y libpcap-1.4.0-1.el6.rft.x86_64.rpm libpcap-devel-1.4.0-1.el6.rft.x86_64.rpm

rpmbuild -tb moosefs-3.0.97.tar.gz   #将源码包编译成rpm包

cd /root/rpmbuild/RPMS/x86_64   #便以后rpm包存放地址

rpm -ivh moosefs-cgi-3.0.97-1.x86_64.rpm moosefs-cgiserv-3.0.97-1.x86_64.rpm moosefs-cli-3.0.97-1.x86_64.rpm moosefs-master-3.0.97-1.x86_64.rpm

vim /etc/hosts    #修改/etc/hosts地址解析文件

172.25.21.1 mfsmaster server1

mfsmaster start   #启动master server

mfscgiserv   #启动CGI监控服务

netstat -anlpt    #可以看到9419，9420，9421，9425 端口

Active Internet connections (servers and established)

Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name

tcp        0      0 0.0.0.0:9419                0.0.0.0:*                   LISTEN      1186/mfsmaster

tcp        0      0 0.0.0.0:9420                0.0.0.0:*                   LISTEN      1186/mfsmaster

tcp        0      0 0.0.0.0:9421                0.0.0.0:*                   LISTEN      1186/mfsmaster

tcp        0      0 0.0.0.0:9425                0.0.0.0:*                   LISTEN      1189/python

tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      921/sshd

tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN      999/master

tcp        0      0 172.25.21.1:22              172.25.21.250:33250         ESTABLISHED 1051/sshd

tcp        0      0 :::22                       :::*                        LISTEN      921/sshd

tcp        0      0 ::1:25                      :::*                        LISTEN      999/master

在浏览器地址栏输入http://172.25.21.1:9425即可查看 master的运行情况

同样的server2也可以访问它的9425端口

存储服务器 Chunk servers 安装

yum install -y moosefs-chunkserver-3.0.97-1.x86_64.rpm

vim /etc/hosts    #修改/etc/hosts地址解析文件

172.25.21.1 mfsmaster server1

mkdir /mnt/chunk1

chown mfs.mfs /mnt/chunk1/

vim /etc/mfs/mfshdd.cfg

/mnt/chunk1

每个Chunk servers 端创建的目录不同，要做相应修改

mfschunkserver start

现在再通过浏览器访问 http://172.25.21.1:9425/ 可以看见这个 MFS系统的信息

客户端 client安装

yum install -y moosefs-client-3.0.97-1.x86_64.rpm

mkdir /mnt/mfs   #默认挂载点

mfsmount /mnt/mfs/

查看

df -h

每台主机都为20G，可以看到两台Chunk servers的存储都挂载了过来

挂载命令安装好客户端软件moosefs-client

然后使用

mkdir /mnt/data

mfsmount /mnt/data -H 172.20.10.8

简单测试结果：

写：time dd if=/dev/zero of=/usr/mfstest/test2/zhhtest500M bs=1024k count=500

读：time dd if=/usr/mfstest/test2/zhhtest500M of=/dev/null

	1copy写	2copy写	1copy读	2copy读
1M	0m0.042s	0m0.042s	0m0.017s	0m0.017s
2M	0m0.056s	0m0.066s	0m0.030s	0m0.055s
5M	0m0.073s	0m0.079s	0m0.070s	0m0.071s
10M	0m0.119s	0m0.131s	0m0.136s	0m0.197s
20M	0m0.250s	0m0.288s	0m0.291s	0m0.376s
50M	0m0.514s	0m0.589s	0m0.896s	0m0.886s
100M	0m0.977s	0m7.497s	0m1.677s	0m1.787s
200M	0m7.910s	0m22.270s	0m2.683s	0m3.129s
500M	0m22.465s	0m51.735s	0m6.559s	0m6.990s
1G	0m52.346s	1m48.056s	0m17.319s	0m17.154s
2G	1m46.224s	3m46.283s	0m41.608s	0m34.435s
5G	4m41.956s	9m34.798s	2m9.200s	1m56.183s
10G	9m29.037s	19m26.237s	3m55.222s	3m24.914s
100G	95m53.500s	195m7.173s	32m4.295s	41m26.995s

总结规律：

1、读速度：ca 71M/s 写速度：ca 22M/s 9M/s （以500M计算）

2、goal的设置和读写速度的关系？