Azure disk iops的测试

在Public Cloud中，VM、Storage和Network是IaaS的三大基础。本文将介绍在Azure的VM上测试磁盘IOPS的工具和方法。

一、添加磁盘、初始化磁盘

1.添加磁盘

把相应的信息填写好后，点击OK，加载一块Disk到相应的VM。可以注意到这块Disk的IOPS是500，吞吐量是60MB/s。

2.分区和格式化

加载好后，可以在VM中看到这块Disk：

sda是系统盘，sdb是Azure VM的临时盘，之前加的一块盘是sdc，这块盘是sdd。

查看：

[root@hwcentos ~]# fdisk -cul /dev/sdd
Disk /dev/sdd: 1098.4 GB,  bytes
 heads,  sectors/track,  cylinders, total  sectors
Units = sectors of  *  =  bytes
Sector size (logical/physical):  bytes /  bytes
I/O size (minimum/optimal):  bytes /  bytes
Disk identifier: 0x00000000

磁盘容量1TB。

[root@hwcentos ~]# fdisk -cu /dev/sdd
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xabbaf8a0.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
 
Warning: invalid flag 0x0000 of partition table  will be corrected by w(rite)
 
Command (m for help): p
 
Disk /dev/sdd: 1098.4 GB,  bytes
 heads,  sectors/track,  cylinders, total  sectors
Units = sectors of  *  =  bytes
Sector size (logical/physical):  bytes /  bytes
I/O size (minimum/optimal):  bytes /  bytes
Disk identifier: 0xabbaf8a0
 
Device Boot Start End Blocks Id System
 
Command (m for help): n
Command action
e extended
p primary partition (-)
p
Partition number (-):
First sector (-, default ):
Using default value
Last sector, +sectors or +size{K,M,G} (-, default ):
Using default value
 
Command (m for help): p
 
Disk /dev/sdd: 1098.4 GB,  bytes
 heads,  sectors/track,  cylinders, total  sectors
Units = sectors of  *  =  bytes
Sector size (logical/physical):  bytes /  bytes
I/O size (minimum/optimal):  bytes /  bytes
Disk identifier: 0xabbaf8a0
 
Device Boot Start End Blocks Id System
/dev/sdd1     Linux
Command (m for help): w

3.格式化硬盘

用mkfs.ext4格式化硬盘前，先查找一下帮助文件：

man mkfs.ext4

查找lazy，

lazy_itable_init[= < to disable,  to enable>]
If enabled and the uninit_bg feature is enabled, the inode table will not be fully initialized by mke2fs. This speeds up
filesystem initialization noticeably, but it requires the kernel to finish initializing the filesystem in the background when
the filesystem is first mounted. If the option value is omitted, it defaults to  to enable lazy inode table initialization.

通过这个选项可以快速对磁盘进行格式化。并且在第一次挂载的时候，系统在后台把系统初始化好。

[root@hwcentos ~]# mkfs.ext4 -E lazy_itable_init /dev/sdd1
mke2fs 1.41. (-May-)
Filesystem label=
OS type: Linux
Block size= (log=)
Fragment size= (log=)
Stride= blocks, Stripe width= blocks
 inodes,  blocks
 blocks (5.00%) reserved for the super user
First data block=
Maximum filesystem blocks=
 block groups
 blocks per group,  fragments per group
 inodes per group
Superblock backups stored on blocks:
    , , , , , , , , ,
    , , , , , , ,
    ,
 
Writing inode tables: done
Creating journal ( blocks): done
Writing superblocks and filesystem accounting information:
done
 
This filesystem will be automatically checked every  mounts or
 days, whichever comes first. Use tune2fs -c or -i to override.

1T的硬盘瞬间就格式化好了。

挂载并查看：

[root@hwcentos ~]# mkdir /1T
[root@hwcentos ~]# mount /dev/sdd1 /1T
[root@hwcentos ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 30G .0G 27G % /
tmpfs 841M  841M % /dev/shm
/dev/sdb1 69G 180M 66G % /mnt/resource
/dev/sdd1 1007G 200M 956G % /1T

二、测试工具IOPS

在Github搜索iops，排名第一的是cxcv/iops。

这是一个python的程序。

https://raw.githubusercontent.com/cxcv/iops/master/iops

运行这个程序测试刚刚创建的盘：

[root@hwcentos ~]# ./iops.py /dev/sdd1
/dev/sdd1, 1.10 T, sectorsize=512B, #threads=, pattern=random:
 B blocks: 562.5 IO/s, 288.0 kB/s ( 2.3 Mbit/s)
 kB blocks: 579.3 IO/s, 593.2 kB/s ( 4.7 Mbit/s)
 kB blocks: 288.9 IO/s, 591.6 kB/s ( 4.7 Mbit/s)
 kB blocks: 229.6 IO/s, 940.4 kB/s ( 7.5 Mbit/s)
 kB blocks: 261.9 IO/s, 2.1 MB/s ( 17.2 Mbit/s)
 kB blocks: 175.8 IO/s, 2.9 MB/s ( 23.0 Mbit/s)
 kB blocks: 221.9 IO/s, 7.3 MB/s ( 58.2 Mbit/s)
 kB blocks: 259.7 IO/s, 17.0 MB/s (136.1 Mbit/s)
 kB blocks: 142.9 IO/s, 18.7 MB/s (149.9 Mbit/s)
 kB blocks: 106.1 IO/s, 27.8 MB/s (222.5 Mbit/s)
 kB blocks: 84.1 IO/s, 44.1 MB/s (352.6 Mbit/s)
 MB blocks: 49.6 IO/s, 52.0 MB/s (416.1 Mbit/s)
 MB blocks: 28.5 IO/s, 59.8 MB/s (478.2 Mbit/s)
 MB blocks: 14.8 IO/s, 62.3 MB/s (498.3 Mbit/s)
 MB blocks: 5.7 IO/s, 48.1 MB/s (384.6 Mbit/s)

可以看到IOPS在500左右，吞吐量在60MB/s左右。

三、fio测试磁盘性能

同样在Github上查找fio，axboe/fio就是fio工具。

同样，如果CentOS里设置了epel源，也可以直接yum安装：

[root@hwcentos yum.repos.d]# yum search fio
Loaded plugins: fastestmirror, security
Repository 'epel' is missing name in configuration, using id
Loading mirror speeds from cached hostfile
epel | 4.4 kB :
epel/primary_db | 6.3 MB :
epel/pkgtags | 1.2 MB :
============================================================================ N/S Matched: fio =============================================================================
dpm-rfio-server.x86_64 : DPM RFIO server
gfal2-plugin-rfio.x86_64 : Provide the rfio support for gfal2
fio.x86_64 : Multithreaded IO generation tool
root-io-rfio.x86_64 : Remote File input/output library for ROOT
 
Name and summary matches only, use "search all" for everything.
[root@hwcentos yum.repos.d]# yum install fio -y
Loaded plugins: fastestmirror, security
Repository 'epel' is missing name in configuration, using id
Loading mirror speeds from cached hostfile
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package fio.x86_64 :2.0.-.el6 will be installed
--> Finished Dependency Resolution
 
Dependencies Resolved
 
===========================================================================================================================================================================
Package Arch Version Repository Size
===========================================================================================================================================================================
Installing:
fio x86_64 2.0.-.el6 epel  k
 
Transaction Summary
===========================================================================================================================================================================
Install  Package(s)
 
Total download size:  k
Installed size: 1.1 M
Downloading Packages:
fio-2.0.-.el6.x86_64.rpm |  kB :
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : fio-2.0.-.el6.x86_64 /
Verifying : fio-2.0.-.el6.x86_64 /
 
Installed:
fio.x86_64 :2.0.-.el6
 
Complete!
[root@hwcentos yum.repos.d]# yum info fio
Loaded plugins: fastestmirror, security
Repository 'epel' is missing name in configuration, using id
Loading mirror speeds from cached hostfile
Installed Packages
Name : fio
Arch : x86_64
Version : 2.0.
Release : .el6
Size : 1.1 M
Repo : installed
From repo : epel
Summary : Multithreaded IO generation tool
URL : http://git.kernel.dk/?p=fio.git;a=summary
License : GPLv2
Description : fio is an I/O tool that will spawn a number of threads or processes doing
: a particular type of io action as specified by the user. fio takes a
: number of global parameters, each inherited by the thread unless
: otherwise parameters given to them overriding that setting is given.
: The typical use of fio is to write a job file matching the io load
: one wants to simulate.

Yum安装的版本是2.0.13，在Github上的是2.15版本。

安装好后，配置fio的测试配置文件：

[global]
ioengine=psync
direct=
thread=
norandommap=
randrepeat=
runtime=
ramp_time=
size=1g
directory=/path/to/test
 
[read4k-rand]
stonewall
group_reporting
bs=4k
rw=randread
numjobs=
iodepth=
 
[read64k-seq]
stonewall
group_reporting
bs=64k
rw=read
numjobs=
iodepth=
 
[write4k-rand]
stonewall
group_reporting
bs=4k
rw=randwrite
numjobs=
iodepth=
 
[write64k-seq]
stonewall
group_reporting
bs=64k
rw=write
numjobs=
iodepth=

这个配置文件中有4个测试，分别是顺序读、随机读、顺序写、随机写。同时测试的ioengine采用的是同步模式。

具体配置文件中的参数可以通过man fio查看。

fio fio.conf > sync.txt
grep iops sync.txt

可以看到不同测试场景下的测试结果

[root@hwcentos ~]# grep iops sync.txt
read : io=113516KB, bw=.2KB/s, iops= , runt= 60152msec
read : io=.8MB, bw=29416KB/s, iops= , runt= 60005msec
write: io=109516KB, bw=.2KB/s, iops= , runt= 60004msec
write: io=926336KB, bw=15409KB/s, iops= , runt= 60116msec

在配置文件中把测试的ioengine换成libaio：

[global]
ioengine=libaio

fio fio.conf > async.txt
grep iops async.txt
[root@hwcentos ~]# grep iops async.txt
read : io=119896KB, bw=.7KB/s, iops= , runt= 60503msec
read : io=.5MB, bw=37954KB/s, iops= , runt= 60043msec
write: io=115040KB, bw=.8KB/s, iops= , runt= 60019msec
write: io=.8MB, bw=29377KB/s, iops= , runt= 60155msec

iops有所增加，特别是随机写入增加比较大。