是否开启raid卡缓存的影响

开启raid卡缓存 Write back 对IO性能的影响

---

背景

公司买了一台服务器. 想进行一下升级
但是因为管理员担心数据丢失, 使用了write through + (raid6 + hotspare)
的终极保护模式.
但是发现磁盘的IO比较差, 测试结果也不是非常好.
基于此进行了一下raid卡的设置, 希望能够提高一下性能.

---

调优背景

计划打开raid卡缓存前后进行一下简要的性能测试：
工具为： fio
还有数据库测试软件： sysbench

---

对比结果

IO块大小	是否开启RAID缓存	顺序写入	顺序读取	随机写入	随机读取
128KB	是	10100	5098	910	266
128KB	否	272	11500	117	1103
16KB	是	33200	28300	1902	363
16KB	否	591	22900	137	959
8KB	是	36500	33700	1863	390
8KB	否	837	37800	173	946
1KB	是	39500	36300	837	358
1KB	否	700	33900	115	884

---

数据库部分的差异

是否开启超线程	事务效率(每秒)	查询效率(每秒)	最小响应时间(ms)	平均响应时间	99%的响应时间
直写 32核心/32线程	244.78	5011.98	14.43	40.82	97.55
写回 32核心/32线程	497.56	10012.29	14.19	20.09	27.17

---

图形展示

• FIO

• sysbench

---

部分原始数据

• FIO 写回

write128k
   write: IOPS=10.1k, BW=1258MiB/s (1319MB/s)(4096MiB/3257msec)
read128k
   read: IOPS=5098, BW=637MiB/s (668MB/s)(4096MiB/6427msec)
randwrite128k
  write: IOPS=910, BW=114MiB/s (119MB/s)(3420MiB/30065msec)
randread128k
   read: IOPS=266, BW=33.4MiB/s (34.0MB/s)(1001MiB/30006msec)
write16k
  write: IOPS=33.2k, BW=519MiB/s (544MB/s)(4096MiB/7896msec)
read16k
   read: IOPS=28.3k, BW=442MiB/s (464MB/s)(4096MiB/9265msec)
randwrite16k
  write: IOPS=1902, BW=29.7MiB/s (31.2MB/s)(893MiB/30026msec)
randread16k
   read: IOPS=363, BW=5814KiB/s (5953kB/s)(170MiB/30003msec)
write8k
  write: IOPS=36.5k, BW=285MiB/s (299MB/s)(4096MiB/14369msec)
read8k
   read: IOPS=33.7k, BW=264MiB/s (276MB/s)(4096MiB/15542msec)
randwrite8k
  write: IOPS=1863, BW=14.6MiB/s (15.3MB/s)(437MiB/30001msec)
randread8k
   read: IOPS=390, BW=3128KiB/s (3203kB/s)(91.7MiB/30006msec)
write1k
  write: IOPS=39.5k, BW=38.5MiB/s (40.4MB/s)(1156MiB/30001msec)
read1k
   read: IOPS=36.3k, BW=35.5MiB/s (37.2MB/s)(1064MiB/30001msec)
randwrite1k
  write: IOPS=837, BW=838KiB/s (858kB/s)(24.5MiB/30013msec)
randread1k
   read: IOPS=358, BW=358KiB/s (367kB/s)(10.5MiB/30002msec)

---

• FIO 直写

write128k
   write: IOPS=272, BW=34.1MiB/s (35.8MB/s)(1023MiB/30002msec)
read128k
   read: IOPS=11.5k, BW=1442MiB/s (1512MB/s)(4096MiB/2841msec)
randwrite128k
  write: IOPS=117, BW=14.7MiB/s (15.4MB/s)(441MiB/30008msec)
randread128k
   read: IOPS=1103, BW=138MiB/s (145MB/s)(4096MiB/29697msec)
write16k
  write: IOPS=591, BW=9456KiB/s (9683kB/s)(277MiB/30005msec)
read16k
   read: IOPS=22.9k, BW=358MiB/s (375MB/s)(4096MiB/11442msec)
randwrite16k
  write: IOPS=137, BW=2202KiB/s (2255kB/s)(64.5MiB/30013msec)
randread16k
   read: IOPS=959, BW=14.0MiB/s (15.7MB/s)(450MiB/30011msec)
write8k
  write: IOPS=837, BW=6700KiB/s (6861kB/s)(196MiB/30001msec)
read8k
   read: IOPS=37.8k, BW=295MiB/s (310MB/s)(4096MiB/13862msec)
randwrite8k
  write: IOPS=143, BW=1144KiB/s (1172kB/s)(33.5MiB/30001msec)
randread8k
   read: IOPS=946, BW=7569KiB/s (7750kB/s)(222MiB/30001msec)
write1k
  write: IOPS=700, BW=700KiB/s (717kB/s)(20.5MiB/30003msec)
read1k
   read: IOPS=33.9k, BW=33.1MiB/s (34.8MB/s)(995MiB/30001msec)
randwrite1k
  write: IOPS=115, BW=116KiB/s (118kB/s)(3471KiB/30005msec)
randread1k
   read: IOPS=884, BW=885KiB/s (906kB/s)(25.9MiB/30005msec)

---

sysbench的结果

• 写回

[ 10s ] thds: 10 tps: 493.60 qps: 9952.25 (r/w/o: 6974.03/1975.31/1002.90) lat (ms,99%): 34.33 err/s: 3.70 reconn/s: 0.00
[ 20s ] thds: 10 tps: 451.21 qps: 9067.76 (r/w/o: 6349.91/1804.43/913.42) lat (ms,99%): 26.68 err/s: 2.80 reconn/s: 0.00
[ 30s ] thds: 10 tps: 547.70 qps: 11024.03 (r/w/o: 7723.12/2189.41/1111.50) lat (ms,99%): 25.28 err/s: 3.70 reconn/s: 0.00
SQL statistics:
    queries performed:
        read:                            210532
        write:                           59728
        other:                           30291
        total:                           300551
    transactions:                        14936  (497.56 per sec.)
    queries:                             300551 (10012.29 per sec.)
    ignored errors:                      102    (3.40 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          30.0166s
    total number of events:              14936

Latency (ms):
         min:                                   14.19
         avg:                                   20.09
         max:                                 1041.03
         99th percentile:                       27.17
         sum:                               300053.39

Threads fairness:
    events (avg/stddev):           1493.6000/82.27
    execution time (avg/stddev):   30.0053/0.01

---

• 直写

[ 10s ] thds: 10 tps: 243.35 qps: 4991.08 (r/w/o: 3504.25/972.02/514.80) lat (ms,99%): 82.96 err/s: 6.10 reconn/s: 0.00
[ 20s ] thds: 10 tps: 288.50 qps: 5892.27 (r/w/o: 4134.45/1152.11/605.71) lat (ms,99%): 68.05 err/s: 6.90 reconn/s: 0.00
[ 30s ] thds: 10 tps: 202.30 qps: 4163.91 (r/w/o: 2923.90/814.10/425.90) lat (ms,99%): 893.56 err/s: 6.50 reconn/s: 0.00
SQL statistics:
    queries performed:
        read:                            105658
        write:                           29400
        other:                           15475
        total:                           150533
    transactions:                        7352   (244.78 per sec.)
    queries:                             150533 (5011.98 per sec.)
    ignored errors:                      195    (6.49 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          30.0331s
    total number of events:              7352

Latency (ms):
         min:                                   14.43
         avg:                                   40.82
         max:                                 2938.61
         99th percentile:                       97.55
         sum:                               300120.33

Threads fairness:
    events (avg/stddev):           735.2000/39.34
    execution time (avg/stddev):   30.0120/0.01