原文地址:http://antirez.com/news/52

Hello! As promised today I did some SSD testing.

The setup: a Linux box with 24 GB of RAM, with two disks.

A) A spinning disk.
b) An SSD (Intel 320 series). The idea is, what happens if I set the SSD disk partition as a swap partition and fill Redis with a dataset larger than RAM?
It is a lot of time I want to do this test, especially now that Redis focus is only on RAM and I abandoned the idea of targeting disk for a number of reasons. I already guessed that the SSD swap setup would perform in a bad way, but I was not expecting it was *so bad*. Before testing this setup, let's start testing Redis in memory with in the same box with a 10 GB data set. IN MEMORY TEST
=== To start I filled the instance with: ./redis-benchmark -r 1000000000 -n 1000000000 -P 32 set key:rand:000000000000 foo Write load in this way is very high, more than half million SET commands processed per second using a single core: instantaneous_ops_per_sec:629782 This is possible because we using a pipeline of 32 commands per time (see -P 32), so it is possible to limit the number of sys calls involved in the processing of commands, and the network latency component as well. After a few minutes I reached 10 GB of memory used by Redis, so I tried to save the DB while still sending the same write load to the server to see what the additional memory usage due to copy on write would be in such a stress conditions: [31930] 07 Mar 12:06:48.682 * RDB: 6991 MB of memory used by copy-on-write almost 7GB of additional memory used, that is 70% more memory.
Note that this is an interesting value since it is exactly the worst case scenario you can get with Redis: 1) Peak load of more than 0.6 million writes per second.
2) Writes are completely distributed across the data set, there is no working set in this case, all the DB is the working set. But given the enormous pressure on copy on write exercised by this workload, what is the write performance in this case while the system is saving? To find the value I started a BGSAVE and at the same time started the benchmark again: $ redis-cli bgsave; ./redis-benchmark -r 1000000000 -n 1000000000 -P 32 set key:rand:000000000000 foo
Background saving started
^Ct key:rand:000000000000 foo: 251470.34 250k ops/sec was the lower number I was able to get, as once copy on write starts to happen, there is less and less copy on write happening every second, and the benchmark soon returns to 0.6 million ops per second.
The number of keys was in the order of 100 million here. Basically the result of this test is, with real hardware and persisting to a normal spinning disk, Redis performs very well as long as you have enough RAM for your data, and for the additional memory used while saving. No big news so far. SSD SWAP TEST
=== For the SSD test we still use the spinning disk attached to the system in order to persist, so that the SSD is just working as a swap partition. To fill the instance even more I just started again redis-benchmark with the same command line, since with the specific parameters, if running forever, it would set 1 billion keys, that's enough :-) Since the instance has 24 GB of physical RAM, for the test to be meaningful I wanted to add enough data to reach 50 GB of used memory. In order to speedup the process of filling the instance I disabled persistence for some time using: CONFIG SET SAVE "" While filling the instance, at some point I started a BGSAVE to force some more swapping.
Then when the BGSAVE finished, I started the benchmark again: $ ./redis-benchmark -r 1000000000 -n 1000000000 -P 32 set key:rand:000000000000 foo
^Ct key:rand:000000000000 foo: 1034.16 As you can see the results were very bad initially, probably the main hash table ended swapped. After some time it started to perform in a decent way again: $ ./redis-benchmark -r 1000000000 -n 1000000000 -P 32 set key:rand:000000000000 foo
^Ct key:rand:000000000000 foo: 116057.11 I was able to stop and restart the benchmark multiple times and still get decent performances on restarts, as long I was not saving at the same time. However performances continued to be very erratic, jumping from 200k to 50k sets per second. …. and after 10 minutes … It only went from 23 GB of memory used to 24 GB, with 2 GB of data set swapped on disk. As soon as it started to have a few GB swapped performances started to be simply too poor to be acceptable. I then tried with reads: $ ./redis-benchmark -r 1000000000 -n 1000000000 -P 32 get key:rand:000000000000
^Ct key:rand:000000000000 foo: 28934.12 Same issue, 30k ops per second both for GET and SET, and *a lot* of swap activity at the same time.
What's worse is that the system was pretty unresponsive as a whole at this point. At this point I stopped the test, the system was slow enough that filling it even more would require a lot of time, and as more data was swapped performances started to get worse. WHAT HAPPENS?
=== What happens is simple, Redis is designed to work in an environment where random access of memory is very fast.
Hash tables, and the way Redis objects are allocated is all based on this concept. Now let's give a look at the SSD 320 disk specifications: Random write (100% Span) -> 400 IOPS
Random write (8GB Span) -> 23000 IOPS Basically what happens is that at some point Redis starts to force the OS to move memory pages between RAM and swap at *every* operation performed, since we are accessed keys at random, and there are no more spare pages. CONCLUSION
=== Redis is completely useless in this way. Systems designed to work in this kind of setups like Twitter fatcache or the recently announced Facebook McDipper need to be SSD-aware, and can probably work reasonably only when a simple GET/SET/DEL model is used. I also expect that the pathological case for this systems, that is evenly distributed writes with big span, is not going to be excellent because of current SSD disk limits, but that's exactly the case Redis is trying to solve for most users. The freedom Redis gets from the use of memory allows us to serve much more complex tasks at very good peak performance and with minimal system complexity and underlying assumptions. TL;DR: the outcome of this test was expected and Redis is an in-memory system :-)

SSD卡对redis的影响的更多相关文章

  1. SSD卡对mongodb的影响

    结论 1:SSD卡显著改善磁盘IO,io占用在50%以下 2:SSD卡使mongodb性能稳定.在200并发,数据量是内存5倍的情况下仍然保证每秒1500次插入和4500次查询.     数据如下: ...

  2. win10 ssd 卡顿

    http://www.pconline.com.cn/win10/739/7395324.html

  3. 你知道CPU结构也会影响Redis性能吗?

    啦啦啦,我是卖身不卖艺的二哈,ε=(´ο`*)))唉错啦(我是开车的二哈),我又来了,铁子们一起开车呀! 今天来分析下CPU结构对Redis性能会有影响吗? 在进行Redis性能分析的时候,通常我们会 ...

  4. 深度评测丨 GaussDB(for Redis) 大 Key 操作的影响

    本文分享自华为云社区<墨天轮评测:GaussDB(for Redis)大Key操作的影响>,作者: 高斯 Redis 官方博客. 在前一篇文章<墨天轮评测:GaussDB(for R ...

  5. [转]细说Redis监控和告警

    原文  https://zhuoroger.github.io/2016/08/20/redis-monitor-and-alarm/? 对于任何应用服务和组件,都需要一套完善可靠谱监控方案. 尤其r ...

  6. mac下的改装人生——关于ssd

    这两天研究了很多关于ssd的东西,想想还是写下来把,毕竟花了这么多时间进去. 先说一下我自己的电脑把.前几天,因为嫌我的电脑是在是太卡了,准备来次升级,然后先买了个8g的内存装上,发现的确是没有死机的 ...

  7. 【转】花开正当时,十四款120/128GB SSD横向评测

    原文地址:http://www.expreview.com/19604-all.html SSD横评是最具消费指导意义的评测文章,也是各类热门SSD固态硬盘的决斗疆场.SSD评测在行业内已经有不少网站 ...

  8. Redis计算地理位置距离-GeoHash

    Redis 在 3.2 版本以后增加了地理位置 GEO 模块,意味着我们可以使用 Redis 来实现摩拜单车「附近的 Mobike」.美团和饿了么「附近的餐馆」这样的功能了. 地图元素的位置数据使用二 ...

  9. [转]SSD固态存储大观(一)

    From: http://blog.51cto.com/alanwu/1405874 Contents 1.概述... 1 2.FusionIO:Pcie SSD的始作俑者... 2 3.Intel ...

随机推荐

  1. Springboot实现filter拦截token验证和跨域

    背景 web验证授权合法的一般分为下面几种 使用session作为验证合法用户访问的验证方式 使用自己实现的token 使用OCA标准 在使用API接口授权验证时,token是自定义的方式实现起来不需 ...

  2. mysql 不区分大小写的解决

    mysql查询默认是不区分大小写的 如: select * from some_table where str=‘abc';select * from some_table where str='AB ...

  3. Storm(三)Storm的原理机制

    一.Storm的数据分发策略 1. Shuffle Grouping 随机分组,随机派发stream里面的tuple,保证每个bolt task接收到的tuple数目大致相同. 轮询,平均分配 2. ...

  4. jenkins X实践系列(1) —— 背景知识

    本文介绍jenkins X(以下简称jx)相关的背景技术. jenkins X 简介 Jenkins X 是一个高度集成化的CI/CD平台,基于Jenkins和Kubernetes实现,旨在解决微服务 ...

  5. Color the ball HDU1556

    这题整整debug了两个小时 不同组居然要初始化  本以为built函数里面已经初始化好了!!!!! 其他无需注意 #include<cstdio> #include<cstring ...

  6. [OpenCV-Python] OpenCV 中的 Gui特性 部分 II

    部分 IIOpenCV 中的 Gui 特性 OpenCV-Python 中文教程(搬运)目录 4 图片 目标 • 在这里你将学会怎样读入一幅图像,怎样显示一幅图像,以及如何保存一幅图像 • 你将要学习 ...

  7. 对扫描的pdf文件生成目录

    很多pdf文件是直接扫描生成的,于是它的内容都是一张张的图片,当然就更没有目录索引了. 有的时候想找某些内容,只能一点点的移动滚动条,非常不方便. 那么有什么方法能生成目录呢? 方法一:使用福昕pdf ...

  8. java enum的一种写法记录

    public enum TestEnum { provider { @Override public void provide() { this.name = "hjzgg"; } ...

  9. vue 之 加载 iframe 的处理

    vue中加载 iframe  会出现跨域问题.以及iframe的高度自适应问题,以下是本人的解决办法: getGoodsContentHtml---- 你的iframe页面的地址,  如不同域的情况下 ...

  10. AGC 016B.Colorful Hats(思路)

    题目链接 对于某个元素分类讨论一下,就可以知道n个元素的总颜色种数了. 比如对a[1]分类讨论: 若1的颜色和某个元素相同,则总颜色数为a[1].a[i]要么等于a[1](i与某个元素颜色相同,记个数 ...