Redislabs的简单使用与benchmark测试结果

自己的理解

  • 仅是理解. 没有看过源码
哨兵可以实现高可用, 但是对高吞吐是没有太大帮助的.

虽然可以实现主写从读, 但是在高并发的场景下延迟肯定会增加.

集群模式可以实现hash分片, 虽有有了一个根据key值进行hash的动作

增加了延时, 但是也增加系统容量, 在降低一定响应时间的情况下提高了系统容量.

能够扩展出更大的吞吐量. 降低redis单节点连接数过多的问题.

背景

周六一早和晚上安装了下两个节点的Redislabs

睡觉前简单整理了下redislabs 在openEuler上面的安装.

今天白天继续进行redislabs的相关工作处理

第一步 是让产品能够连的上redislabs 至少可以正常启动

第二步 可以跑一下一些基本的benchmark的方式验证一下性能

第三步 是需要进行功能和性能测试进行简单验证.

第三步我自己做不了,但是前两步应该还行. 然后基于此水这一篇文章.

第一部分: Spingboot连接redis

springboot 连接redis有很多种方式

我们产品复杂, 使用了jedis和redisson两种连接池.

所以需要在两个地方进行设置.

# 不过我感觉太过复杂并不是好事情. 容易产生误解. 

jedis可以直接写. 在配置文件中:

caching-configuration:

  enableRedis: true

  redisManagers:

  - name: default

    mode: cluster

    nodes: 10.110.139.191:14085,10.110.139.192:14085

    max-redirects: 8

    password: Yourpassword?!

# 注意 mode 是 cluster

# 注意 不是host 是 ndoes

# 注意 没有port了 是在nodes 里面直接添加上了.

第一部分: Spingboot连接redis

redisson的稍微复杂一点,需要单独处理一个文档.

yaml 文件里面需要修改一下配置:

yaml 内部的配置:

  redis:

    redisson:

      config: classpath:redisson.yaml

然后在runtime/libs 目录下面增加

redisson.yaml

第一部分: Spingboot连接redis

  • redisson.yaml 的相关配置为:
clusterServersConfig:

  # 连接空闲超时，单位：毫秒 默认10000

  idleConnectionTimeout: 10000

  pingTimeout: 1000

  # 同任何节点建立连接时的等待超时。时间单位是毫秒 默认10000

  connectTimeout: 10000

  # 等待节点回复命令的时间。该时间从命令发送成功时开始计时。默认3000

  timeout: 3000

  # 命令失败重试次数

  retryAttempts: 3

  # 命令重试发送时间间隔，单位：毫秒

  retryInterval: 6000

  # 重新连接时间间隔，单位：毫秒

  reconnectionTimeout: 6000

  # 执行失败最大次数

  failedAttempts: 3

  # 密码

  password: Yourpasswords?!

  # 单个连接最大订阅数量

  subscriptionsPerConnection: 5

  clientName: null

  # loadBalancer 负载均衡算法类的选择

  loadBalancer: !<org.redisson.connection.balancer.RoundRobinLoadBalancer> {}

  #从节点发布和订阅连接的最小空闲连接数

  slaveSubscriptionConnectionMinimumIdleSize: 1

  #从节点发布和订阅连接池大小 默认值50

  slaveSubscriptionConnectionPoolSize: 50

  # 从节点最小空闲连接数 默认值32

  slaveConnectionMinimumIdleSize: 32

  # 从节点连接池大小 默认64

  # 感觉需要同步修改

  slaveConnectionPoolSize: 512

  # 主节点最小空闲连接数 默认32

  masterConnectionMinimumIdleSize: 32

  # 主节点连接池大小 默认64

  # 赵本帅增加默认值

  masterConnectionPoolSize: 512

  # 订阅操作的负载均衡模式

  subscriptionMode: SLAVE

  # 只在从服务器读取

  readMode: SLAVE

  # 集群地址

  nodeAddresses:

    - "redis://10.110.139.191:14085"

    - "redis://10.110.139.192:14085"

  # 对Redis集群节点状态扫描的时间间隔。单位是毫秒。默认1000

  scanInterval: 1000

#这个线程池数量被所有RTopic对象监听器，RRemoteService调用者和RExecutorService任务共同共享。默认2

threads: 400

#这个线程池数量是在一个Redisson实例内，被其创建的所有分布式数据类型和服务，以及底层客户端所一同共享的线程池里保存的线程数量。默认2

nettyThreads: 400

# 编码方式 默认org.redisson.codec.JsonJacksonCodec

codec: !<org.redisson.codec.JsonJacksonCodec> {}

#传输模式

transportMode: NIO

# 分布式锁自动过期时间，防止死锁，默认30000

lockWatchdogTimeout: 30000

# 通过该参数来修改是否按订阅发布消息的接收顺序出来消息，如果选否将对消息实行并行处理，该参数只适用于订阅发布消息的情况, 默认true

keepPubSubOrder: true

第一部分: Spingboot连接redis

配置一定准确, 不要有错别字..

配置完成后应该就可以启动了

还可以通过 redislabs 的 8443

端口进行查询性能

第二部分: 性能验证

  • 先说结论部分
1. redislabs的吞吐量跟同配置的开源版本并无大的提升.

   集群模式下的损耗和shard导致吞吐量有下降.

   但是企业版的着重点不在于吞吐量 而在于其他稳定性和监控等.

2. 多线程能够大幅提高吞吐量, 并且能够规避掉仅能够使用一个CPU核心的问题

   今天测试了四线程的操作. PING命令的性能提升了接近3倍.MSET的性能提升2倍+

   认为有较大的优化空间.

3. OpenEuler2203的5.10的内存的同配置机器的性能表现要优于CentOS7 3.10内核的虚拟机.

   5.10内核的机器 最高的吞吐量能到10万. 但是 3.10内核的基本上只有7万左右.

   差距比较明显主要说的是 MSET (10 keys) 的性能数据.

4. Redis5.0.4的性能明显比6.2.6的性能表现要好. 原因不明.

第二部分: 性能测试

测试方法 redis-benchmark -h -p -a 导出数据

转储到具体文件 ,然后执行如下语句进行分析

cat xxxx .log |grep -E "requests per second|====="

具体结果可以做成excel  但是大周末的 我只放原始数据算了..

既然是水 就要水到底...

第三部分: 性能测试原始数据

机器配置信息大部分为:

Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz

CPU 为8C-16C

内存为 48-64G的虚拟机

磁盘为 HDD RAID6 12块 4T的盘

虚拟机操作系统分别为:

OpenEuler 2203 5.10

CentOS7 2009   3.10

第三部分: 性能测试原始数据

这里准备将元数据放出.

不进行完整的对比了

章节名字就是测试的场景.

redis 5.0.4 单线程 OpenEuler without swap

====== PING_INLINE ======

108813.92 requests per second

====== PING_BULK ======

104931.80 requests per second

====== SET ======

107874.87 requests per second

====== GET ======

105932.20 requests per second

====== INCR ======

108342.37 requests per second

====== LPUSH ======

107642.62 requests per second

====== RPUSH ======

108225.10 requests per second

====== LPOP ======

108342.37 requests per second

====== RPOP ======

105708.25 requests per second

====== SADD ======

105932.20 requests per second

====== HSET ======

109170.30 requests per second

====== SPOP ======

105708.25 requests per second

====== LPUSH (needed to benchmark LRANGE) ======

109170.30 requests per second

====== LRANGE_100 (first 100 elements) ======

58582.31 requests per second

====== LRANGE_300 (first 300 elements) ======

23110.70 requests per second

====== LRANGE_500 (first 450 elements) ======

16641.71 requests per second

====== LRANGE_600 (first 600 elements) ======

12479.72 requests per second

====== MSET (10 keys) ======

110864.74 requests per second

6.2.6 OpenEuler with SWAP

====== PING_INLINE ======

107066.38 requests per second

====== PING_BULK ======

97560.98 requests per second

====== SET ======

106609.80 requests per second

====== GET ======

108459.87 requests per second

====== INCR ======

108342.37 requests per second

====== LPUSH ======

107296.14 requests per second

====== RPUSH ======

107411.38 requests per second

====== LPOP ======

110375.27 requests per second

====== RPOP ======

109649.12 requests per second

====== SADD ======

106382.98 requests per second

====== HSET ======

103092.78 requests per second

====== SPOP ======

105152.48 requests per second

====== LPUSH (needed to benchmark LRANGE) ======

107642.62 requests per second

====== LRANGE_100 (first 100 elements) ======

51948.05 requests per second

====== LRANGE_300 (first 300 elements) ======

22691.17 requests per second

====== LRANGE_500 (first 450 elements) ======

17111.57 requests per second

====== LRANGE_600 (first 600 elements) ======

12597.63 requests per second

====== MSET (10 keys) ======

107066.38 requests per second

redis 6.2.6 OpenEuler 单线程 without Swap

====== PING_INLINE ======

104493.20 requests per second

====== PING_BULK ======

100200.40 requests per second

====== SET ======

105152.48 requests per second

====== GET ======

108108.11 requests per second

====== INCR ======

104058.27 requests per second

====== LPUSH ======

110253.59 requests per second

====== RPUSH ======

106269.93 requests per second

====== LPOP ======

110497.24 requests per second

====== RPOP ======

106496.27 requests per second

====== SADD ======

106496.27 requests per second

====== HSET ======

108695.65 requests per second

====== SPOP ======

106382.98 requests per second

====== LPUSH (needed to benchmark LRANGE) ======

109289.62 requests per second

====== LRANGE_100 (first 100 elements) ======

48875.86 requests per second

====== LRANGE_300 (first 300 elements) ======

21910.60 requests per second

====== LRANGE_500 (first 450 elements) ======

16450.07 requests per second

====== LRANGE_600 (first 600 elements) ======

12280.49 requests per second

====== MSET (10 keys) ======

88417.33 requests per second

redislabs 企业版 OpenEuler 主节点测试主节点

====== PING_INLINE ======

67659.00 requests per second

====== PING_BULK ======

67980.97 requests per second

====== SET ======

63572.79 requests per second

====== GET ======

62814.07 requests per second

====== INCR ======

63331.22 requests per second

====== LPUSH ======

67385.45 requests per second

====== RPUSH ======

67842.61 requests per second

====== LPOP ======

68446.27 requests per second

====== RPOP ======

67704.80 requests per second

====== SADD ======

67704.80 requests per second

====== HSET ======

68073.52 requests per second

====== SPOP ======

67796.61 requests per second

====== LPUSH (needed to benchmark LRANGE) ======

67842.61 requests per second

====== LRANGE_100 (first 100 elements) ======

43497.17 requests per second

====== LRANGE_300 (first 300 elements) ======

18559.76 requests per second

====== LRANGE_500 (first 450 elements) ======

13627.69 requests per second

====== LRANGE_600 (first 600 elements) ======

10636.03 requests per second

====== MSET (10 keys) ======

64641.24 requests per second

Redislabs OpenEuler 从节点测试主节点

====== PING_INLINE ======

70921.98 requests per second

====== PING_BULK ======

75357.95 requests per second

====== SET ======

83056.48 requests per second

====== GET ======

80321.28 requests per second

====== INCR ======

83056.48 requests per second

====== LPUSH ======

80321.28 requests per second

====== RPUSH ======

80710.25 requests per second

====== LPOP ======

76863.95 requests per second

====== RPOP ======

77821.02 requests per second

====== SADD ======

77519.38 requests per second

====== HSET ======

78125.00 requests per second

====== SPOP ======

74794.31 requests per second

====== LPUSH (needed to benchmark LRANGE) ======

79113.92 requests per second

====== LRANGE_100 (first 100 elements) ======

45578.85 requests per second

====== LRANGE_300 (first 300 elements) ======

18037.52 requests per second

====== LRANGE_500 (first 450 elements) ======

13059.95 requests per second

====== LRANGE_600 (first 600 elements) ======

10339.12 requests per second

====== MSET (10 keys) ======

77220.08 requests per second

redislabs OpenEuler 从节点测试从节点

====== PING_INLINE ======

67024.13 requests per second

====== PING_BULK ======

66225.16 requests per second

====== SET ======

65530.80 requests per second

====== GET ======

68212.83 requests per second

====== INCR ======

67934.78 requests per second

====== LPUSH ======

63572.79 requests per second

====== RPUSH ======

62539.09 requests per second

====== LPOP ======

63091.48 requests per second

====== RPOP ======

61881.19 requests per second

====== SADD ======

62893.08 requests per second

====== HSET ======

62344.14 requests per second

====== SPOP ======

63091.48 requests per second

====== LPUSH (needed to benchmark LRANGE) ======

61996.28 requests per second

====== LRANGE_100 (first 100 elements) ======

63051.70 requests per second

====== LRANGE_300 (first 300 elements) ======

62893.08 requests per second

====== LRANGE_500 (first 450 elements) ======

63331.22 requests per second

====== LRANGE_600 (first 600 elements) ======

62617.41 requests per second

====== MSET (10 keys) ======

68634.18 requests per second

6.2.6 centos7 3.10 保护模式 单节点

====== PING_INLINE ======

71073.21 requests per second

====== PING_BULK ======

69979.01 requests per second

====== SET ======

69013.11 requests per second

====== GET ======

71225.07 requests per second

====== INCR ======

72411.30 requests per second

====== LPUSH ======

74019.25 requests per second

====== RPUSH ======

74019.25 requests per second

====== LPOP ======

74183.98 requests per second

====== RPOP ======

74074.07 requests per second

====== SADD ======

71428.57 requests per second

====== HSET ======

73260.07 requests per second

====== SPOP ======

73475.39 requests per second

====== LPUSH (needed to benchmark LRANGE) ======

74074.07 requests per second

====== LRANGE_100 (first 100 elements) ======

43802.02 requests per second

====== LRANGE_300 (first 300 elements) ======

19992.00 requests per second

====== LRANGE_500 (first 450 elements) ======

14723.20 requests per second

====== LRANGE_600 (first 600 elements) ======

11544.68 requests per second

====== MSET (10 keys) ======

71942.45 requests per second

6.2.6 centos7 3.10 非保护模式

====== PING_INLINE ======

71787.51 requests per second

====== PING_BULK ======

71073.21 requests per second

====== SET ======

71994.23 requests per second

====== GET ======

72150.07 requests per second

====== INCR ======

72202.16 requests per second

====== LPUSH ======

72098.05 requests per second

====== RPUSH ======

73099.41 requests per second

====== LPOP ======

73367.57 requests per second

====== RPOP ======

73313.78 requests per second

====== SADD ======

72516.32 requests per second

====== HSET ======

72992.70 requests per second

====== SPOP ======

71994.23 requests per second

====== LPUSH (needed to benchmark LRANGE) ======

73529.41 requests per second

====== LRANGE_100 (first 100 elements) ======

43591.98 requests per second

====== LRANGE_300 (first 300 elements) ======

20036.06 requests per second

====== LRANGE_500 (first 450 elements) ======

14795.09 requests per second

====== LRANGE_600 (first 600 elements) ======

11498.22 requests per second

====== MSET (10 keys) ======

75815.01 requests per second

redis 6.2.6CentOS7 with out password without protected CentOS7 3.10

====== PING_INLINE ======

72621.64 requests per second

====== PING_BULK ======

72833.21 requests per second

====== SET ======

73046.02 requests per second

====== GET ======

72411.30 requests per second

====== INCR ======

74019.25 requests per second

====== LPUSH ======

73421.44 requests per second

====== RPUSH ======

73313.78 requests per second

====== LPOP ======

74294.21 requests per second

====== RPOP ======

73800.73 requests per second

====== SADD ======

72886.30 requests per second

====== HSET ======

72780.20 requests per second

====== SPOP ======

72150.07 requests per second

====== LPUSH (needed to benchmark LRANGE) ======

73909.83 requests per second

====== LRANGE_100 (first 100 elements) ======

44091.71 requests per second

====== LRANGE_300 (first 300 elements) ======

20695.37 requests per second

====== LRANGE_500 (first 450 elements) ======

14779.78 requests per second

====== LRANGE_600 (first 600 elements) ======

11739.85 requests per second

====== MSET (10 keys) ======

75987.84 requests per second

Redis 6.2.6 centos 7 3.10 内核 四线程测试数据

====== PING_INLINE ======

  throughput summary: 199203.20 requests per second

====== PING_MBULK ======

  throughput summary: 199203.20 requests per second

====== SET ======

  throughput summary: 199600.80 requests per second

====== GET ======

  throughput summary: 199203.20 requests per second

====== INCR ======

  throughput summary: 199600.80 requests per second

====== LPUSH ======

  throughput summary: 199203.20 requests per second

====== RPUSH ======

  throughput summary: 199203.20 requests per second

====== LPOP ======

  throughput summary: 199203.20 requests per second

====== RPOP ======

  throughput summary: 199203.20 requests per second

====== SADD ======

  throughput summary: 199203.20 requests per second

====== HSET ======

  throughput summary: 199203.20 requests per second

====== SPOP ======

  throughput summary: 199203.20 requests per second

====== ZADD ======

  throughput summary: 199203.20 requests per second

====== ZPOPMIN ======

  throughput summary: 199203.20 requests per second

====== LPUSH (needed to benchmark LRANGE) ======

  throughput summary: 199203.20 requests per second

====== LRANGE_100 (first 100 elements) ======

  throughput summary: 79744.82 requests per second

====== LRANGE_300 (first 300 elements) ======

  throughput summary: 44286.98 requests per second

====== LRANGE_500 (first 500 elements) ======

  throughput summary: 28457.60 requests per second

====== LRANGE_600 (first 600 elements) ======

  throughput summary: 24888.00 requests per second

====== MSET (10 keys) ======

  throughput summary: 132802.12 requests per second

Redislabs的简单使用与benchmark测试结果的更多相关文章

  1. [转帖]TPC-C解析系列01_TPC-C benchmark测试介绍

    TPC-C解析系列01_TPC-C benchmark测试介绍 http://www.itpub.net/2019/10/08/3334/ 学习一下. 自从蚂蚁金服自研数据库OceanBase获得TP ...

  2. SoapUI 利用SoapUI进行简单的接口并发测试

    利用SoapUI进行简单的接口并发测试 by:授客 QQ:1033553122 测试环境: SoapUI Pro 5.1.2 步骤如下 1.   把请求添加到测试套件 1.1.     途径1 1.新 ...

  3. 简单网络搭建与测试 mininet

    简介 本实验是基于pox搭建简单的网络并测试网络的连通性,利用mininet代码创建一个交换机四个主机的拓扑,测试各主机之间的连通性以及h1.h4之间的带宽. 代码 实验代码如下所示,SingleSw ...

  4. jmeter教程--简单的做压力测试

    Jmeter是一个非常好用的压力测试工具.  Jmeter用来做轻量级的压力测试,非常合适,只需要十几分钟,就能把压力测试需要的脚本写好. 什么是压力测试 顾名思义:压力测试,就是  被测试的系统,在 ...

  5. Java8 Stream代码详解+BenchMark测试

    Java8 Stream基础.深入.测试 1.基本介绍 1.创建方式 1.Array的Stream创建 1.直接创建 // main Stream stream = Stream.of("a ...

  6. YCSB benchmark测试mongodb性能——和web服务器测试性能结果类似

    转自:http://blog.sina.com.cn/s/blog_48c95a190102v9kg.html         YCSB(Yahoo! Cloud Serving Benchmark) ...

  7. 使用jmh框架进行benchmark测试

    性能问题 最近在跑flink社区1.15版本使用json_value函数时,发现其性能很差,通过jstack查看堆栈经常在执行以下堆栈 可以看到这里的逻辑是在等锁,查看jsonpath的LRUCach ...

  8. 分享一个.NET实现的简单高效WEB压力测试工具

    在Linux下对Web进行压力测试的小工具有很多,比较出名的有AB.虽然AB可以运行在windows下,但对于想简单界面操作的朋友有点不太习惯.其实vs.net也提供压力测试功能但显然显得太重了,在测 ...

  9. 简单的freemarker解析测试

    本文是一个很简单很基础的Freemarker模板解析测试类,复杂的也是在此基础上添加一些代码优化而来,懂得基础流程后就能融会贯通了 POM: <dependency> <groupI ...

  10. mybatis学习笔记(一）-- 简单入门（附测试Demo详细过程）

    写在最前 MyBatis 是支持定制化 SQL.存储过程以及高级映射的优秀的持久层框架.MyBatis 避免了几乎所有的 JDBC 代码和手动设置参数以及获取结果集.MyBatis 可以对配置和原生M ...

随机推荐

  1. 火山引擎DataTester分享：A/B实验中常见的8个错误

    火山引擎DataTester是科学可信的A/B测试与智能优化平台,源自字节跳动长期沉淀,能够深度耦合推荐.广告.搜索.UI.产品功能等多种行业场景需求,为业务增长.转化.产品迭代,运营提效等各环节提供 ...

  2. Mac Parallels (PD) 常规设置

    Mac 上安装了 Parallels (PD) 程序默认使用 Parallels 打开,比较烦人,取消设置: 虚机中不显示 mac 的菜单栏 ,按 Control + Option 就会临时显示 备份 ...

  3. 大数据-业务数据采集-FlinkCDC 读取 MySQL 数据存入 Kafka

    目录 作用 app 产生各层数据的 flink 任务 bean 数据对象 common 公共常量 utils 工具类 app.ods.FlinkCDC.java package com.atguigu ...

  4. CXF WebService 授权&动态调用

    SpringBoot WebService 源代码:https://gitee.com/VipSoft/VipWebService 服务端添加拦截器 import org.apache.cxf.bin ...

  5. python 提供一段文本和关键词列表进行标红处理

    def highlight_keywords_html(text, keywords): for keyword in keywords: text = text.replace(keyword, ' ...

  6. PS CJ37/CJ38 增加和返回预算

    一.预算补充CJ37/预算返回CJ38 二.补充预算CJ37,点击保存 预算返回CJ38,点击保存 三.代码示例 预算补充代码 "------------------------------ ...

  7. 【ToolChains】| CMake 技巧

    判断 CMake 编译环境 编译类型 CMAKE_BUILD_TYPE 可取值为:Debug, Release, RelWithDebInfo, MinSizeRel 等预设值 if (CMAKE_B ...

  8. Linux 安装Jupyter notebook 并开启远程访问

    一. Ubuntu下安装jupyter notebook 1. 使用Anaconda安装 conda install jupyter notebook 2. 使用pip安装 pip install j ...

  9. Hottest 30 of codeforce

    1. 4A.Watermelon 题目链接:https // s.com/problemset/problem/4/A 题意:两人分瓜,但每一部分都得是偶数 分析:直接 对2取余,且 w != 2 # ...

  10. Codeforces Round #671 (Div. 2) (A - B、D1题）

    比赛链接:https://codeforces.com/contest/1419 https://codeforces.com/contest/1419/problems A. Digit Game ...