Redis: Reducing Memory Usage
High Level Tips for Redis
Most of Stream-Framework's users start out with Redis and eventually move to Cassandra because Redis becomes too expensive to host. Stream, our hosted API for building scalable newsfeeds & activity streams is also based on Cassandra.
There are quite a few things you can do to reduce Redis' memory usage though. Here are my favourite 6 high level tips for reducing Redis memory usage. For lower level tips, see the Redis docs on memory optimization.
Tip 1 - Serialization
It is often possible to reduce your Redis memory usage by using the right serialization. Serialize objects using built-in serialization mechanism is very tempting (eg. Python's pickle, PHP's serialize...) but it will likely waste lot of memory for nothing. You should stay away from it and make sure you only serialize what you need. Using something like JSON or Protobuf is often a good idea.
Tip 2 - Compression
If the data you're storing is large enough and contains a lot of text you can often reduce memory usage by adding compression. LZO or Snappy are good options for this use case.
Tip 3 - Fallback to disk based storage (aka your rdbms)
Many times you don't need to store all the data in Redis. If your model allows it, try to store only hot or recent data and hit the database when a request for missing data is performed.
If you go for this approach be sure to include an endmarker in your data. Assuming there are 3 items in your database, Redis would store the 3 items and the end marker. This way you know you don't have to run a database query after item number 3. The end marker approach vastly reduces how frequently you fall back to the database.
Tip 4 - Normalize your storage
Memory is expensive and Redis only uses memory. Take this into account when designing your data structure. A normalized approach is often the better choice compared to a denormalized option.
Tip 5 - Expire and trim the data
If possible, set expirations on everything. This presents data which is rarely used from sticking around in your cluster. It can be a few days, a few weeks, but definitely set some sort of expiration on your data. If you're inserting into lists or sets be sure to trim them occasionally.
Tip 6 - Use the right eviction policy
If the amount of data that you store in Redis grows over time and you can't afford to keep it all in memory, you probably want to configure Redis as a LRU cache storage. Redis allows for 6 different eviction policies, make sure you pick the right one!
Most of these tips are obvious, but before you switch away from Redis, be sure to give them a try.
Redis: Reducing Memory Usage的更多相关文章
- Reducing and Profiling GPU Memory Usage in Keras with TensorFlow Backend
keras 自适应分配显存 & 清理不用的变量释放 GPU 显存 Intro Are you running out of GPU memory when using keras or ten ...
- Shell script for logging cpu and memory usage of a Linux process
Shell script for logging cpu and memory usage of a Linux process http://www.unix.com/shell-programmi ...
- 5 commands to check memory usage on Linux
Memory Usage On linux, there are commands for almost everything, because the gui might not be always ...
- SHELL:Find Memory Usage In Linux (统计每个程序内存使用情况)
转载一个shell统计linux系统中每个程序的内存使用情况,因为内存结构非常复杂,不一定100%精确,此shell可以在Ghub上下载. [root@db231 ~]# ./memstat.sh P ...
- Why does the memory usage increase when I redeploy a web application?
That is because your web application has a memory leak. A common issue are "PermGen" memor ...
- GPU Memory Usage占满而GPU-Util却为0的调试
最近使用github上的一个开源项目训练基于CNN的翻译模型,使用THEANO_FLAGS='floatX=float32,device=gpu2,lib.cnmem=1' python run_nn ...
- 【转】C++ Incorrect Memory Usage and Corrupted Memory(模拟C++程序内存使用崩溃问题)
http://www.bogotobogo.com/cplusplus/CppCrashDebuggingMemoryLeak.php Incorrect Memory Usage and Corru ...
- Memory usage of a Java process java Xms Xmx Xmn
http://www.oracle.com/technetwork/java/javase/memleaks-137499.html 3.1 Meaning of OutOfMemoryError O ...
- detect data races The cost of race detection varies by program, but for a typical program, memory usage may increase by 5-10x and execution time by 2-20x.
小结: 1. conflicting access 2.性能危害 优化 The cost of race detection varies by program, but for a typical ...
随机推荐
- bsearch的使用
懒得写二分查找,结果发现stdlib里自带了二分查找. C 库函数 void *bsearch(const void *key, const void *base, size_t nitems, si ...
- Java数组去重的方法
//第一种方式:最开始想到的是利用Set集合的不可重复性进行元素过滤 public static Object[] oneClear(Object[] arr){ Set set = new Has ...
- 选择本地文件上传控件 input标签
当要通过控件来选择本地文件上传的时候的一种方式 <input type="file" id="input-file"/> 注意 type类型一定要是 ...
- Linux-lsxxx
Linux-lsxxx ls list directory contents 列出文件及目录 lsattr List file attributes on a Linux second extende ...
- C++代做,C++编程代做,C++程序代做,留学生C++ Lab代写
C++代做,C++编程代做,C++程序代做 我们主要面向留学生,广泛接美加澳国内港台等地编程作业代写,中英文均可. C语言代写 C++代写 Python代写 Golang代写 Java代写 一年半的时 ...
- 8个最常用的Python内置函数,小白必备!
题图:Photo by Markus Spiske on Unsplash Python给我们内置了大量功能函数,官方文档上列出了69个,有些是我们是平时开发中经常遇到的,也有一些函数很少被用到,这里 ...
- 01.Delphi最简单的接口
我想学习一个插件框架,但是那个框架里面大量用到了接口,于是不得不把接口看一下了.总感觉接口编程这一块非常的绕,每一行都注释了. unit Unit1; interface uses Windows, ...
- leetcode1042 Flower Planting With No Adjacent
""" You have N gardens, labelled 1 to N. In each garden, you want to plant one of 4 t ...
- bean的autowire属性及其生命周期
一:sutowire属性 1.no:默认值,禁用自动装配: 2.byName:根据属性名称自动装配: 3.byType:根据属性类型自动装配: 4.constructor:通过构造方法自动装配,不推荐 ...
- Vue+Axios+Nigix+SpringCloud前端和后端搭建及其碰到的问题
一.Axios.Router的安装和使用 1.如何安装Axios和Router 1).进入到工程所在的文件夹,通过cmd指令,进入到window的dos界面 2).输入:npm install axi ...