mongodb 压缩——3.0+支持zlib和snappy
转自:https://scalegrid.io/blog/enabling-data-compression-in-mongodb-3-0/
MongoDB 3.0 with the wired tiger storage engine enables you to transparently compress the data stored in your database. This is a fairly exciting and useful feature that can be used to reduce the disk space usage of your fast growing data. By default wired tiger uses the ‘Snappy’ block compression engine for all the collections. You can turn off compression by default using the following options in the mongodb server config file .
storage:
engine: wiredTiger
wiredTiger:
collectionConfig:
blockCompressor: none
The compression algorithm can be specified at the collection level during cluster creation. Here is an example of creating a collection with ‘zlib’ compression
db.createCollection( "test", {storageEngine:{wiredTiger:{configString:'block_compressor=zlib'}}} );
MongoDB wiredtiger storage engine provides two options for compression – snappy and zlib. There is essentially a tradeoff between the extent of compression and the amount of CPU load to decompress. ‘Zlib’ achieves a lot more compression and is correspondingly less performant. ‘Snappy’ aims for ‘aims for very high speeds and reasonable compression’.
We ran some simple unscientific tests to measure the compression performance. We used one of data sets storing strings which we felt would compress well. Here is the basic structure of each document.
{
'_id': <ObjectID>,
'name': <Five character string>,
'value': <Random 1MB string>
}
We inserted about 5000 of these documents (about 5GB of data). The results were fairly impressive. Zlib achieves considerable amount of compression. Snappy also achieves a fair amount of compression with little or no load on the system.
Zlib | Snappy | uncompressed | |
Data size (MB) | 5000.5 | 5000.5 | 5000.5 |
Storage size (MB) | 19.62 | 254.37 | 5019 |
As always you need to run some tests to understand the performance gains for your data set. Here are some more detailed benchmark studies on compression performance and tradeoffs
http://www.mongodb.com/blog/post/new-compression-options-mongodb-30
http://www.acmebenchmarking.com/2015/02/mongodb-v30-compression-benchmarks.htmlhttps://comerford.cc/2015/02/04/mongodb-3-0-testing-compression/
mongodb 压缩——3.0+支持zlib和snappy的更多相关文章
- 饼干是这样压缩的——PHP使用zlib扩展实现页面GZIP压缩输出
饼干是这样压缩的——PHP使用zlib扩展实现页面GZIP压缩输出 GZIP(GNU-ZIP)是一种压缩技术.经过GZIP压缩后页面大小可以变为原来的30%甚至更小.这样用户浏览的时候就会感觉很爽很愉 ...
- hadoop对于压缩文件的支持及算法优缺点
hadoop对于压缩文件的支持及算法优缺点 hadoop对于压缩格式的是透明识别,我们的MapReduce任务的执行是透明的,hadoop能够自动为我们 将压缩的文件解压,而不用我们去关心. 如果 ...
- C#5.0支持的await格式
C#5.0支持的await格式 C#5.0引入了编译器支持的 async 和 await 关键字,这就为开发者提供了使用同步思想写异步代码的方便. 但是有些传统函数仅提供了异步回调实现,如何对其封装, ...
- WebStorm 7.0 支持更多的Web技术
JetBrains刚刚发布了WebStorm 7.0 GA,支持EJS.Mustache.Handlebars.Web组件.Stylus.Karma.Istanbul.Compass,并增强了很多功能 ...
- hadoop对于压缩文件的支持
转载:https://www.cnblogs.com/ggjucheng/archive/2012/04/22/2465580.html hadoop对于压缩格式的是透明识别,我们的MapReduce ...
- 干货来袭:Redis5.0支持的新功能说明
Redis5.0支持的新特性说明 本文内容来自华为云帮助中心 华为云DCS的Redis5.x版本继承了4.x版本的所有功能增强以及新的命令,同时还兼容开源Redis5.x版本的新增特性. Stream ...
- NoSQL Manager for MongoDB 4.6.0.3 带key
NoSQL Manager for MongoDB 4.6.0.3 是一个Windows平台的MongoDB高级管理工具.请低调使用. 博客园文件一次最大不超过10M. 官方安装包: mongodbm ...
- mongodb压缩——snappy、zlib块压缩,btree索引前缀压缩
MongoDB 3.0 WiredTiger Compression and Performance One of the most exciting developments over the li ...
- 跨平台的zip文件压缩处理,支持压缩解压文件夹
根据minizip改写的模块,需要zlib支持 输出的接口: #define RG_ZIP_FILE_REPLACE 0 #define RG_ZIP_FILE_APPEND 1 //压缩文件夹目录, ...
随机推荐
- wchar_t * 与 char * 互相转换小记
wchar_t * 与char *之间的转化只需要借助标准库里面的std::wcstombs和std::mbstowcs就能实现了
- 构建一个最简单的web应用并部署及启动
第一种构建方式:不使用maven File-new-Dynamic Web Project,用这种方式构建的web项目是在web.xml文件中配置了welcome-file的,但是却没有对应的文件,所 ...
- ubuntu中安装myeclipse提示Insufficient Memory解决方法
经过查看资料发现出现这个问题的原因是因为计算机中swap分区的内存不足,或者没有创建swap分区,google中http://www.bkjia.com/webzh/1003601.html提供了一种 ...
- ios 基础学习二
1:@property @synthesize 属性(类似java里的get set) 上面那个类原先的实现是这样: #import @interface Photo : NSObject { NS ...
- 转:Jmeter--google plugin插件监控被测系统资源方法
一.插件准备 1.插件下载地址 http://jmeter-plugins.org/downloads/all/ 以下有两个版本的,1.1.2和1.1.3,注意Jmeter版本 1.1.2支持Jmet ...
- 百度地图移动版API 1.2.2版本(Android)地图偏移的最佳解决办法
Import import com.baidu.mapapi.CoordinateConvert;import com.baidu.mapapi.GeoPoint; Code GeoPoint p = ...
- wordpress安装插件--su
Add to Any: Subscribe Button 让读者方便的订阅你的博客到任何Feed阅读器 Google XML Sitemaps 生成完全兼容各大搜索引擎的Sitemaps/网站地图. ...
- html5中拨打电话代码
<a href="tel:18600000000">给我打电话</a> <a href="sms:18600000000"&g ...
- elisp
cons cell? 构建内存对象! 通过寄存器的基地址car和偏移地址cdr来--寻址内存对象,我是这样理解的. http://wiki.dourok.info/doku.php/%E5%B7%A5 ...
- jsp 获取cookie 的值的方法
Cookie cookies[]=request.getCookies(); //读出用户硬盘上的Cookie,并将所有的Cookie放到一个cookie对象数组里面 Cookie sCookie=n ...