Small things are better
Yesterday I had fun time repairing 1.5Tb ext3 partition, containing many millions of files. Of course it should have never happened – this was decent PowerEdge 2850 box with RAID volume, ECC memory and reliable CentOS 4.4 distribution but still it did. We had “journal failed” message in kernel log and filesystem needed to be checked and repaired even though it is journaling file system which should not need checks in normal use, even in case of power failures. Checking and repairing took many hours especially as automatic check on boot failed and had to be manually restarted.
Same may happen with Innodb tables. They are designed to never crash, surviving power failures and even partial page writes but still they can get corrupted because of MySQL bugs, OS Bugs or hardware bugs, misconfiguration or failures.
Sometimes
corruption kind be mild, so ALTER TABLE to rebuild the table fixes it.
Sometimes table needs to be dropped and recovered from backup but in
certain cases you may need to reimport whole database – if corruption is
happens to be in undo tablespace or log files.
So do not forget
to have your recovery plan this kind failures. This is one thing you
better to have backups for. Backups however take time to restore,
especially if you do point in time recovery using binary log to get to
actual database state.
The good practice to approach this kind of
problem is first to have enough redundancy. I always assume any
component, such as piece of hardware or software can fail, even if this
piece of hardware has some internal redundancy by itself, such as RAID
or SAN solutions.
If you can’t afford full redundancy for
everything (and probably even if you do) the good idea is to keep your
objects smaller so if you need to do any maintenance with them it will
take less times. Smaller RAID volumes would typically rebuild faster,
smaller database size per system (yet another reason to like medium
end commodity hardware) makes it faster to recover, smaller tables
allow per table backup and recovery to happen faster.
With MySQL
and blocking ALTER TABLE there is yet another reason to keep tables
small, so you do not have to use complicated scenarios to do simple
things. Assume for example you need to add extra column to 500GB
Innodb table. It will probably take long hours or even days for ALTER
TABLE to complete and about 500GB of temporary space will be required
which you simply might not have. You can of course use MASTER-MASTER
replication and run statement on one server, switch role and then do it
on other, but if alter table takes several days do you really can
afford having no box to fall back to for such a long time ?
On
other hand if you would have 500 of 1GB tables it would be very easy –
you can simply move small pieces of data offline for a minute and alter
them live. Also all process will be much faster this way as whole
indexes will well fit in memory for such small tables.
Not to mention splitting 500 tables to several servers will likely be easy than splitting one big one.
There
are bunch of complications with many tables of course, it is not always
easy to partition your data appropriately, also code gets complicated
but for many applications it is worth the trouble
At NNSEEK
for example we have data split at 256 groups of tables. Current data
size is small enough so even single table would not be big problem but
it is much easier to write your code to handle split from very beginning
rather than try to add in later on when there are 100 helper scripts
written etc.
For the same reason I would recommend setting up
multiple virtual servers even if you work with physical one in the
beginning. Different accounts with different permissions will be good
enough. Doing so will ensure you will not have problems once you will
really need to scale to multiple servers.
随机推荐
- (数据科学学习手札20)主成分分析原理推导&Python自编函数实现
主成分分析(principal component analysis,简称PCA)是一种经典且简单的机器学习算法,其主要目的是用较少的变量去解释原来资料中的大部分变异,期望能将现有的众多相关性很高的变 ...
- LeetCode:15. 3Sum(Medium)
1. 原题链接 https://leetcode.com/problems/3sum/description/ 2. 题目要求 数组S = nums[n]包含n个整数,请问S中是否存在a,b,c三个整 ...
- 云计算之路-阿里云上:Web服务器请求到达量突降
今天下午遇到了自使用阿里云以来首次遇到的新情况——http.sys的ArrivalRate突降(说明请求到达IIS的请求数量少了),而且SLB中的3台ECS都出现了这个问题. 1. 10.161.24 ...
- 《数据结构与算法分析:C语言描述》读书笔记
我们数据结构的课用了这本英文教材,作者是Mark Allen Weiss.总体来说比<算法导论>简单很多,但内容上交集非常大.其实是因为去掉了大多数证明和数学,对于没有耐心看符号和公式的人 ...
- qt 编译unresolved external symbol的错误解决
题外问题:.rc文件报错,里面引用的.h文件打不开. 方法:rc文件移除,然后重新添加就可以: unresolved external symbol的原因: 1.没有添加编译生成的moc文件,添加对应 ...
- (原)Android到IOS开发的转换(一)
序)闲扯几句 很早就想入手ios开发,但是一直没有机会,个人没有水果机器,上个公司上班的那台mac mini虽然就在我身边,灰都有一层了,但是一直没有机会开机学习下,因为事多,自上一篇文章后,离职后, ...
- mybatis <collection>标签 类型为string时无法获取重复数据错误
1.场景: fyq_share_house 表 和 fyq_sh_tag 表 两张表是一对多的关系, 一个楼盘对应多个标签,在实体类ShareHouse中使用 /** * 楼盘标签 */ privat ...
- 名字管理系统demo
# 名字管理系统demo # 打印功能提示 print('欢迎使用名字管理系统v6.6.6') print('1:添加一个名字') print('2:删除一个名字') print('3:修改一个名字' ...
- Java IO学习--输入输出流
一.Java IO的定义 I/O:输入输出系统,由输入输出控制系统和外围设备两部分组成. Java中I/O操作主要是指使用Java进行输入,输出操作. Java所有的I/O机制都是基于数据流进行输入输 ...
- 在阿里云上遇见更好的Oracle(四)
2016.5.13,北京,第七届数据库技术大会. 从最初的itpub社区,到后来被it168收购,DBA社区的线下聚会发展成2010年第一届数据库技术大会(DTCC).第一届大会汇聚了社区内活跃的各位 ...