Mydumper.org web site has been missing in action for a while now. I've uploaded a copy of the Mydumper documentation to this web site for easy reference. I did benchmarks on my blog athttp://vbtechsupport.com/1695/ and http://vbtechsupport.com/1716/

Note: Centmin Mod only hosting documentation and provided as is without any technical support.

Download:

Download latest Mydumper version from https://launchpad.net/mydumper. You may also need aMydumper patch for Mydumper to work with MariaDB 5.5 MySQL.

Authors

The code for mydumper has been written by the following people:

  • Domas Mituzas <http://dom.as/> , Facebook ( domas at fb dot com )
  • Andrew Hutchings <http://www.linuxjedi.co.uk> , SkySQL ( andrew at skysql dot com )
  • Mark Leith <http://www.markleith.co.uk/> , Oracle Corporation ( mark dot leith at oracle dot com )

    Compiling

    Requirements
    ------------

    mydumper requires the following before it can be compiled:

  • CMake <http://www.cmake.org/>
  • Glib2 <http://www.gtk.org/index.php> (with development packages)
  • PCRE <http://www.pcre.org/> (with development packages)
  • MySQL <http://www.mysql.com/> client libraries (with development packages)

    Additionally the following packages are optional:

  • python-sphinx <http://sphinx.pocoo.org/> (for documentation)

    Ubuntu/Debian
    ^^^^^^^^^^^^^

       apt-get install libglib2.0-dev libmysqlclient15-dev zlib1g-dev libpcre3-dev
    

    Fedora/Redhat/CentOS
    ^^^^^^^^^^

       yum install glib2-devel mysql-devel zlib-devel pcre-devel
    

    OpenSUSE
    ^^^^

       zypper install glib2-devel libmysqlclient-devel pcre-devel zlib-devel
    

    Mac OSX
    ^^^^^^^

       port install glib2 mysql5 pcre
    

    CMake
    -----

    CMake is used for mydumper's build system and is executed as follows::

      cmake .
    make

    You can optionally provide parameters for CMake, the possible options are:

  • -DMYSQL_CONFIG=/path/to/mysql_config - The path and filename for the mysql_config executable
  • -DCMAKE_INSTALL_PREFIX=/install/path - The path where mydumper should be installed

    Documentation
    -------------

    If you wish to just compile the documentation you can do so with::

      cmake .
    make doc_html

    or for a man page output::

      cmake .
    make doc_man

    Mydumper Usage

    Synopsis
    --------

    mydumper [OPTION...]

    Description
    -----------

    mydumper is a tool used for backing up MySQL database servers much
    faster than the mysqldump tool distributed with MySQL. It also has the
    capability to retrieve the binary logs from the remote server at the same time
    as the dump itself. The advantages of mydumper are:

  • Parallelism (hence, speed) and performance (avoids expensive character set conversion routines, efficient code overall)
  • Easier to manage output (separate files for tables, dump metadata, etc, easy to view/parse data)
  • Consistency - maintains snapshot across all threads, provides accurate master and slave log positions, etc
  • Manageability - supports PCRE for specifying database and tables inclusions and exclusions

    Options
    -------

    The mydumper tool has several available options:

    mydumper

     --help, -?

    Show help text

     --host, -h

    Hostname of MySQL server to connect to (default localhost)

     --user, -u

    MySQL username with the correct privileges to execute the dump

     --password, -p

    The corresponding password for the MySQL user

     --port, -P

    The port for the MySQL connection.

    Note:For localhost TCP connections use 127.0.0.1 for --host`.

     --socket, -S

    The UNIX domain socket file to use for the connection

     --database, -B

    Database to dump

     --table-list, -T

    A comma separated list of tables to dump

     --threads, -t

    The number of threads to use for dumping data, default is 4

    Note:Other threads are used in mydumper, this option does not control these

     --outputdir, -o

    Output directory name, default is export-YYYYMMDD-HHMMSS

     --statement-size, -s

    The maximum size for an insert statement before breaking into a new
    statement, default 1,000,000 bytes

     --rows, -r

    Split table into chunks of this many rows, default unlimited

     --compress, -c

    Compress the output files

     --compress-input, -C

    Use client protocol compression for connections to the MySQL server

     --build-empty-files, -e

    Create empty dump files if there is no data to dump

     --regex, -x

    A regular expression to match against database and table

     --ignore-engines, -i

    Comma separated list of storage engines to ignore

     --no-schemas, -m

    Do not dump schemas with the data

     --long-query-guard, -l

    Timeout for long query execution in seconds, default 60

     --kill-long-queries, -k

    Kill long running queries instead of aborting the dump

     --version, -V

    Show the program version and exit

     --verbose, -v

    The verbosity of messages. 0 = silent, 1 = errors, 2 = warnings, 3 = info.
    Default is 2.

     --binlogs, -b

    Get the binlogs from the server as well as the dump files

      --daemon, -D

    Enable daemon mode

     --snapshot-interval, -I

    Interval between each dump snapshot (in minutes), requires
    --daemon`, default 60 (minutes)

     --logfile, -L

    A file to log mydumper output to instead of console output. Useful for
    daemon mode.

     --no-locks, -k

    Do not execute the temporary shared read lock.

    .. warning::

    This will cause inconsistent backups.


    Myloader Usage

    Synopsis
    --------

    myloader [OPTION...]

    Description
    -----------

    myloader is a tool used for multi-threaded restoration of mydumper
    backups.

    Options
    -------

    The myloader tool has several available options:

    myloader

     --help, -?

    Show help text

     --host, -h

    Hostname of MySQL server to connect to (default localhost)

     --user, -u

    MySQL username with the correct privileges to execute the restoration

     --password, -p

    The corresponding password for the MySQL user

     --port, -P

    The port for the MySQL connection.

    Note:For localhost TCP connections use 127.0.0.1 for --host`.

     --socket, -S

    The UNIX domain socket file to use for the connection

     --threads, -t

    The number of threads to use for restoring data, default is 4

     --version, -V

    Show the program version and exit

     --compress-protocol, -C

    Use client protocol compression for connections to the MySQL server

     --directory, -d

    The directory of the mydumper backup to restore

     --database, -B

    An alternative database to load the dump into

    Note:For use with single database dumps. When using with multi-database dumps
    that have duplicate table names in more than one database it may cause 
    errors. Alternatively this scenario may give unpredictable results with
    --overwrite-tables`.

     --queries-per-transaction, -q

    Number of INSERT queries to execute per transaction during restore, default
    is 1000.

     --overwrite-tables, -o

    Drop any existing tables when restoring schemas

     --enable-binlog, -e

    Log the data loading in the MySQL binary log if enabled (off by default)

     --verbose, -v

    The verbosity of messages. 0 = silent, 1 = errors, 2 = warnings, 3 = info.
    Default is 2.


    Output Files

    mydumper generates several files during the generation of the dump. Many of
    these are for the table data itself since every table has at least one file.

    Metadata
    --------

    When a dump is executed a file called ``.metadata`` is created in the output
    directory. This contains the start and end time of the dump as well as the
    master binary log positions if applicable.

    This is an example of the content of this file::

      Started dump at: 2011-05-05 13:57:17
    SHOW MASTER STATUS:
    Log: linuxjedi-laptop-bin.000001
    Pos: 106 Finished dump at: 2011-05-05 13:57:17

    Table Data
    ----------

    The data from every table is written into a separate file, also if the
    --rows <mydumper --rows>` option is used then each chunk of table will
    be in a separate file. The file names for this are in the format::

        database.table.sql(.gz)
    

    or if chunked::

        database.table.chunk.sql(.gz)
    

    Where 'chunk' is a number padded with up to 5 zeros.

    Table Schemas
    -------------

    When the --schemas <mydumper --schemas>` option is used mydumper will
    create a file for the schema of every table it is writing data for. The files
    for this are in the following format::

        database.table-schema.sql(.gz)
    

    Binary Logs
    -----------

    Binary logs are retrieved when --binlogs <mydumper --binlogs>` option
    has been set. This will store them in the ``binlog_snapshot/`` sub-directory
    inside the dump directory.

    The binary log files have the same filename as the MySQL server that supplies them and will also have a .gz on the end if they are compressed.

    Daemon mode
    -----------

    Daemon mode does things a little differently. There are the directories ``0``
    and ``1`` inside the dump directory. These alternate when dumping so that if
    mydumper fails for any reason there is still a good snapshot. When a snapshot
    dump is complete the ``last_dump`` symlink is updated to point to that dump.

    If binary logging is enabled mydumper will connect as if it is a slave server
    and constantly retreives the binary logs into the ``binlogs`` subdirectory.


    Examples

    Simple Usage
    ------------

    Just running mydumper without any options will try to connect to a
    server using the default socket path. It will then dump the tables from all
    databases using 4 worker threads.

    Regex
    -----

    To use mydumper's regex feature simply use the
    --regex <mydumper --regex>` option. In the following example mydumper
    will ignore the ``test`` and ``mysql`` databases::

        mydumper --regex '^(?!(mysql|test))'
    

    Restoring a dump
    ----------------

    Mydumper now include myloader which is a multi-threaded restoration tool. To
    use myloader with a mydumper dump you simply need to pass it the directory of
    the dump along with a user capable of restoring the schemas and data. As an
    example the following will restore a dump overwriting any existing tables::

        myloader --directory=export-20110614-094953 --overwrite-tables --user=root
    

    Daemon mode
    -----------

    Mydumper has a daemon mode which will snapshot the dump data every so often
    whilst continuously retreiving the binary log files. This gives a continuous
    consistent backup right up to the point where the database server fails. To use
    this you simply need to use the --daemon <mydumper --daemon>` option.

    In the following example mydumper will use daemon mode, creating a snapshot
    every half an hour and log to an output file::

        mydumper --daemon --snapshot-interval=30 --logfile=dump.log
    

mydumper用例

# mydumper -B ttlsa_com -o /home/data/bak -b
# ls
binlog_snapshot metadata ttlsa_com.my_area-schema.sql ttlsa_com.my_area.sql
# cat metadata
Started dump at: 2012-02-20 11:40:21
SHOW MASTER STATUS:
Log: mysqld-bin.000001
Pos: 191363
Finished dump at: 2012-02-20 11:40:22
# mydumper -B ttlsa_com -o /home/data/bak -b -D --snapshot-interval=30
# myloader -d /home/data/bak/last_dump -o -B ttlsa_com //还原
参考:
http://centminmod.com/mydumper.html
http://www.ttlsa.com/mysql/mysql-backup-tool-mydumper/

Mydumper & Myloader Documentation的更多相关文章

  1. mydumper/myloader使用详解

      mydumper安装:http://www.cnblogs.com/lizhi221/p/7010174.html mydumper原理:http://www.cnblogs.com/lizhi2 ...

  2. MYSQL mydumper & myloader

    第三方逻辑备份工具myduper和myloader | xiaoyu的数据库小窝-技术交流http://www.dbaxiaoyu.com/archives/1643 myloader原理0 - ze ...

  3. mydumper,myloader原理及实战

    mydumper 特性 (1)多线程备份(和mysqlpump的多线程不同,mysqlpump多线程备份的粒度是表,mydumper多线程备份的粒度是行,这对于备份大表特别有用)(2)因为是多线程逻辑 ...

  4. mysql小白系列_06 备份与恢复 - mydumper与myloader

    http://blog.csdn.net/leshami/article/details/46815553 https://www.cnblogs.com/zhoujinyi/p/3423641.ht ...

  5. mydumper下载安装

    下载地址   https://github.com/maxbube/mydumper [root@gg ~]#cd mydumper [root@gg mydumper]# cmake . -bash ...

  6. mydumper原理5

    前言 之前介绍了Oracle官方的多线程逻辑导入导出工具mysqlpump,但已经操作过的同学会发现其多线程的单位还是表,换句话说, 单表依然是 单线程导出 .网易早已使用mydumper/myloa ...

  7. MySQL备份与恢复-mydumper

    上一片博文中,我们说明了mysqldump的备份与恢复.因为mysqldump是单线程导出,单线程恢复的,因此备份与恢复的时间比较长! 首先来安装mydumper: 下载源码:https://gith ...

  8. 关于mydumper的.metadata文件丢失

    今天要进行MySQL的数据迁移,所以把数据库通过mydumper工具备份的文件解压后.通过myloader进行导入 可是导入的时间出现这个报错: ** (myloader:766): CRITICAL ...

  9. mydumper 介绍及使用

    1 Mydumper 介绍 Mydumper是一个针对MySQL和Drizzle的高性能多线程备份和恢复工具. Mydumper主要特性: 轻量级C语言写的 多线程备份,备份后会生成多个备份文件 事务 ...

随机推荐

  1. Javascript 的类型转换之减号

    专职写JS已经有一个月了(对,没错就是一个月),从2014年11月24实习开始到今的2月份,我做的工作一直都是切图,另外跟着老板学产品,现在我一听到切图两字,我就想吐...所以我找了一份专职写JS的工 ...

  2. application:didFinishLaunchingWithOptions:详解

    iOS 程序启动时总会调用application:didFinishLaunchingWithOptions:,其中第二个参数launchOptions为NSDictionary类型的对象,里面存储有 ...

  3. JS - The react framework

    这几天因为赶时间 , 所以理解上可能有许多的误差 , 如果你不幸点进来了 , 请不要看我的代码 , 这几天我会重新修改 , 然后把错误的不足的 全部修正一下 . /hwr/src/index.js i ...

  4. Android Glide加载图片时转换为圆形、圆角、毛玻璃等图片效果

     Android Glide加载图片时转换为圆形.圆角.毛玻璃等图片效果 附录1简单介绍了Android开源的图片加载框架.在实际的开发中,虽然Glide解决了快速加载图片的问题,但还有一个问题悬 ...

  5. Smart210学习记录------块设备

    转自:http://bbs.chinaunix.net/thread-2017377-1-1.html 本章的目的用尽可能最简单的方法写出一个能用的块设备驱动.所谓的能用,是指我们可以对这个驱动生成的 ...

  6. Magento后台表单字段添加备注

    Magento的后台表单封装的非常好,各种字段都能够直接找到方法调用.在最近的一个项目中,为客户定制了一款定时变价功能,该功能需要导入一个csv作为变价的基础.为了方便客户,我们需要在上传表单位置添加 ...

  7. vs2005无法启动

    vs2005无法启动   今天打开vs.net 2005,VS2005无法启动了,出现初始化界面后,一直没有响应,后来在网上找到问题,发现是前段时间装VMWARE后,修改注册表引起的问题 转载如下 电 ...

  8. js获取非行内样式

    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/ ...

  9. OVS ARP Responder – Theory and Practice

    Prefix In the GRE tunnels post I’ve explained how overlay networks are used for connectivity and ten ...

  10. (实用篇)PHP JSON数组与对象的理解

    在PHP后端和客户端数据交互的过程中,JSON数据中有时格式不定,一会儿是数组,一会儿是对象,弄得客户端开发人员要崩溃的感觉. 因此,前后端相关人员先对PHP的json_encode函数原理有必要的了 ...