mongdb 备份还原导入导出

-------------------MongoDB数据导入与导出-------------------

1、导出工具：mongoexport

    1、概念：

        mongoDB中的mongoexport工具可以把一个collection导出成JSON格式或CSV格式的文件。可以通过参数指定导出的数据项，也可以根据指定的条件导出数据。

    2、语法：

        mongoexport -d dbname -c collectionname -o file --type json/csv -f field

        参数说明：

            -d ：数据库名

            -c ：collection名

            -o ：输出的文件名

            --type ： 输出的格式，默认为json

            -f ：输出的字段，如果-type为csv，则需要加上-f "字段名"

    3、示例：

      sudo mongoexport -d mongotest -c users -o /home/python/Desktop/mongoDB/users.json --type json -f  "_id,user_id,user_name,age,status"

 

2、数据导入：mongoimport

    1、语法：

        mongoimport -d dbname -c collectionname --file filename --headerline --type json/csv -f field

        参数说明：

            -d ：数据库名

            -c ：collection名

            --type ：导入的格式默认json

            -f ：导入的字段名

            --headerline ：如果导入的格式是csv，则可以使用第一行的标题作为导入的字段

            --file ：要导入的文件

 

    2、示例：

sudo mongoimport -d mongotest -c users --file /home/mongodump/articles.json --type json

 

-------------------MongoDB备份与恢复-------------------

1、MongoDB数据库备份

    1、语法：

        mongodump -h dbhost -d dbname -o dbdirectory

        参数说明：

            -h： MongDB所在服务器地址，例如：127.0.0.1，当然也可以指定端口号：127.0.0.1:27017

            -d： 需要备份的数据库实例，例如：test

            -o： 备份的数据存放位置，例如：/home/mongodump/，当然该目录需要提前建立，这个目录里面存放该数据库实例的备份数据。

    2、实例：

        sudo rm -rf /home/momgodump/
        sudo mkdir -p /home/momgodump
        sudo mongodump -h 192.168.17.129: -d itcast -o /home/mongodump/

        -

2、MongoDB数据库恢复

    1、语法：

        mongorestore -h dbhost -d dbname --dir dbdirectory

 

        参数或名：

            -h： MongoDB所在服务器地址

            -d： 需要恢复的数据库实例，例如：test，当然这个名称也可以和备份时候的不一样，比如test2

            --dir： 备份数据所在位置，例如：/home/mongodump/itcast/

            --drop： 恢复的时候，先删除当前数据，然后恢复备份的数据。就是说，恢复后，备份后添加修改的数据都会被删除，慎用！

    2、实例：

mongorestore -h 192.168.17.129: -d itcast_restore --dir /home/mongodump/itcast/

备份mmongodb的方式多种多样,下面介绍几种备份方式
一.复制数据文件
复制数据库目录的所有文件,在复制的时候防止数据文件的改变,可以使用fsyncLock()命令来锁定数据库,禁止任何写入.
> db.fsyncLock()

{ "info" : "now locked against writes, use db.fsyncUnlock() to unlock", "seeAlso" : "http://dochub.mongodb.org/core/fsynccommand", "ok" : 1}此时任何修改都会加入到队列等待.复制完成之后使用fsyncUnlock()命令来解除锁定.
> db.fsyncUnlock()

{ "info" : "unlock completed", "ok" : 1 }

二.使用mongodump/mongorestore备份恢复

mongodump/mongorestore是用来备份和恢复bson数据.mongodump不会备份local数据库.如果使用了访问控制,那么备份的用户必须拥有backup角色.
mongodump通过连接mongod或者mongos来备份数据,mongodump可以用来备份整个数据,数据库,集合或者还可以通过查询备份集合的一部分.
如果你运行mongodump而不添加任何参数,那么此命令将会连接本地的27017端口,然后创建一个数据库的备份在当前的dump目录下面.
从2.2版本以后的mongodump和以前版本的mongod不兼容,不要使用新版本的mongodump来备份老版本的mognodb.
mongodump的帮助如下:
[root@mongodb3 ~]# mongodump --helpUsage: mongodump <options> Export the content of a running server into .bson files. Specify a database with -d and a collection with -c to only dump that database or collection. See http://docs.mongodb.org/manual/reference/program/mongodump/ for more information. general options: --help print usage --version print the tool version and exit verbosity options: -v, --verbose=<level> more detailed log output (include multiple times for more verbosity, e.g. -vvvvv, or specify a numeric value, e.g. --verbose=N) --quiet hide all log output connection options: -h, --host=<hostname> mongodb host to connect to (setname/host1,host2 for replica sets) --port=<port> server port (can also use --host hostname:port) authentication options: -u, --username=<username> username for authentication -p, --password=<password> password for authentication --authenticationDatabase=<database-name> database that holds the user's credentials --authenticationMechanism=<mechanism> authentication mechanism to use namespace options: -d, --db=<database-name> database to use -c, --collection=<collection-name> collection to use query options: -q, --query= query filter, as a JSON string, e.g., '{x:{$gt:1}}' --queryFile= path to a file containing a query filter (JSON) --readPreference=<string>|<json> specify either a preference name or a preference json object --forceTableScan force a table scan output options: -o, --out=<directory-path> output directory, or '-' for stdout (defaults to 'dump') --gzip compress archive our collection output with Gzip --repair try to recover documents from damaged data files (not supported by all storage engines) --oplog use oplog for taking a point-in-time snapshot --archive=<file-path> dump as an archive to the specified path. If flag is specified without a value, archive is written to stdout --dumpDbUsersAndRoles dump user and role definitions for the specified database --excludeCollection=<collection-name> collection to exclude from the dump (may be specified multiple times to exclude additional collections) --excludeCollectionsWithPrefix=<collection-prefix> exclude all collections from the dump that have the given prefix (may be specified multiple times to exclude additional prefixes) -j, --numParallelCollections= number of collections to dump in parallel (4 by default)

1.备份全库
最简单的一个备份全库语句如下:
[root@mongodb3 ~]# mongodump --host 127.0.0.1 --port 37017

2016-06-27T21:29:49.121+0800 writing admin.system.users to 2016-06-27T21:29:49.121+0800 done dumping admin.system.users (3 documents)2016-06-27T21:29:49.121+0800 writing admin.system.roles to 2016-06-27T21:29:49.121+0800 done dumping admin.system.roles (2 documents)2016-06-27T21:29:49.122+0800 writing admin.system.version to 2016-06-27T21:29:49.122+0800 done dumping admin.system.version (1 document)2016-06-27T21:29:49.122+0800 writing suq.t2 to 2016-06-27T21:29:49.122+0800 writing suq.t1 to 2016-06-27T21:29:49.123+0800 done dumping suq.t2 (1 document)2016-06-27T21:29:49.123+0800 done dumping suq.t1 (0 documents)

那么会在当前目录下创建一个dump目录,dump下面是备份的数据库名称:
[root@mongodb3 ~]# ls dumpadmin suq使用-o可以指定目录:
[root@mongodb3 ~]# mongodump --port 37017 -o backup

如果数据使用的副本集,那么在备份的时候可以添加--oplog选项,在备份结束之后会将oplog的所有操作记录下来,得到某个时间点的快照.

2.备份数据库
使用-d选项备份某个数据库:

[root@mongodb3 ~]# mongodump --port 37017 -d suq -o backup2016-06-27T21:35:00.406+0800 writing suq.t2 to 2016-06-27T21:35:00.406+0800 writing suq.t1 to 2016-06-27T21:35:00.406+0800 done dumping suq.t2 (1 document)2016-06-27T21:35:00.407+0800 done dumping suq.t1 (0 documents)[root@mongodb3 ~]# lsanaconda-ks.cfg backup Desktop Documents Downloads install.log install.log.syslog mm mongod.conf Music Pictures Public soft Templates Videos[root@mongodb3 ~]# ls backupsuq3.备份指定集合
使用-c选项指定集合的名字
[root@mongodb3 ~]# mongodump --port 37017 -c t1 -d suq -o backup2016-06-27T22:10:43.047+0800 writing suq.t1 to 2016-06-27T22:10:43.047+0800 done dumping suq.t1 (0 documents)[root@mongodb3 ~]# ls backupsuq[root@mongodb3 ~]# ls backup/suq/t1.bson t1.metadata.json4.使用mongorestore恢复
mongorestore用来恢复一个使用mongodump的备份.默认的mongorestore寻常当前目录下的dump目录.
下面是mongorestore的参数说明:

[root@mongodb3 ~]# mongorestore --helpUsage: mongorestore <options> <directory or file to restore> Restore backups generated with mongodump to a running server. Specify a database with -d to restore a single database from the target directory,or use -d and -c to restore a single collection from a single .bson file. See http://docs.mongodb.org/manual/reference/program/mongorestore/ for more information. general options: --help print usage --version print the tool version and exit verbosity options: -v, --verbose=<level> more detailed log output (include multiple times for more verbosity, e.g. -vvvvv, or specify a numeric value, e.g. --verbose=N) --quiet hide all log output connection options: -h, --host=<hostname> mongodb host to connect to (setname/host1,host2 for replica sets) --port=<port> server port (can also use --host hostname:port) authentication options: -u, --username=<username> username for authentication -p, --password=<password> password for authentication --authenticationDatabase=<database-name> database that holds the user's credentials --authenticationMechanism=<mechanism> authentication mechanism to use namespace options: -d, --db=<database-name> database to use -c, --collection=<collection-name> collection to use input options: --objcheck validate all objects before inserting --oplogReplay replay oplog for point-in-time restore --oplogLimit=<seconds>[:ordinal] only include oplog entries before the provided Timestamp --archive=<filename> restore dump from the specified archive file. If flag is specified without a value, archive is read from stdin --restoreDbUsersAndRoles restore user and role definitions for the given database --dir=<directory-name> input directory, use '-' for stdin --gzip decompress gzipped input restore options: --drop drop each collection before import --writeConcern=<write-concern> write concern options e.g. --writeConcern majority, --writeConcern '{w: 3, wtimeout: 500, fsync: true, j: true}' (defaults to 'majority') --noIndexRestore don't restore indexes --noOptionsRestore don't restore collection options --keepIndexVersion don't update index version --maintainInsertionOrder preserve order of documents during restoration -j, --numParallelCollections= number of collections to restore in parallel (4 by default) --numInsertionWorkersPerCollection= number of insert operations to run concurrently per collection (1 by default) --stopOnError stop restoring if an error is encountered on insert (off by default) --bypassDocumentValidation bypass document validation默认的在restore的时候不会删除和清空原集合.可以使用--drop选项.
最简单的一个restore语法如下:
[root@mongodb3 ~]# mongorestore --port 37017 backup

/2016-06-27T22:18:27.571+0800 building a list of dbs and collections to restore from backup dir2016-06-27T22:18:27.572+0800 reading metadata for suq.t1 from backup/suq/t1.metadata.json2016-06-27T22:18:27.572+0800 restoring suq.t1 from backup/suq/t1.bson2016-06-27T22:18:27.572+0800 restoring indexes for collection suq.t1 from metadata2016-06-27T22:18:27.572+0800 finished restoring suq.t1 (1 document)2016-06-27T22:18:27.572+0800 done

同样如果mongodump使用了--oplog,那么在恢复的hi和必须使用--oplogRelay来应用oplog的内容,得到一个时间点快照.
三.使用mongoexport/mongoimport备份恢复
1.mongoexport
mongoexport/mongoimport可以用来将数据导出成json和csv格式,注意mongoexport并不能完全用来备份数据库,因为json只是bson的子集,有一些bson type对于json是不支持的.
如果你只是想拷贝数据库或者集合到其它的实例,可以使用copydb,clone或者clonecollection命令.详见官方文档的reference说明.
下面介绍mongoexport实例.在3.0以后移除了--csv选项,而使用--type=csv来代替.
[root@mongodb3 log]# mongoexport --helpUsage: mongoexport <options> Export data from MongoDB in CSV or JSON format. See http://docs.mongodb.org/manual/reference/program/mongoexport/ for more information. general options: --help print usage --version print the tool version and exit verbosity options: -v, --verbose=<level> more detailed log output (include multiple times for more verbosity, e.g. -vvvvv, or specify a numeric value, e.g. --verbose=N) --quiet hide all log output connection options: -h, --host=<hostname> mongodb host to connect to (setname/host1,host2 for replica sets) --port=<port> server port (can also use --host hostname:port) authentication options: -u, --username=<username> username for authentication -p, --password=<password> password for authentication --authenticationDatabase=<database-name> database that holds the user's credentials --authenticationMechanism=<mechanism> authentication mechanism to use namespace options: -d, --db=<database-name> database to use -c, --collection=<collection-name> collection to use output options: -f, --fields=<field>[,<field>]* comma separated list of field names (required for exporting CSV) e.g. -f "name,age" --fieldFile=<filename> file with field names - 1 per line --type=<type> the output format, either json or csv (defaults to 'json') -o, --out=<filename> output file; if not specified, stdout is used --jsonArray output to a JSON array rather than one object per line --pretty output JSON formatted to be human-readable querying options: -q, --query=<json> query filter, as a JSON string, e.g., '{x:{$gt:1}}' --queryFile=<filename> path to a file containing a query filter (JSON) -k, --slaveOk allow secondary reads if available (default true) --readPreference=<string>|<json> specify either a preference name or a preference json object --forceTableScan force a table scan (do not use $snapshot) --skip=<count> number of documents to skip --limit=<count> limit the number of documents to export --sort=<json> sort order, as a JSON string, e.g. '{x:1}'其中主要的参数:
--host:主机地址
--port:端口号
-u:用户名
-p:密码
-d:数据库名
-c:集合名
-f:列名,当--type为csv的时候必须指定
-o:文件名
-q:查询条件
下面是导出csv格式实例:

[root@mongodb3 test]# mongoexport --port 37017 -d suq -c t1 --type=csv -f name,age -o 1.csv2016-06-28T20:01:23.951+0800 connected to: localhost:370172016-06-28T20:01:23.951+0800 exported 2 records[root@mongodb3 test]# cat field.txt
name
age[root@mongodb3 test]# mongoexport --port 37017 -d suq -c t1 --type=csv --fieldFile field.txt -o 1.csv2016-06-28T20:08:55.504+0800 connected to: localhost:370172016-06-28T20:08:55.505+0800 exported 2 records默认导出为json格式,下面是导出json格式:
[root@mongodb3 test]# mongoexport --port 37017 -d suq -c t1 -o suq.json2016-06-28T20:12:11.756+0800 connected to: localhost:370172016-06-28T20:12:11.756+0800 exported 2 records[root@mongodb3 test]# cat suq.json {"_id":{"$oid":"577266483ef351522beaefe4"},"name":"tony","age":10.0}{"_id":{"$oid":"577266603ef351522beaefe5"},"name":"tony","age":10.0,"sex":1.0}导出部分数据,使用--query接查询条件:

[root@mongodb3 test]# mongoexport --port 37017 -d suq -c t1 --query '{"sex":1}' -o suq.json2016-06-28T20:14:00.656+0800 connected to: localhost:370172016-06-28T20:14:00.657+0800 exported 1 record2.mongoimport
mongoimport帮助说明:
[root@mongodb3 test]# mongoimport --helpUsage: mongoimport <options> <file> Import CSV, TSV or JSON data into MongoDB. If no file is provided, mongoimport reads from stdin. See http://docs.mongodb.org/manual/reference/program/mongoimport/ for more information. general options: --help print usage --version print the tool version and exit verbosity options: -v, --verbose=<level> more detailed log output (include multiple times for more verbosity, e.g. -vvvvv, or specify a numeric value, e.g. --verbose=N) --quiet hide all log output connection options: -h, --host=<hostname> mongodb host to connect to (setname/host1,host2 for replica sets) --port=<port> server port (can also use --host hostname:port) authentication options: -u, --username=<username> username for authentication -p, --password=<password> password for authentication --authenticationDatabase=<database-name> database that holds the user's credentials --authenticationMechanism=<mechanism> authentication mechanism to use namespace options: -d, --db=<database-name> database to use -c, --collection=<collection-name> collection to use input options: -f, --fields=<field>[,<field>]* comma separated list of field names, e.g. -f name,age --fieldFile=<filename> file with field names - 1 per line --file=<filename> file to import from; if not specified, stdin is used --headerline use first line in input source as the field list (CSV and TSV only) --jsonArray treat input source as a JSON array --type=<type> input format to import: json, csv, or tsv (defaults to 'json') ingest options: --drop drop collection before inserting documents --ignoreBlanks ignore fields with empty values in CSV and TSV --maintainInsertionOrder insert documents in the order of their appearance in the input source -j, --numInsertionWorkers=<number> number of insert operations to run concurrently (defaults to 1) --stopOnError stop importing at first insert/upsert error --upsert insert or update objects that already exist --upsertFields=<field>[,<field>]* comma-separated fields for the query part of the upsert --writeConcern=<write-concern-specifier> write concern options e.g. --writeConcern majority, --writeConcern '{w: 3, wtimeout: 500, fsync: true, j: true}' (defaults to 'majority') --bypassDocumentValidation bypass document validation

json格式的import:
[root@mongodb3 test]# mongoimport --port 37017 -d suq -c suq --file suq.json2016-06-28T20:40:25.148+0800 connected to: localhost:370172016-06-28T20:40:25.176+0800 imported 1 document

csv格式的import:
其中注意使用--headerline指定列的名字.
[root@mongodb3 test]# mongoimport --port 37017 --type=csv -d suq -c suq2 --headerline --file= 1.csv2016-06-28T20:45:28.430+0800 connected to: localhost:370172016-06-28T20:45:28.458+0800 imported 2 documents