有个在线的Mongo数据库，其中有个dialog的字表，集合。
想要导出，导入本地的Mongo中，用于开发测试。
mongo 导出
mongodb 备份、还原、导入、导出简单操作 – 开发笔记 – SegmentFault 思否
好像用：
mongodump
mongorestore
即可。
提到了：
mongoimport
MongoDB导入导出以及数据库备份 · Issue #22 · webplus/blog
mongoexport
mongoimport
和dump，restore有何区别？
【已解决】mongoimport/mongoexport和mongodump/mongorestore的区别
mongodb 备份 还原 导出 导入«海底苍鹰(tank)博客
Mongodb数据导出工具mongoexport和导入工具mongoimport介绍 – 青葱岁月 – ITeye博客
好像相对新的帖子，都是推荐：
mongoexport
mongoimport
mongodb export and import
Import and Export Data — MongoDB Compass stable
说的是：
MongoDB Compass
的工具，支持导入和导出，支持JSON和CSV格式
-》抽空去试试：MONGODB COMPASS
再去看看：
而关于备份，详见官网：
MongoDB Backup Methods — MongoDB Manual
-》
https://docs.mongodb.com/manual/core/backups/#back-up-with-mongodump
数据导入和导出 — MongoDB Manual
https://mongodb-documentation.readthedocs.io/en/latest/administration/import-export.html
参考具体的使用介绍：
https://docs.mongodb.com/manual/reference/program/mongoimport/#use
https://docs.mongodb.com/manual/reference/program/mongoexport/#use
然后去试试
➜ backup_restore mongoexport –help
Usage:
mongoexport <options>
Export data from MongoDB in CSV or JSON format.
See http://docs.mongodb.org/manual/reference/program/mongoexport/ for more information.
general options:
–help print usage
–version print the tool version and exit
verbosity options:
-v, –verbose=<level> more detailed log output (include multiple times for more verbosity, e.g. -vvvvv, or specify a numeric value,
e.g. –verbose=N)
–quiet hide all log output
connection options:
-h, –host=<hostname> mongodb host to connect to (setname/host1,host2 for replica sets)
–port=<port> server port (can also use –host hostname:port)
ssl options:
–ssl connect to a mongod or mongos that has ssl enabled
–sslCAFile=<filename> the .pem file containing the root certificate chain from the certificate authority
–sslPEMKeyFile=<filename> the .pem file containing the certificate and key
–sslPEMKeyPassword=<password> the password to decrypt the sslPEMKeyFile, if necessary
–sslCRLFile=<filename> the .pem file containing the certificate revocation list
–sslAllowInvalidCertificates bypass the validation for server certificates
–sslAllowInvalidHostnames bypass the validation for server name
–sslFIPSMode use FIPS mode of the installed openssl library
authentication options:
-u, –username=<username> username for authentication
-p, –password=<password> password for authentication
–authenticationDatabase=<database-name> database that holds the user’s credentials
–authenticationMechanism=<mechanism> authentication mechanism to use
namespace options:
-d, –db=<database-name> database to use
-c, –collection=<collection-name> collection to use
uri options:
–uri=mongodb-uri mongodb uri connection string
output options:
-f, –fields=<field>[,<field>]* comma separated list of field names (required for exporting CSV) e.g. -f "name,age"
–fieldFile=<filename> file with field names – 1 per line
–type=<type> the output format, either json or csv (defaults to ‘json’) (default: json)
-o, –out=<filename> output file; if not specified, stdout is used
–jsonArray output to a JSON array rather than one object per line
–pretty output JSON formatted to be human-readable
–noHeaderLine export CSV data without a list of field names at the first line
querying options:
-q, –query=<json> query filter, as a JSON string, e.g., ‘{x:{$gt:1}}’
–queryFile=<filename> path to a file containing a query filter (JSON)
-k, –slaveOk allow secondary reads if available (default true) (default: false)
–readPreference=<string>|<json> specify either a preference name or a preference json object
–forceTableScan force a table scan (do not use $snapshot)
–skip=<count> number of documents to skip
–limit=<count> limit the number of documents to export
–sort=<json> sort order, as a JSON string, e.g. ‘{x:1}’
–assertExists if specified, export fails if the collection does not exist (default: false)
然后去试试：
➜ backup_restore mongoexport -h ip_address –port PORT -u root -p password -d dialog -o mongodb_dialog.json
2018-06-29T11:38:29.852+0800 error connecting to db server: server returned error on SASL authentication step: Authentication failed.
➜ backup_restore mongoexport -h ip_address –port PORT -u root -p password –authenticationDatabase dialog -d dialog -o mongodb_dialog.json
2018-06-29T11:42:31.022+0800 error connecting to db server: server returned error on SASL authentication step: Authentication failed.
➜ backup_restore mongoexport -h ip_address –port PORT -u dialpg -p password –authenticationDatabase dialog -d dialog -o mongodb_dialog.json
2018-06-29T11:42:58.890+0800 error connecting to db server: server returned error on SASL authentication step: Authentication failed.
➜ backup_restore mongoexport -h ip_address –port PORT -u dialog -p password –authenticationDatabase dialog -d dialog -o mongodb_dialog.json
2018-06-29T11:43:06.318+0800 error validating settings: must specify a collection
2018-06-29T11:43:06.319+0800 try ‘mongoexport –help’ for more information
最后用：
mongoexport -h IP –port PORT -u dialog -p PASSWORD –authenticationDatabase dialog -d dialog -c dialog -o mongodb_dialog.json
可以了：
但是却一直是0%
➜ backup_restore mongoexport -h IP –port PORT -u dialog -p pwd –authenticationDatabase dialog -d dialog -c dialog -o mongodb_dialog.json
2018-06-29T11:44:45.057+0800 connected to: ip:port
2018-06-29T11:44:46.018+0800 [……………………] dialog.dialog 0/41714 (0.0%)
2018-06-29T11:46:32.022+0800 [……………………] dialog.dialog 0/41714 (0.0%)
2018-06-29T11:46:33.018+0800 [……………………] dialog.dialog 0/41714 (0.0%)
2018-06-29T11:46:34.020+0800 [……………………] dialog.dialog 0/41714 (0.0%)
^C2018-06-29T11:46:34.689+0800 signal ‘interrupt’ received; forcefully terminating
先去看看：
error validating settings: must specify a collection
mongoimport – MongoDB – validating settings error – Stack Overflow
Could not export whole database using MongoDB in Ubuntu – Stack Overflow
mongoexport 0.0%
[TOOLS-1609] mongoexport fails on mac os 10.12.3 – MongoDB
➜ backup_restore mongo –version
MongoDB shell version v3.6.3
git version: 9586e557d54ef70f9ca4b43c26892cd55257e1a5
OpenSSL version: OpenSSL 1.0.2o 27 Mar 2018
allocator: system
modules: none
build environment:
distarch: x86_64
target_arch: x86_64
➜ backup_restore mongoexport –version
mongoexport version: 3.6.3
git version: homebrew
Go version: go1.10
os: darwin
arch: amd64
compiler: gc
OpenSSL version: OpenSSL 1.0.2n 7 Dec 2017
此处版本是3.6.3，大于3.2.10，不存在兼容性问题。
好像继续多等等，就可以了？
mongodb Failed: error connecting to db server: no reachable servers – Stack Overflow
error connecting to db server: no reachable servers
不过想到了：
还是去在线服务器中导出，然后压缩导出的json文件，再下载文件，效率更高些。
结果发现在线服务器的本地中去export，速度也很慢：
-》或许是因为的确数据太多了，实际不慢但显得很慢？
的确是，稍等几十秒就可可以看到进度了：
[root@xx-general-01 mongo_backup_restore]# mongoexport –port pport -u dialog -p pwd –authenticationDatabase dialog -d dialog -c dialog -o mongodb_dialog_dialog.json
2018-06-29T13:31:20.841+0800 connected to: localhost:PORT
2018-06-29T13:31:21.841+0800 [……………………] dialog.dialog 0/41714 (0.0%)
…
2018-06-29T13:31:51.841+0800 [####………………..] dialog.dialog 8000/41714 (19.2%)…
2018-06-29T13:32:18.841+0800 [####………………..] dialog.dialog 8000/41714 (19.2%)
2018-06-29T13:32:19.841+0800 [#########……………] dialog.dialog 16000/41714 (38.4%)
…
2018-06-29T13:32:24.841+0800 [##################……] dialog.dialog 32000/41714 (76.7%)
…32000/41714 (76.7%)
2018-06-29T13:32:49.841+0800 [##################……] dialog.dialog 32000/41714 (76.7%)
2018-06-29T13:32:50.291+0800 [########################] dialog.dialog 41714/41714 (100.0%)
2018-06-29T13:32:50.291+0800 exported 41714 records
果然是，导出后，500多MB：
[root@xx-general-01 mongo_backup_restore]# ll -ha
total 584M
drwxr-xr-x 2 root root 4.0K Jun 29 13:31 .
drwxr-xr-x 3 root root 4.0K Jun 29 13:31 ..
-rw-r–r– 1 root root 584M Jun 29 13:32 mongodb_dialog_dialog.json
去压缩，再下载：
-rw-r–r– 1 root root 584M Jun 29 13:32 mongodb_dialog_dialog.json
[root@xx-general-01 mongo_backup_restore]# 7za a -t7z -r -bt mongodb_dialog_dialog.json.7z mongodb_dialog_dialog.json
7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs x64)
Scanning the drive:
1 file, 611756316 bytes (584 MiB)
Creating archive: mongodb_dialog_dialog.json.7z
Items to compress: 1
Files read from disk: 1
Archive size: 92200295 bytes (88 MiB)
Everything is Ok
[root@xx-general-01 mongo_backup_restore]# ll
total 687464
-rw-r–r– 1 root root 611756316 Jun 29 13:32 mongodb_dialog_dialog.json
-rw-r–r– 1 root root 92200295 Jun 29 13:39 mongodb_dialog_dialog.json.7z
[root@xx-general-01 mongo_backup_restore]# ll -lha
total 672M
drwxr-xr-x 2 root root 4.0K Jun 29 13:35 .
drwxr-xr-x 3 root root 4.0K Jun 29 13:31 ..
-rw-r–r– 1 root root 584M Jun 29 13:32 mongodb_dialog_dialog.json
-rw-r–r– 1 root root 88M Jun 29 13:39 mongodb_dialog_dialog.json.7z
[root@xx-general-01 mongo_backup_restore]# sz mongodb_dialog_dialog.json.7z
rz
Starting zmodem transfer. Press Ctrl+C to cancel.
Transferring mongodb_dialog_dialog.json.7z…
100% 90039 KB 119 KB/sec 00:12:36 0 Errors
然后去导入本地mongo数据库：
➜ backup_restore mongoimport –help
Usage:
mongoimport <options> <file>
Import CSV, TSV or JSON data into MongoDB. If no file is provided, mongoimport reads from stdin.
See http://docs.mongodb.org/manual/reference/program/mongoimport/ for more information.
general options:
–help print usage
–version print the tool version and exit
verbosity options:
-v, –verbose=<level> more detailed log output (include multiple times for more verbosity, e.g. -vvvvv, or specify a numeric value,
e.g. –verbose=N)
–quiet hide all log output
connection options:
-h, –host=<hostname> mongodb host to connect to (setname/host1,host2 for replica sets)
–port=<port> server port (can also use –host hostname:port)
ssl options:
–ssl connect to a mongod or mongos that has ssl enabled
–sslCAFile=<filename> the .pem file containing the root certificate chain from the certificate authority
–sslPEMKeyFile=<filename> the .pem file containing the certificate and key
–sslPEMKeyPassword=<password> the password to decrypt the sslPEMKeyFile, if necessary
–sslCRLFile=<filename> the .pem file containing the certificate revocation list
–sslAllowInvalidCertificates bypass the validation for server certificates
–sslAllowInvalidHostnames bypass the validation for server name
–sslFIPSMode use FIPS mode of the installed openssl library
authentication options:
-u, –username=<username> username for authentication
-p, –password=<password> password for authentication
–authenticationDatabase=<database-name> database that holds the user’s credentials
–authenticationMechanism=<mechanism> authentication mechanism to use
namespace options:
-d, –db=<database-name> database to use
-c, –collection=<collection-name> collection to use
uri options:
–uri=mongodb-uri mongodb uri connection string
input options:
-f, –fields=<field>[,<field>]* comma separated list of fields, e.g. -f name,age
–fieldFile=<filename> file with field names – 1 per line
–file=<filename> file to import from; if not specified, stdin is used
–headerline use first line in input source as the field list (CSV and TSV only)
–jsonArray treat input source as a JSON array
–parseGrace=<grace> controls behavior when type coercion fails – one of: autoCast, skipField, skipRow, stop (defaults to ‘stop’)
(default: stop)
–type=<type> input format to import: json, csv, or tsv (defaults to ‘json’) (default: json)
–columnsHaveTypes indicated that the field list (from –fields, –fieldsFile, or –headerline) specifies types; They must be in
the form of ‘<colName>.<type>(<arg>)’. The type can be one of: auto, binary, bool, date, date_go, date_ms,
date_oracle, double, int32, int64, string. For each of the date types, the argument is a datetime layout string.
For the binary type, the argument can be one of: base32, base64, hex. All other types take an empty argument.
Only valid for CSV and TSV imports. e.g. zipcode.string(), thumbnail.binary(base64)
ingest options:
–drop drop collection before inserting documents
–ignoreBlanks ignore fields with empty values in CSV and TSV
–maintainInsertionOrder insert documents in the order of their appearance in the input source
-j, –numInsertionWorkers=<number> number of insert operations to run concurrently (defaults to 1) (default: 1)
–stopOnError stop importing at first insert/upsert error
–mode=[insert|upsert|merge] insert: insert only. upsert: insert or replace existing documents. merge: insert or modify existing documents.
defaults to insert
–upsertFields=<field>[,<field>]* comma-separated fields for the query part when –mode is set to upsert or merge
–writeConcern=<write-concern-specifier> write concern options e.g. –writeConcern majority, –writeConcern ‘{w: 3, wtimeout: 500, fsync: true, j: true}’
–bypassDocumentValidation bypass document validation
然后去试试
➜ backup_restore mongoimport -d dialog -c dialog –file mongodb_dialog_dialog.json
2018-06-29T14:15:57.110+0800 connected to: localhost
2018-06-29T14:16:00.066+0800 [#…………………..] dialog.dialog 47.5MB/583MB (8.1%)
2018-06-29T14:16:03.070+0800 [####………………..] dialog.dialog 99.0MB/583MB (17.0%)
2018-06-29T14:16:06.079+0800 [######………………] dialog.dialog 157MB/583MB (26.9%)
2018-06-29T14:16:09.078+0800 [########…………….] dialog.dialog 212MB/583MB (36.3%)
2018-06-29T14:16:12.066+0800 [##########…………..] dialog.dialog 262MB/583MB (44.9%)
2018-06-29T14:16:15.097+0800 [#############………..] dialog.dialog 320MB/583MB (54.9%)
2018-06-29T14:16:18.066+0800 [###############………] dialog.dialog 371MB/583MB (63.5%)
2018-06-29T14:16:21.068+0800 [#################…….] dialog.dialog 417MB/583MB (71.4%)
2018-06-29T14:16:24.068+0800 [###################…..] dialog.dialog 474MB/583MB (81.2%)
2018-06-29T14:16:27.074+0800 [#####################…] dialog.dialog 521MB/583MB (89.3%)
2018-06-29T14:16:30.066+0800 [#######################.] dialog.dialog 571MB/583MB (97.9%)
2018-06-29T14:16:31.378+0800 [########################] dialog.dialog 583MB/583MB (100.0%)
2018-06-29T14:16:31.378+0800 imported 41714 documents
然后本地去看看数据如何：
先去用PyCharm中mongo插件看看
然后再去用MongoDB Compass看看：
【已解决】安装MongoDB Compass去图形化查看Mongo数据
【总结】
此处，就是去在线服务器中导出：
方式1: 本地连接在线host去导出
mongoexport -h IP –port PORT -u dialog -p pwd –authenticationDatabase dialog -d dialog -c dialog -o mongodb_dialog_dialog.json
方式2: 登录在线服务器去本地导出：
mongoexport –port PORT -u dialog -p pwd –authenticationDatabase dialog -d dialog -c dialog -o mongodb_dialog_dialog.json
后，然后去本地导入：
mongoimport -d dialog -c dialog –file mongodb_dialog_dialog.json
即可。
转载请注明：在路上 » 【已解决】同步导出在线Mongo中某个集合（表）到本地