http://www.2cto.com/database/201309/240552.html

MongoDB分布式操作——分片操作 描述： 像其它分布式數據庫一樣，MongoDB同樣支持分布式操作，且MongoDB將分布式已經集成到數據庫中，其分布式體系如下圖所示： 所謂的片，其實就是一個單獨的MongoDB進程，它可以運行在不同的節點上，也可以運行在同一節點但監聽不同的端口。同一個集合的數據可能在每個片中都存儲的有數據，但它們在邏輯上整合成一個整體。分片的依據往往是根據表中的某一列或者是集合中的某一 組屬性。且分片是系統自動的，但是分片的依據需要用戶自己指定，分片的依據稱之為“片鍵”。常見的片鍵選擇為“隨機片鍵”或者是“遞增片鍵”。如果使用遞增片鍵，數據的分布則會按照一定的順序來寫入數據，即當一個節點寫滿才會寫入到下一個片鍵。而如果使用隨機片鍵，數據則可能在任何一個片中都有數據。 對于分布式中片的管理（片與片之間的通信），MongoDB中通過mongos進程來進行管理，客戶端對數據的操作，都由mongos進程來處理。對于節點、分片的管理，MongoDB由配置服務器來管理，配置服務器其實也是一個單獨MongoDB進程。 那么，一般在什么時候使用分片呢？在單個節點的磁盤不足時會分片，另外，當單個mongod不能滿足寫數據的性能要求時，可以使用分片；當單臺服務器內存太小，將大量的數據放到內存中速度會很慢，為了提高性能，我們可以使用分片操作。 以下是對于MongoDB分片操作的實驗： 操作環境： Win8 ?64位操作系統，虛擬機CentOS 32位操作系統。MongoDB 2.4.5 實驗步驟： （1）啟動配置服務器（放置一些元數據）： [plain]? [root@h3 dbs]# mongod -dbpath /var/lib/mongo/dbs/config --nojournal --port 20000 ? ? Mon Aug 12 13:36:25.999 [initandlisten] MongoDB starting : pid=22951 port=20000 ? dbpath=/var/lib/mongo/dbs/config 32-bit host=h3 ? Mon Aug 12 13:36:26.001 [initandlisten] ? Mon Aug 12 13:36:26.002 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary ?. ? Mon Aug 12 13:36:26.003 [initandlisten] ** ? ? ? 32 bit builds are limited to le ?ss than 2GB of data (or less with --journal). ? Mon Aug 12 13:36:26.004 [initandlisten] ** ? ? ? Note that journaling defaults t ?o off for 32 bit and is currently off. ? Mon Aug 12 13:36:26.005 [initandlisten] ** ? ? ? See http://dochub.mongodb.org/c ?ore/32bit ? Mon Aug 12 13:36:26.005 [initandlisten] ? Mon Aug 12 13:36:26.006 [initandlisten] db version v2.4.5 ? Mon Aug 12 13:36:26.007 [initandlisten] git version: a2ddc68ba7c9cee17bfe69ed840 ?383ec3506602b ? Mon Aug 12 13:36:26.007 [initandlisten] build info: Linux bs-linux32.10gen.cc 2. ?6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49 ? Mon Aug 12 13:36:26.008 [initandlisten] allocator: system ? Mon Aug 12 13:36:26.009 [initandlisten] options: { dbpath: "/var/lib/mongo/dbs/c ?onfig", nojournal: true, port: 20000 } ? Mon Aug 12 13:36:26.071 [FileAllocator] allocating new datafile /var/lib/mongo/d ?bs/config/local.ns, filling with zeroes... ? Mon Aug 12 13:36:26.072 [FileAllocator] creating directory /var/lib/mongo/dbs/co ?nfig/_tmp ? Mon Aug 12 13:36:26.582 [FileAllocator] done allocating datafile /var/lib/mongo/ ?dbs/config/local.ns, size: 16MB, ?took 0.504 secs ? Mon Aug 12 13:36:26.585 [FileAllocator] allocating new datafile /var/lib/mongo/d ?bs/config/local.0, filling with zeroes... ? Mon Aug 12 13:36:27.725 [FileAllocator] done allocating datafile /var/lib/mongo/ ?dbs/config/local.0, size: 16MB, ?took 1.138 secs ? Mon Aug 12 13:36:27.757 [initandlisten] command local.$cmd command: { create: "s ?tartup_log", size: 10485760, capped: true } ntoreturn:1 keyUpdates:0 ?reslen:37 ? 1714ms ? Mon Aug 12 13:36:27.764 [websvr] admin web console waiting for connections on po ?rt 21000 ? Mon Aug 12 13:36:27.771 [initandlisten] waiting for connections on port 20000 ? （2）啟動mongos： [plain]? [root@h3 ~]# mongos --port 30000 ?--configdb localhost:20000 ? Mon Aug 12 13:40:13.042 warning: running with 1 config server should be done only for testing purposes and is not recommended for production ? Mon Aug 12 13:40:13.075 [mongosMain] MongoS version 2.4.5 starting: pid=22968 port=30000 32-bit host=h3 (--help for usage) ? Mon Aug 12 13:40:13.076 [mongosMain] git version: a2ddc68ba7c9cee17bfe69ed840383ec3506602b ? Mon Aug 12 13:40:13.076 [mongosMain] build info: Linux bs-linux32.10gen.cc 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49 ? Mon Aug 12 13:40:13.076 [mongosMain] options: { configdb: "localhost:20000", port: 30000 } ? Mon Aug 12 13:40:13.250 [LockPinger] creating distributed lock ping thread for localhost:20000 and process h3:30000:1376286013:1804289383 (sleeping for 30000ms) ? Mon Aug 12 13:40:17.105 [mongosMain] distributed lock 'configUpgrade/h3:30000:1376286013:1804289383' acquired, ts : 52087541ff5991dd77979f2c ? Mon Aug 12 13:40:17.126 [mongosMain] starting upgrade of config server from v0 to v4 ? Mon Aug 12 13:40:17.128 [mongosMain] starting next upgrade step from v0 to v4 ? Mon Aug 12 13:40:17.130 [mongosMain] about to log new metadata event: { _id: "h3-2013-08-12T05:40:17-52087541ff5991dd77979f2d", server: "h3", clientAddr: "N/A", time: new Date(1376286017130), what: "starting upgrade of config database", ns: "config.version", details: { from: 0, to: 4 } } ? Mon Aug 12 13:40:17.138 [mongosMain] writing initial config version at v4 ? Mon Aug 12 13:40:17.232 [mongosMain] about to log new metadata event: { _id: "h3-2013-08-12T05:40:17-52087541ff5991dd77979f2f", server: "h3", clientAddr: "N/A", time: new Date(1376286017232), what: "finished upgrade of config database", ns: "config.version", details: { from: 0, to: 4 } } ? Mon Aug 12 13:40:17.237 [mongosMain] upgrade of config server to v4 successful ? Mon Aug 12 13:40:17.244 [mongosMain] distributed lock 'configUpgrade/h3:30000:1376286013:1804289383' unlocked. ? Mon Aug 12 13:40:17.474 [websvr] admin web console waiting for connections on port 31000 ? Mon Aug 12 13:40:17.476 [Balancer] about to contact config servers and shards ? Mon Aug 12 13:40:17.492 [Balancer] config servers and shards contacted successfully ? Mon Aug 12 13:40:17.494 [Balancer] balancer id: h3:30000 started at Aug 12 13:40:17 ? Mon Aug 12 13:40:17.527 [mongosMain] waiting for connections on port 30000 ? （3）添加分片1： [plain]? [root@h3 ~]# cd /var/lib/mongo/dbs/ ? [root@h3 dbs]# mkdir shard1 ? [root@h3 dbs]# mongod --dbpath /var/lib/mongo/dbs/shard1 ?--port 10000 --nojournal ? Mon Aug 12 13:43:10.560 [initandlisten] MongoDB starting : pid=23016 port=10000 ?? dbpath=/var/lib/mongo/dbs/shard1 32-bit host=h3 ? Mon Aug 12 13:43:10.562 [initandlisten] ? Mon Aug 12 13:43:10.562 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary. ? Mon Aug 12 13:43:10.563 [initandlisten] ** ? ? ? 32 bit builds are limited to less than 2GB of data (or less with --journal). ? Mon Aug 12 13:43:10.564 [initandlisten] ** ? ? ? Note that journaling defaults to off for 32 bit and is currently off. ? Mon Aug 12 13:43:10.565 [initandlisten] ** ? ? ? See http://dochub.mongodb.org/core/32bit ? Mon Aug 12 13:43:10.566 [initandlisten] ? Mon Aug 12 13:43:10.566 [initandlisten] db version v2.4.5 ? Mon Aug 12 13:43:10.567 [initandlisten] git version: a2ddc68ba7c9cee17bfe69ed840383ec3506602b ? Mon Aug 12 13:43:10.568 [initandlisten] build info: Linux bs-linux32.10gen.cc 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49 ? Mon Aug 12 13:43:10.568 [initandlisten] allocator: system ? Mon Aug 12 13:43:10.569 [initandlisten] options: { dbpath: "/var/lib/mongo/dbs/shard1", nojournal: true, port: 10000 } ? Mon Aug 12 13:43:10.626 [FileAllocator] allocating new datafile /var/lib/mongo/dbs/shard1/local.ns, filling with zeroes... ? Mon Aug 12 13:43:10.628 [FileAllocator] creating directory /var/lib/mongo/dbs/shard1/_tmp ? Mon Aug 12 13:43:11.547 [FileAllocator] done allocating datafile /var/lib/mongo/dbs/shard1/local.ns, size: 16MB, ?took 0.907 secs ? Mon Aug 12 13:43:11.549 [FileAllocator] allocating new datafile /var/lib/mongo/dbs/shard1/local.0, filling with zeroes... ? Mon Aug 12 13:43:13.339 [FileAllocator] done allocating datafile /var/lib/mongo/dbs/shard1/local.0, size: 16MB, ?took 1.787 secs ? Mon Aug 12 13:43:13.427 [initandlisten] command local.$cmd command: { create: "startup_log", size: 10485760, capped: true } ntoreturn:1 keyUpdates:0 ?reslen:372801ms ? Mon Aug 12 13:43:13.472 [initandlisten] waiting for connections on port 10000 ? Mon Aug 12 13:43:13.497 [websvr] admin web console waiting for connections on port 11000 ? （4）添加分片2： [plain]? [root@h3 dbs]# cd /var/lib/mongo/dbs/ ? [root@h3 dbs]# mkdir shard2 ? [root@h3 dbs]# mongod --dbpath /var/lib/mongo/dbs/shard2 --port 10001 --nojournal ? Mon Aug 12 13:48:25.549 [initandlisten] MongoDB starting : pid=23129 port=10001 dbpath=/var/lib/mongo/dbs/shard2 32-bit host=h3 ? Mon Aug 12 13:48:25.550 [initandlisten] ? Mon Aug 12 13:48:25.550 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary. ? Mon Aug 12 13:48:25.550 [initandlisten] ** ? ? ? 32 bit builds are limited to less than 2GB of data (or less with --journal). ? Mon Aug 12 13:48:25.550 [initandlisten] ** ? ? ? Note that journaling defaults to off for 32 bit and is currently off. ? Mon Aug 12 13:48:25.550 [initandlisten] ** ? ? ? See http://dochub.mongodb.org/core/32bit ? Mon Aug 12 13:48:25.551 [initandlisten] ? Mon Aug 12 13:48:25.551 [initandlisten] db version v2.4.5 ? Mon Aug 12 13:48:25.552 [initandlisten] git version: a2ddc68ba7c9cee17bfe69ed840383ec3506602b ? Mon Aug 12 13:48:25.552 [initandlisten] build info: Linux bs-linux32.10gen.cc 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49 ? Mon Aug 12 13:48:25.552 [initandlisten] allocator: system ? Mon Aug 12 13:48:25.552 [initandlisten] options: { dbpath: "/var/lib/mongo/dbs/shard2", nojournal: true, port: 10001 } ? Mon Aug 12 13:48:25.570 [FileAllocator] allocating new datafile /var/lib/mongo/dbs/shard2/local.ns, filling with zeroes... ? Mon Aug 12 13:48:25.572 [FileAllocator] creating directory /var/lib/mongo/dbs/shard2/_tmp ? Mon Aug 12 13:48:26.956 [FileAllocator] done allocating datafile /var/lib/mongo/dbs/shard2/local.ns, size: 16MB, ?took 1.378 secs ? Mon Aug 12 13:48:26.957 [FileAllocator] allocating new datafile /var/lib/mongo/dbs/shard2/local.0, filling with zeroes... ? Mon Aug 12 13:48:28.023 [FileAllocator] done allocating datafile /var/lib/mongo/dbs/shard2/local.0, size: 16MB, ?took 1.065 secs ? Mon Aug 12 13:48:28.038 [initandlisten] command local.$cmd command: { create: "startup_log", size: 10485760, capped: true } ntoreturn:1 keyUpdates:0 ?reslen:37 2469ms ? Mon Aug 12 13:48:28.041 [websvr] admin web console waiting for connections on port 11001 ? Mon Aug 12 13:48:28.042 [initandlisten] waiting for connections on port 10001 ? （5）添加分片3： [plain]? [root@h3 ~]# cd /var/lib/mongo/dbs/ ? [root@h3 dbs]# mkdir shard3 ? [root@h3 dbs]# mongod --dbpath /var/lib/mongo/dbs/shard3 --port 10002 --nojournal ? Mon Aug 12 13:50:43.735 [initandlisten] MongoDB starting : pid=23142 port=10002 dbpath=/var/lib/mongo/dbs/shard3 32-bit host=h3 ? Mon Aug 12 13:50:43.737 [initandlisten] ? Mon Aug 12 13:50:43.737 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary. ? Mon Aug 12 13:50:43.738 [initandlisten] ** ? ? ? 32 bit builds are limited to less than 2GB of data (or less with --journal). ? Mon Aug 12 13:50:43.739 [initandlisten] ** ? ? ? Note that journaling defaults to off for 32 bit and is currently off. ? Mon Aug 12 13:50:43.739 [initandlisten] ** ? ? ? See http://dochub.mongodb.org/core/32bit ? Mon Aug 12 13:50:43.740 [initandlisten] ? Mon Aug 12 13:50:43.740 [initandlisten] db version v2.4.5 ? Mon Aug 12 13:50:43.740 [initandlisten] git version: a2ddc68ba7c9cee17bfe69ed840383ec3506602b ? Mon Aug 12 13:50:43.740 [initandlisten] build info: Linux bs-linux32.10gen.cc 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49 ? Mon Aug 12 13:50:43.740 [initandlisten] allocator: system ? Mon Aug 12 13:50:43.741 [initandlisten] options: { dbpath: "/var/lib/mongo/dbs/shard3", nojournal: true, port: 10002 } ? Mon Aug 12 13:50:43.756 [FileAllocator] allocating new datafile /var/lib/mongo/dbs/shard3/local.ns, filling with zeroes... ? Mon Aug 12 13:50:43.756 [FileAllocator] creating directory /var/lib/mongo/dbs/shard3/_tmp ? Mon Aug 12 13:50:45.079 [FileAllocator] done allocating datafile /var/lib/mongo/dbs/shard3/local.ns, size: 16MB, ?took 1.32 secs ? Mon Aug 12 13:50:45.099 [FileAllocator] allocating new datafile /var/lib/mongo/dbs/shard3/local.0, filling with zeroes... ? Mon Aug 12 13:50:45.948 [FileAllocator] done allocating datafile /var/lib/mongo/dbs/shard3/local.0, size: 16MB, ?took 0.848 secs ? Mon Aug 12 13:50:45.957 [initandlisten] command local.$cmd command: { create: "startup_log", size: 10485760, capped: true } ntoreturn:1 keyUpdates:0 ?reslen:37 2202ms ? Mon Aug 12 13:50:45.960 [initandlisten] waiting for connections on port 10002 ? Mon Aug 12 13:50:45.961 [websvr] admin web console waiting for connections on port 11002 ? （6）添加剛剛啟動的三個片： [plain]? [root@h3 dbs]# mongo localhost:30000/admin ? MongoDB shell version: 2.4.5 ? connecting to: localhost:30000/admin ? mongos> db.runCommand({addshard : "localhost:10000", allowLocal : true}); ? { "shardAdded" : "shard0000", "ok" : 1 } ? mongos> db.runCommand({addshard : "localhost:10001", allowLocal : true}); ? { "shardAdded" : "shard0001", "ok" : 1 } ? mongos> db.runCommand({addshard : "localhost:10002", allowLocal : true}); ? { "shardAdded" : "shard0002", "ok" : 1 } ? mongos> ? （7）打開數據庫foo的分片功能： [plain]? mongos> db.runCommand({"enablesharding" : "foo"}); ? { "ok" : 1 } ? mongos> ? （8）打開數據庫foo中數據集合bar的分片功能： [plain]? mongos> db.runCommand({"shardcollection" : "foo.bar", "key" : {"_id" : 1}}); ? { "collectionsharded" : "foo.bar", "ok" : 1 } ? mongos> ? （9）查看所有的分片： [plain]? mongos> use config ? switched to db config ? mongos> db.shards.find(); ? { "_id" : "shard0000", "host" : "localhost:10000" } ? { "_id" : "shard0001", "host" : "localhost:10001" } ? { "_id" : "shard0002", "host" : "localhost:10002" } ? mongos> ? （10）查看databases集合： [plain]? mongos> db.databases.find(); ? { "_id" : "admin", "partitioned" : false, "primary" : "config" } ? { "_id" : "foo", "partitioned" : true, "primary" : "shard0000" } ? mongos> ? （11） 查看chucks集合： [plain]? mongos> db.chunks.find(); ? { "_id" : "foo.bar-_id_MinKey", "lastmod" : Timestamp(1, 0), "lastmodEpoch" : ObjectId("52087d0a1072e2912d4242ea"), "ns" : "foo.bar", "min" : { "_id" : { "$minKey" : 1 } }, "max" : { "_id" : { "$maxKey" : 1 } }, "shard" : "shard0000" } ? mongos> ? （12）獲得概要信息： [plain]? mongos> db.printShardingStatus(); ? --- Sharding Status --- ? sharding version: { ? "_id" : 1, ? "version" : 3, ? "minCompatibleVersion" : 3, ? "currentVersion" : 4, ? "clusterId" : ObjectId("52087bf41072e2912d4242b8") ? } ? shards: ? { ?"_id" : "shard0000", ?"host" : "localhost:10000" } ? { ?"_id" : "shard0001", ?"host" : "localhost:10001" } ? { ?"_id" : "shard0002", ?"host" : "localhost:10002" } ? databases: ? { ?"_id" : "admin", ?"partitioned" : false, ?"primary" : "config" } ? { ?"_id" : "foo", ?"partitioned" : true, ?"primary" : "shard0000" } ? foo.bar ? shard key: { "_id" : 1 } ? chunks: ? shard0000 ? ? ? 1 ? { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard0000 Timestamp(1, 0) ? mongos> ? （13）移除分片： [plain]? mongos> use admin ? switched to db admin ? mongos> db.runCommand({"removeshard" : "localhost:10000"}); ? { ? "msg" : "draining started successfully", ? "state" : "started", ? "shard" : "shard0000", ? "note" : "you need to drop or movePrimary these databases", ? "dbsToMove" : [ ? "foo" ? ], ? "ok" : 1 ? } ? mongos> ? 總結：對MongoDB的分片操作執行如下順序： （1）啟動配置服務器； （2）啟動mongos進程； （3）添加mongo實例（片）； （4）對數據庫或者數據集使用分片操作。

總結

以上是生活随笔為你收集整理的MongoDB分布式操作——分片操作的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。