為什么80%的碼農(nóng)都做不了架構(gòu)師？>>> ??

數(shù)據(jù)血緣通常是指數(shù)據(jù)產(chǎn)生的鏈路，其采集主要通過自動解析（存儲過程、SQL、ETL過程等文件）結(jié)合人工收集的方式實現(xiàn)。本文不涉及數(shù)據(jù)血緣如何獲取，只對如何通過python操作mongodb并可視化數(shù)據(jù)血緣關系提供一些思路。

首先通過pymongo連接本地數(shù)據(jù)庫，并插入測試數(shù)據(jù)

import pymongo myclient = pymongo.MongoClient("mongodb://127.0.0.1:27017/") mydb = myclient["world"] mycol = mydb["areas"] mylist = [ { "_id" : 1, "name" : "Asia" }, { "_id" : 2, "name" : "China", "belongsto" : "Asia" }, { "_id" : 3, "name" : "ZheJiang", "belongsto" : "China" }, { "_id" : 4, "name" : "HangZhou", "belongsto" : "ZheJiang" }, { "_id" : 5, "name" : "NingBo", "belongsto" : "ZheJiang" }, { "_id" : 6, "name" : "Xihu", "belongsto" : "HangZhou" } ] x = mycol.insert_many(mylist) for x in mycol.find():print(x)

輸出結(jié)果：?

{'_id': 1, 'name': 'Asia'}
{'_id': 2, 'name': 'China', 'belongsto': 'Asia'}
{'_id': 3, 'name': 'ZheJiang', 'belongsto': 'China'}
{'_id': 4, 'name': 'HangZhou', 'belongsto': 'ZheJiang'}
{'_id': 5, 'name': 'NingBo', 'belongsto': 'ZheJiang'}
{'_id': 6, 'name': 'Xihu', 'belongsto': 'HangZhou'}

遞歸查詢name='Xihu'這個節(jié)點的父節(jié)點

pipeline = [{'$graphLookup': {'from': "areas",'startWith': "$belongsto",'connectFromField': "belongsto",'connectToField': "name",'as': "belongHierarchy"}},{'$match': {'name' : 'Xihu'}}] for doc in (mycol.aggregate(pipeline)):print (doc)

輸出結(jié)果：?

{'_id': 6, 'name': 'Xihu', 'belongsto': 'HangZhou','reportingHierarchy':

[{'_id': 1, 'name': 'Asia'},

{'_id': 2, 'name': 'China', 'belongsto': 'Asia'},

{'_id': 3, 'name': 'ZheJiang', 'belongsto': 'China'},

{'_id': 4, 'name': 'HangZhou', 'belongsto': 'ZheJiang'}]}

解析輸出結(jié)果并可視化展示節(jié)點關系

import networkx as nx import matplotlib.pyplot as plt rs = list(mycol.aggregate(pipeline)) def get_relation(rs):G = nx.DiGraph()for node in rs:try:G.add_edge(node['name'], node['belongsto'])for item in node['belongHierarchy']:if 'belongsto' in item.keys():G.add_edge(item['name'], item['belongsto'])else:passexcept:passreturn G G = get_relation(rs) nx.draw(G, with_labels=True, font_weight='bold') plt.show()

展示area這個collection中所有的節(jié)點

pipeline = [{'$graphLookup': {'from': "areas",'startWith': "$belongsto",'connectFromField': "belongsto",'connectToField': "name",'as': "belongHierarchy"}}] rs = list(mycol.aggregate(pipeline)) def get_relation(rs):G = nx.DiGraph()for node in rs:try:G.add_edge(node['name'], node['belongsto'])for item in node['belongHierarchy']:if 'belongsto' in item.keys():G.add_edge(item['name'], item['belongsto'])else:passexcept:passreturn G G = get_relation(rs) nx.draw(G, with_labels=True, font_weight='bold') plt.show()

轉(zhuǎn)載于:https://my.oschina.net/aubao/blog/3035977

超強干貨來襲 云風專訪：近40年碼齡，通宵達旦的技術人生

總結(jié)

以上是生活随笔為你收集整理的python pymongo+networkx 实现mongo数据血缘关系可视化的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。