這個是在之前的微博爬取（Python）–中大微博前100條微博內(nèi)容以及評論轉(zhuǎn)發(fā)點贊數(shù)目爬取
的并發(fā)版本

代碼

import requests from gevent import monkey import geventmonkey.patch_all(select=False) from pyquery import PyQuery as pqheaders = {'Host': 'm.weibo.cn','Referer': 'https://m.weibo.cn/u/1892723783?uid=1892723783&luicode=10000011&lfid=1076031892723783&featurecode=20000320','User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36','X-Requested-With': 'XMLHttpRequest','X-DevTools-Emulate-Network-Conditions-Client-Id': 'A20EA5B172E6DC82709D213A40AD0E8F' }def get_page(page):url = 'https://m.weibo.cn/api/container/getIndex?uid=1892723783&luicode=10000011&lfid=1076031892723783&featurecode=20000320&type=uid&value=1892723783&containerid=1076031892723783&page=%d' % pagetry:res = requests.get(url, headers=headers)if res.status_code == 200:return res.json()except requests.ConnectionError as e:print("Error", e.args)def parse_page(json):if json:items = json.get('data').get('cards')for item in items:item = item.get('mblog')weibo = {}weibo['text'] = pq(item.get('text')).text()weibo['attitudes'] = item.get('attitudes_count')weibo['comments'] = item.get('comments_count')weibo['reposts'] = item.get('reposts_count')yield weibodef oper(page):global datajson = get_page(page)results = parse_page(json)count = 0for res in results:data[page * 20 + count] = '\n'.join([res['text'], '【評論數(shù): ' + str(res['comments']) + ' 轉(zhuǎn)發(fā)數(shù): ' + str(res['reposts']) + ' 點贊數(shù): ' + str(res['attitudes']) + '】\n\n'])count += 1if __name__ == '__main__':data = {}gevent.joinall([gevent.spawn(oper, page) for page in range(1, 11)])with open('weibo.txt', 'w', encoding='utf-8') as f:f.write(''.join(data.values()))

總結(jié)

以上是生活随笔為你收集整理的多协程爬取中大微博内容（以及转发数，点赞数，评论数）的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。