首先分析糗百頁面：

目標是獲取文字內容和文章鏈接ID，并保存到文件。

#!/usr/bin/env/python #coding:utf-8if __name__ == '__main__':from urllib import request,parseimport re#抓取,獲取源代碼url='https://www.qiushibaike.com/textnew/'headers={"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36"}req=request.Request(url,headers=headers)resp=request.urlopen(req)html=resp.read().decode('utf-8')if resp.getcode()!=200:print('url open failed!')#解析數據pattern_id=re.compile(r'<a href="/article/(\d+)".*class=.contentHerf. >')pattern_content=re.compile(r'<div class="content">.*?<span>(.*?)</span>',re.S)ids=re.findall(pattern_id,html)contents=re.findall(pattern_content,html)data={}for i in range(len(ids)):pattern=r'<br/>'content=re.sub(pattern,'\n', contents[i])data[ids[i]]=content#保存數據directory='spider_qiubai/'for key in data.keys():f=open(directory+key+'.txt','w')try:f.write(data[key])except Exception as e:print("write file "+key+" failed! "+e)else:print("write file "+key+" successfully!")finally:f.close()

　　運行結果：

轉載于:https://www.cnblogs.com/jasonhaven/p/7375503.html

總結

以上是生活随笔為你收集整理的小爬糗事百科的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。