【python】使用爬虫爬取动漫之家漫画更新信息
生活随笔
收集整理的這篇文章主要介紹了
【python】使用爬虫爬取动漫之家漫画更新信息
小編覺得挺不錯(cuò)的,現(xiàn)在分享給大家,幫大家做個(gè)參考.
網(wǎng)站名稱為:
https://manhua.dmzj.com/update_1.shtml本篇僅在于交流學(xué)習(xí)
1.首先將相應(yīng)的庫(kù)導(dǎo)入:
import re import requests from bs4 import BeautifulSoup import pandas as pd2.截取網(wǎng)站信息進(jìn)行分析:(此處是網(wǎng)頁的第一頁)
a = requests.get('https://manhua.dmzj.com/update_1.shtml') html = a.content soup = BeautifulSoup(html, 'html.parser') print(soup)效果如下:
3.截取信息,通過正則表達(dá)式進(jìn)行網(wǎng)頁內(nèi)容截取:
findnewlink = re.compile(r'<a href="(.*?)"',re.S) #最新更新連接 findtitle = re.compile(r'title="(.*)"') #作品標(biāo)題 findname = re.compile(r'<span class="gray12">(.*)</span></li>') findle = re.compile(r'<li>(.*)</li>') #狀態(tài) findtime1 = re.compile(r'<li class="numfont"><span class="color_red">(.*)</span></li>')#今天更新作品時(shí)間 #findtime2 = re.compile(r'<li class="numfont">(.*)</li>') #最近更新時(shí)間?4.循環(huán)獲取:
for i in soup.find_all('div',class_='pictext'):#print(i) #測(cè)試全部信息#save = []i = str(i)link1 = re.findall(findnewlink,i)[0]newlink = "http://manhua.dmzj.com" + link1s = link1.rpartition('/')link = "http://manhua.dmzj.com"+s[0] + s[1]titie = re.findall(findtitle,i)[0]name = re.findall(findname,i)[0]le = re.findall(findle,i)[0]time1 = re.findall(findtime1,i)[0]#time2 = re.findall(findtime2,i)5.輸出內(nèi)容:
print(titie)print(name)print(le)print(time1)print(link)print(newlink)#print(time2)6.設(shè)置csv文件內(nèi)容:
savecsv = {'mhtitle':[],'mhname':[],'mhle':[],'mhtime1':[],'mhlink':[],'mhnewlink':[]} savecsv['mhtitle'].append(titie)savecsv['mhname'].append(name)savecsv['mhle'].append(le)savecsv['mhtime1'].append(time1)savecsv['mhlink'].append(link)savecsv['mhnewlink'].append(newlink)?7.保存至csv文件:
save = pd.DataFrame(savecsv) save.to_csv("漫畫更新.csv", encoding="utf_8_sig")?8.輸出:
9.完整代碼:
#漫畫網(wǎng)站爬取 import re import requests from bs4 import BeautifulSoup import pandas as pda = requests.get('https://manhua.dmzj.com/update_1.shtml') html = a.content soup = BeautifulSoup(html, 'html.parser') #print(soup)savecsv = {'mhtitle':[],'mhname':[],'mhle':[],'mhtime1':[],'mhlink':[],'mhnewlink':[]}findnewlink = re.compile(r'<a href="(.*?)"',re.S) #最新更新連接 findtitle = re.compile(r'title="(.*)"') #作品標(biāo)題 findname = re.compile(r'<span class="gray12">(.*)</span></li>') findle = re.compile(r'<li>(.*)</li>') #狀態(tài) findtime1 = re.compile(r'<li class="numfont"><span class="color_red">(.*)</span></li>')#今天更新作品時(shí)間 #findtime2 = re.compile(r'<li class="numfont">(.*)</li>') #最近更新時(shí)間 for i in soup.find_all('div',class_='pictext'):#print(i) #測(cè)試全部信息#save = []i = str(i)link1 = re.findall(findnewlink,i)[0]newlink = "http://manhua.dmzj.com" + link1s = link1.rpartition('/')link = "http://manhua.dmzj.com"+s[0] + s[1]titie = re.findall(findtitle,i)[0]name = re.findall(findname,i)[0]le = re.findall(findle,i)[0]time1 = re.findall(findtime1,i)[0]#time2 = re.findall(findtime2,i)print(titie)print(name)print(le)print(time1)print(link)print(newlink)#print(time2)savecsv['mhtitle'].append(titie)savecsv['mhname'].append(name)savecsv['mhle'].append(le)savecsv['mhtime1'].append(time1)savecsv['mhlink'].append(link)savecsv['mhnewlink'].append(newlink)save = pd.DataFrame(savecsv) save.to_csv("漫畫更新.csv", encoding="utf_8_sig")?
?
?
?
總結(jié)
以上是生活随笔為你收集整理的【python】使用爬虫爬取动漫之家漫画更新信息的全部?jī)?nèi)容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: 11-1-败者树-外部排序-第11章-《
- 下一篇: python处理换行符