爬蟲模板幾乎一個(gè)樣兒
提前安裝pandas用于存儲(chǔ)數(shù)據(jù)、requests處理URL資源、Bs4(BeautifulSoup4)提取數(shù)據(jù)、lxml讀取網(wǎng)頁(yè)

爬取鏈家二手房信息

# -*- coding: utf-8 -*- # @Author : LEHOSO # @FileName: Lianjia2.py # @Time : 2021/10/11 16:55import timeimport pandas as pd import requests from bs4 import BeautifulSoup# 表頭 header = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) ''AppleWebKit/537.36 (KHTML, like Gecko) ''Chrome/95.0.4638.54 Safari/537.36 Edg/95.0.1020.30' } a = []def get_info(url):wb_data = requests.get(url, headers=header)# 爬取整個(gè)網(wǎng)頁(yè)soup = BeautifulSoup(wb_data.text, 'lxml')# 網(wǎng)頁(yè)單個(gè)元素ranks = soup.select('div.totalPrice.totalPrice2 > span')title = soup.select('div.title > a')location = soup.select('div.flood > div > a')area = soup.select('div.flood > div > a:nth-child(3)')fllowInfo = soup.select('div.followInfo')# 存入進(jìn)列表for ranks, titles, locaitons, areas, fllowInfos in zip(ranks, title, location, area, fllowInfo):data = {'價(jià)格': ranks.get_text().strip(),'標(biāo)題': titles.get_text().strip(),'位置': locaitons.get_text().strip() + '-' + areas.get_text().strip(),'關(guān)注': fllowInfos.get_text().strip().split('/')[0],'距今發(fā)布日期': fllowInfos.get_text().strip().split('/')[1]}a.append(data)print(data)if __name__ == '__main__':# 網(wǎng)址路徑urls = ['https://cq.lianjia.com/ershoufang/']for url in urls:get_info(url)time.sleep(2)# pandas存入數(shù)據(jù)df_out = pd.DataFrame(a, columns=['價(jià)格', '標(biāo)題', '位置', '關(guān)注', '距今發(fā)布日期'])#導(dǎo)出為xlsx格式df_out.to_excel('aaa.xlsx')

總結(jié)

以上是生活随笔為你收集整理的爬取链家二手房信息【爬虫模板】的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。