?

幫朋友抓了一些代理IP，并根據測試聯的通性，放在了不通的文件夾下。特將源碼分享

注意：

1，環境Python3.5

2，安裝BeautifulSoup4 ?requests

?

代碼如下:

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66

#-*-?coding:gb18030?-*- from?bs4?import?BeautifulSoup import?requests import?time import?os,sys all_url_add={ ??????'url2':'http://ip84.com/gn/', ??????} def?func(url): ????r?=?requests.get(url) ????content?=?r.text ????soup?=?BeautifulSoup(content,?"html.parser") ????ListTable?=?soup.find_all("table",?class_="list") ????for?table?in?ListTable: ????????ListTr?=?table.find_all("tr") ????????for?tr?in?ListTr: ????????????try: ????????????????ListTd?=?tr.find_all("td") ????????????????ipaddress?=?str(ListTd[0].get_text()).strip() ????????????????port?=?str(ListTd[1].get_text()).strip() ????????????????city?=?str(ListTd[2].get_text()).strip().replace("\n",?"") ????????????????leixing?=?str(ListTd[3].get_text()).strip() ????????????????xieyi?=?str(ListTd[4].get_text()).strip() ????????????????shudu?=?str(ListTd[5].get_text()).strip() ????????????????time1?=?str(ListTd[6].get_text()).strip() ????????????????f?=?open("ip"?+?'.txt',?'a') ????????????????f.write(ipaddress+":"+port+'\n') ????????????????f.close() ????????????????print('地址:'+ipaddress?+?"端口:"?+?port?+?"地區:"?+?city?+?"類型:"?+?leixing?+?"協議"?+?xieyi?+?"速度"?+?shudu?+?"時間:"?+?time1) ????????????except?Exception?as?e: ????????????????print?(u"-------------------程序異常-----------------------") ????????return?'success' ????print?(u'本頁抓取結束，正在跳轉下一頁') def?pin(): ????f2?=?open('ip.txt',?'r') ????count?=?len(open('ip.txt',?'rU').readlines()) ????for?x?in?range(count): ????????ip?=?f2.readline().split(':')[0] ????????return1?=?os.system('ping?-n?5?-w?5?%s'?%?ip)?? ????????if?return1: ????????????print('測試失敗') ????????else: ????????????print('測試成功，正在寫入新文件') ????????????f3?=?open('SuccessIp.txt',?'a') ????????????f3.write(f2.readline()?+?'\n') ????????????f3.close() ????f2.close() ????print('程序結束，可用IP已放在SuccessIp中') if??__name__=='__main__': ????????for?x?in?all_url_add: ????????????print?(x) ????????????for?y?in?range(1,50): ????????????????url=all_url_add[x]+str(y) ????????????????print?(url) ????????????????status=func(url) ????????????????if?status=='success': ????????????????????print(y,'頁結束') ????????print?(u'****程序抓取運行結束,正在檢查所得IP連通性，請勿關閉窗口*****') ????????pin()

?

有點亂，有時間將數據存儲在數據庫，再將這個功能集成在博客當中。

?

?

Rex博客保留所有權利

轉載于:https://www.cnblogs.com/rexyan/p/6212923.html

總結

以上是生活随笔為你收集整理的Python 爬虫抓取代理IP，并检测联通性的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。