Python爬取數據并寫入MySQL數據庫的實例

來源：中文源碼網????瀏覽： 次????日期：2018年9月2日

【下載文檔:??Python爬取數據并寫入MySQL數據庫的實例.txt?】

(友情提示:右鍵點上行txt文檔名->目標另存為)

Python爬取數據并寫入MySQL數據庫的實例 首先我們來爬取 http://html-color-codes.info/color-names/ 的一些數據。按 F12 或 ctrl+u 審查元素，結果如下:結構很清晰簡單，我們就是要爬 tr 標簽里面的 style 和 tr 下幾個并列的 td 標簽，下面是爬取的代碼:

#!/usr/bin/env python

# coding=utf-8

import requests

from bs4 import BeautifulSoup

import MySQLdb

print('連接到mysql服務器...')

db = MySQLdb.connect("localhost","hp","Hp12345.","TESTDB")

print('連接上了!')

cursor = db.cursor()

cursor.execute("DROP TABLE IF EXISTS COLOR")

sql = """CREATE TABLE COLOR (

Color CHAR(20) NOT NULL,

Value CHAR(10),

Style CHAR(50) )"""

cursor.execute(sql)

hdrs = {'User-Agent':'Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)'}

url = "http://html-color-codes.info/color-names/"

r = requests.get(url, headers = hdrs)

soup = BeautifulSoup(r.content.decode('gbk', 'ignore'), 'lxml')

trs = soup.find_all('tr') # 獲取全部tr標簽成為一個列表

for tr in trs: # 遍歷列表里所有的tr標簽單項

style = tr.get('style') # 獲取每個tr標簽里的屬性style

tds = tr.find_all('td') # 將每個tr標簽下的td標簽獲取為列表

td = [x for x in tds] # 獲取的列表

name = td[1].text.strip() # 直接從列表里取值

hex = td[2].text.strip()

# print u'顏色: ' + name + u'顏色值: '+ hex + u'背景色樣式: ' + style

# print 'color: ' + name + '\tvalue: '+ hex + '\tstyle: ' + style

insert_color = ("INSERT INTO COLOR(Color,Value,Style)" "VALUES(%s,%s,%s)")

data_color = (name, hex, style)

cursor.execute(insert_color, data_color)

db.commit()

# print '******完成此條插入!'

print '爬取數據并插入mysql數據庫完成...'運行結果:$ mysql -u hp -p

Enter password:

Welcome to the MySQL monitor. Commands end with ; or \g.

Your MySQL connection id is 28

Server version: 5.7.17 MySQL Community Server (GPL)

Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its

affiliates. Other names may be trademarks of their respective

owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use TESTDB

Reading table information for completion of table and column names

You can turn off this feature to get a quicker startup with -A

Database changed

mysql> select * from COLOR;

+----------------------+--------+----------------------------------------+

| Color | Value | Style |

+----------------------+--------+----------------------------------------+

| IndianRed | CD5C5C | background-color:indianred; |

| LightCoral | F08080 | background-color:lightcoral; |

| Salmon | FA8072 | background-color:salmon; |

| DarkSalmon | E9967A | background-color:darksalmon; |

| LightSalmon | FFA07A | background-color:lightsalmon; |

| Crimson | DC143C | background-color:crimson; |

| Red | FF0000 | background-color:red; |

| FireBrick | B22222 | background-color:fireBrick; |

| DarkRed | 8B0000 | background-color:darkred; |

| Pink | FFC0CB | background-color:pink; |

| LightPink | FFB6C1 | background-color:lightpink; |

| HotPink | FF69B4 | background-color:hotpink; |

| DeepPink | FF1493 | background-color:deeppink; |

...

| AntiqueWhite | FAEBD7 | background-color:antiquewhite; |

| Linen | FAF0E6 | background-color:linen; |

| LavenderBlush | FFF0F5 | background-color:lavenderblush; |

| MistyRose | FFE4E1 | background-color:mistyrose; |

| Gainsboro | DCDCDC | background-color:gainsboro; |

| LightGrey | D3D3D3 | background-color:lightgrey; |

| Silver | C0C0C0 | background-color:silver; |

| DarkGray | A9A9A9 | background-color:darkgray; |

| Gray | 808080 | background-color:gray; |

| DimGray | 696969 | background-color:dimgray; |

| LightSlateGray | 778899 | background-color:lightslategray; |

| SlateGray | 708090 | background-color:slategray; |

| DarkSlateGray | 2F4F4F | background-color:darkslategray; |

| Black | 000000 | background-color:black; |

+----------------------+--------+----------------------------------------+

143 rows in set (0.00 sec)以上這篇Python爬取數據并寫入MySQL數據庫的實例就是小編分享給大家的全部內容了，希望能給大家一個參考，也希望大家多多支持中文源碼網。

親,試試微信掃碼分享本頁!?*^_^*

總結

以上是生活随笔為你收集整理的python爬取mysql数据_Python爬取数据并写入MySQL数据库的实例的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。