學(xué)習(xí)pyspider 參照《 把 pyspider的結(jié)果存入自定義的mysql數(shù)據(jù)庫中 》文章

設(shè)置了本地的mysql數(shù)據(jù)庫及連接，但運(yùn)行時(shí) 提示 ：process csair:_on_get_info data:,_on_get_info -> [200] len:12 -> result:None fol:0 msg:0 err:ImportError(‘No module named mysqldb’,)

系統(tǒng)環(huán)境：centos7.1

mysql-python(1.2.5) pip 默認(rèn)安裝

python采用默認(rèn)的2.7.5

希望得到幫助 感謝

文件已存到 pyspider/database/mysql/mysqldb.py 腳本文件如下：

#!/usr/bin/env python

# -*- encoding: utf-8 -*-

# Created on 2015-12-21 09:28:49

# Project: csair

from pyspider.libs.base_handler import *

from pyspider.database.mysql.mysqldb import SQL

class Handler(BaseHandler):

crawl_config = {

}

@every(minutes=24 * 60)

def on_start(self):

self.crawl('http://www.csair.cn/cn/pages/NewsList.aspx?listname=%D6%D8%D2%AA%B9%AB%B8%E6', callback=self.index_page)

@config(age=10 * 24 * 60 * 60)

def index_page(self, response):

for each in response.doc('a[href^="http"]').items():

self.crawl(each.attr.href, callback=self.detail_page)

def list_page(self, response):

for each in response.doc('ul_list7 a').items():

self.crawl(each.attr.href, callback=self.detail_page)

for each in response.doc('.currentnumeric').items():

self.crawl(each.attr.href, callback=self.list_page)

@config(priority=2)

def detail_page(self, response):

return {

"url": response.url,

"title": response.doc('ctl00_ContentPlaceHolder1_DetailBar1_NewsTitle ').text(),

"fabushijain" :response.doc('span#ctl00_ContentPlaceHolder1_DetailBar1_ReleaseLabel').text(),

"content" : response.doc('#ctl00_ContentPlaceHolder1_DetailBar1_ContentInfo').text(),

}

def on_result(self, result):

#print result

if not result or not result['title']:

return

sql = SQL()

sql.replace('info',**result)

＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋＋

mysqldb.py 文件內(nèi)容如下：

class SQL:

username = 'pyspider' ? #數(shù)據(jù)庫用戶名

password = 'pyspider' ? #數(shù)據(jù)庫密碼

database = 'result' ? ? #數(shù)據(jù)庫

host = 'localhost' ? ? ?#數(shù)據(jù)庫主機(jī)地址

connection = ''

connect = True

placeholder = '%s'

def __init__(self):

if self.connect:

SQL.connect(self)

def escape(self,string):

return '`%s`' % string

def connect(self):

config = {

'user':SQL.username,

'password':SQL.password,

'host':SQL.host

}

if SQL.database != None:

config['database'] = SQL.database

try:

cnx = mysql.connector.connect(**config)

SQL.connection = cnx

return True

except mysql.connector.Error as err:

if (err.errno == errorcode.ER_ACCESS_DENIED_ERROR):

print "The credentials you provided are not correct."

elif (err.errno == errorcode.ER_BAD_DB_ERROR):

print "The database you provided does not exist."

else:

print "Something went wrong: " , err

return False

def replace(self,tablename=None,**values):

if SQL.connection == '':

print "Please connect first"

return False

tablename = self.escape(tablename )

if values:

_keys = ", ".join(self.escape(k) for k in values)

_values = ", ".join([self.placeholder, ] * len(values))

sql_query = "REPLACE INTO %s (%s) VALUES (%s)" % (tablename, _keys, _values)

else:

、 ? ? sql_query = "REPLACE INTO %s DEFAULT VALUES" % tablename

cur = SQL.connection.cursor()

try:

if values:

cur.execute(sql_query, list(itervalues(values)))

else:

cur.execute(sql_query)

SQL.connection.commit()

return True

except mysql.connector.Error as err:

print ("An error occured: {}".format(err))

return False

有兩個(gè)思路供參考：

檢查一下包下邊的__init__.py是否存在

檢查一下模塊名和導(dǎo)入的模塊名是否一致，特別是這個(gè)模塊被多個(gè)py文件導(dǎo)入

感謝支持 貌似不是這個(gè)原因

有可能是原來的mysql-python的連接版本過低導(dǎo)致，pip安裝后是mysql-connector-python(1.0.11) Mysql-python(1.2.5)

參考版本地址： http://ftp.ntu.edu.tw/MySQL/Downloads/Connector-Python/

采用最新版的mysql-connector-python(2.1.2) 還是不行 繼續(xù)測(cè)試中

版本是centos7的版本 已到 http://ftp.ntu.edu.tw/MySQL/Downloads/Connector-Python/下載的 2.1.2版本

是在什么平臺(tái)上？如果是windows 需要到mysql的主頁上找到對(duì)應(yīng)的驅(qū)動(dòng)包

ubuntu 上需要安裝 apt-get install python-mysqldb 來安裝這個(gè)包

總結(jié)

以上是生活随笔為你收集整理的pyspider 连接mysql_pyspider 导入模块报错 提示 No module named mysqldb的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。