python打开文件报错无效序列_psycopg2.DataError:编码“UTF8”的字节序列无效:0xa0
我對這個錯誤進行了大量的谷歌搜索,并將其歸結為我正在使用的數據庫采用了不同的編碼。
我正在使用的AIX服務器正在運行psql 8.2.4
server_encoding | LATIN1 | | Client Connection Defaults / Locale and Formatting | Sets the server (database) character set encoding.
我正在使用的windows 2008 R2服務器正在運行
psql(9.3.4)CREATE DATABASE postgres
WITH OWNER = postgres
ENCODING = 'UTF8'
TABLESPACE = pg_default
LC_COLLATE = 'English_Australia.1252'
LC_CTYPE = 'English_Australia.1252'
CONNECTION LIMIT = -1;
COMMENT ON DATABASE postgres
IS 'default administrative connection database';
現在,當我嘗試執行下面的python腳本時,我得到了這個錯誤Traceback (most recent call last):
File "datamain.py", line 39, in
sys.exit(main())
File "datamain.py", line 33, in main
write_file_to_table("cms_jobdef.txt", "cms_jobdef", con_S104838)
File "datamain.py", line 21, in write_file_to_table
cur.copy_from(f, table, ",")
psycopg2.DataError: invalid byte sequence for encoding "UTF8": 0xa0
CONTEXT: COPY cms_jobdef, line 15209
這是我的劇本import psycopg2
import StringIO
import sys
import pdb
def connect_db(db, usr, pw, hst, prt):
conn = psycopg2.connect(database=db, user=usr,
password=pw, host=hst, port=prt)
return conn
def write_table_to_file(file, table, connection):
f = open(file, "w")
cur = connection.cursor()
cur.copy_to(f, table, ",")
f.close()
cur.close()
def write_file_to_table(file, table, connection):
f = open(file,"r")
cur = connection.cursor()
cur.copy_from(f, table, ",")
f.close()
cur.close()
def main():
login = open('login.txt','r')
con_tctmsv64 = connect_db("x", "y",
login.readline().strip(),
"d.domain", "c")
con_S104838 = connect_db("x", "y", "z", "a", "b")
try:
write_table_to_file("cms_jobdef.txt", "cms_jobdef", con_tctmsv64)
write_file_to_table("cms_jobdef.txt", "cms_jobdef", con_S104838)
finally:
con_tctmsv64.close()
con_S104838.close()
if __name__ == "__main__":
sys.exit(main())
已刪除一些敏感數據。
所以我不知道該怎么做。據我所知,copy_expert方法可能有助于導出為UTF8編碼。但由于我從中提取數據的服務器運行的是8.2.4,我認為它不支持COPY編碼格式。
我認為我最好的辦法是嘗試在windows服務器上重新安裝postgre數據庫,其中包含拉丁語1的編碼。當我嘗試這樣做時,我得到了下面的錯誤。
所以我很困,任何幫助都將不勝感激!
更新我在windows上安裝了postgre db作為LATIN1編碼,方法是將默認本地更改為“C”。然而,這給了我以下的錯誤,似乎不是一個可能成功/正確的方法
我還嘗試過使用PSQLCOPY函數對文件進行二進制編碼def write_table_to_file(file, table, connection):
f = open(file, "w")
cur = connection.cursor()
#cur.copy_to(f, table, ",")
cur.copy_expert("COPY cms_jobdef TO STDOUT WITH BINARY", f)
f.close()
cur.close()
def write_file_to_table(file, table, connection):
f = open(file,"r")
cur = connection.cursor()
#cur.copy_from(f, table)
cur.copy_expert("COPY cms_jobdef FROM STDOUT WITH BINARY", f)
f.close()
cur.close()
還是不走運我也犯了同樣的錯誤DataError: invalid byte sequence for encoding "UTF8": 0xa0
CONTEXT: COPY cms_jobdef, line 15209, column descript
關于費城人的回答,我嘗試過這種方法,但仍然沒有成功。import psycopg2
import StringIO
import sys
import pdb
import codecs
def connect_db(db, usr, pw, hst, prt):
conn = psycopg2.connect(database=db, user=usr,
password=pw, host=hst, port=prt)
return conn
def write_table_to_file(file, table, connection):
f = open(file, "w")
#fx = codecs.EncodedFile(f,"LATIN1", "UTF8")
cur = connection.cursor()
cur.execute("SHOW client_encoding;")
print cur.fetchone()
cur.copy_to(f, table)
#cur.copy_expert("COPY cms_jobdef TO STDOUT WITH BINARY", f)
f.close()
cur.close()
def write_file_to_table(file, table, connection):
f = open(file,"r")
cur = connection.cursor()
cur.execute("SET CLIENT_ENCODING TO 'LATIN1';")
cur.execute("SHOW client_encoding;")
print cur.fetchone()
cur.copy_from(f, table)
#cur.copy_expert("COPY cms_jobdef FROM STDOUT WITH BINARY", f)
f.close()
cur.close()
def main():
login = open('login.txt','r')
con_tctmsv64 = connect_db("x", "y",
login.readline().strip(),
"ctmtest1.int.corp.sun", "5436")
con_S104838 = connect_db("x", "y", "z", "t", "5432")
try:
write_table_to_file("cms_jobdef.txt", "cms_jobdef", con_tctmsv64)
write_file_to_table("cms_jobdef.txt", "cms_jobdef", con_S104838)
finally:
con_tctmsv64.close()
con_S104838.close()
if __name__ == "__main__":
sys.exit(main())
輸出In [4]: %run datamain.py
('sql_ascii',)
('LATIN1',)
In [5]:
此操作成功完成,但當我運行select * from cms_jobdef;
新數據庫中沒有任何內容
我甚至嘗試過將文件格式從LATIN1轉換為UTF8。還是不走運
奇怪的是,當我只使用postgreCOPY函數手動執行這個過程時,它就工作了。我不知道為什么。再次感謝您的幫助。
《新程序員》:云原生和全面數字化實踐50位技術專家共同創作,文字、視頻、音頻交互閱讀總結
以上是生活随笔為你收集整理的python打开文件报错无效序列_psycopg2.DataError:编码“UTF8”的字节序列无效:0xa0的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: python文件名有空格_python
- 下一篇: 纯文本文件的字符编码未声明_浅谈几种常见