閱讀使人充實，討論使人敏捷，寫作使人精確。

?遷移工具簡介

今天介紹一個MySQL遷移PostgreSQL的自動化工具py-mysql2pgsql，該工具通過python使用不落地方式，或?qū)懗龅轿募绞?可選)，直接將MySQL中的表結(jié)構(gòu)轉(zhuǎn)化到pg，并自動導(dǎo)入數(shù)據(jù)、自動在pg端創(chuàng)建相關(guān)索引。

MySQL遷移PostgreSQL數(shù)據(jù)類型轉(zhuǎn)化

目前該工具可將以下38種數(shù)據(jù)類型自動轉(zhuǎn)化為PostgreSQL對應(yīng)的兼容類型，如需其他類型轉(zhuǎn)化，在配置文件添加即可。

MySQLPostgreSQL

char	character
varchar	character varying
tinytext	text
mediumtext	text
text	text
longtext	text
tinyblob	bytea
mediumblob	bytea
blob	bytea
longblob	bytea
binary	bytea
varbinary	bytea
bit	bit varying
tinyint	smallint
tinyint unsigned	smallint
smallint	smallint
smallint unsigned	integer
mediumint	integer
mediumint unsigned	integer
int	integer
int unsigned	bigint
bigint	bigint
bigint unsigned	numeric
float	real
float unsigned	real
double	double precision
double unsigned	double precision
decimal	numeric
decimal unsigned	numeric
numeric	numeric
numeric unsigned	numeric
date	date
datetime	timestamp without time zone
time	time without time zone
timestamp	timestamp without time zone
year	smallint
enum	character varying (with?check?constraint)
set	ARRAY[]::text[]

?自動化遷移過程

工具運行后，整個過程可以分為三個階段：

l?在PostgreSQL端自動創(chuàng)建表結(jié)構(gòu)；l?在PostgreSQL端自動loading加載數(shù)據(jù);l?在PostgreSQL端自動創(chuàng)建索引、約束。?工具安裝及使用例子l?下載源碼安裝包下載地址https://pypi.org/project/py-mysql2pgsql/#descriptionl?解壓安裝1.需解決的依賴包：python需按照這幾個依賴包：termcolor、mysql-python、psycopg2、argparse2.安裝mysql2pgsql[root@dbhost ]# pwd/data/aken/tools/py-mysql2pgsql-0.1.5[root@dbhost ]# python setup.py install3.驗證安裝，查看help幫助[root@dbhost /data/aken/tools/py-mysql2pgsql-0.1.5]# py-mysql2pgsql -husage: py-mysql2pgsql [-h] [-v] [-f FILE] [-V]Tool for migrating/converting data from mysql to postgresql.optional arguments: -h, --help show this help message and exit -v, --verbose Show progress of data migration. -f FILE, --file FILE Location of configuration file (default:mysql2pgsql.yml). If none exists at that path, one will be created for you. -V, --version Print version and exit.https://github.com/philipsoutham/py-mysql2pgsql[root@dbhost /data/aken/tools/py-mysql2pgsql-0.1.5]#?遷移存量數(shù)據(jù)如下將MySQL中的test.tab_testtext表存量遷移到PostgreSQL中akendb的public下面。l?編輯遷移配置文件vi mysql2pgsql.yml 如下：#source # if a socket is specified we will use that# if tcp is chosen you can use compressionmysql: hostname: 100.66.66.66 port: 15140 socket: username: dbmgr password: 520DBA database: test compress: falsedestination: # if file is given, output goes to file, else postgres. file: postgres: hostname: 100.88.88.88 port: 11005 username: aken password: aken123 database: akendb# 指定遷移的表，默認遷移database下所有表。if only_tables is given, only the listed tables will be converted. leave empty to convert all tables.only_tables:#- table1#- table2 tab_testtext# 指定排除的表。if exclude_tables is given, exclude the listed tables from the conversion.#exclude_tables:#- table3#- table4# 指定是否只導(dǎo)出表結(jié)構(gòu)，ture表示只遷移dll表結(jié)構(gòu)。if supress_data is true, only the schema definition will be exported/migrated, and not the datasupress_data: false# 指定是否只遷移數(shù)據(jù)，true表示只遷移數(shù)據(jù)。if supress_ddl is true, only the data will be exported/imported, and not the schemasupress_ddl: false# 表存在是否清空導(dǎo)入，true表示清空再導(dǎo)入。if force_truncate is true, forces a table truncate before table loadingforce_truncate: false# if timezone is true, forces to append/convert to UTC tzinfo mysql datatimezone: false# if index_prefix is given, indexes will be created whith a name prefixed with index_prefixindex_prefix:l?執(zhí)行數(shù)據(jù)遷移執(zhí)行導(dǎo)入數(shù)據(jù)后，自動執(zhí)行過程分3個階段：1.自動在PostgreSQL創(chuàng)建表結(jié)構(gòu)；2.自動加載數(shù)據(jù)(約1w rows/sec)；3.自動在PostgreSQL創(chuàng)建索引。[root@dbhost]#?py-mysql2pgsql?-v?-f?mysql2pgsql.yml?>>>>>>>>>> STARTING <<<<<<<<<<START CREATING TABLES START - CREATING TABLE tab_testtext FINISH - CREATING TABLE tab_testtextDONE CREATING TABLESSTART WRITING TABLE DATA??START??-?WRITING?DATA?TO?tab_testtext FINISH - WRITING DATA TO tab_testtextDONE WRITING TABLE DATASTART CREATING INDEXES AND CONSTRAINTS START - ADDING INDEXES TO tab_testtext FINISH - ADDING INDEXES TO tab_testtext START - ADDING CONSTRAINTS ON tab_testtext FINISH - ADDING CONSTRAINTS ON tab_testtextDONE CREATING INDEXES AND CONSTRAINTS>>>>>>>>>> FINISHED <<<<<<<<<<數(shù)據(jù)對比驗證l表結(jié)構(gòu)比對

1.MySQL表結(jié)構(gòu)

MySQL [test]> show create table tab_testtext;+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Table | Create Table |+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| tab_testtext | CREATE TABLE `tab_testtext` ( `id` int(11) NOT NULL DEFAULT '0', `name` longtext, `owners` longtext, `parent_id` int(11) DEFAULT NULL, `busid` int(11) DEFAULT NULL, `uid` varchar(255) DEFAULT NULL, `level` int(11) DEFAULT NULL, `update_date` datetime NOT NULL, `create_date` datetime NOT NULL, `limit_load` int(11) DEFAULT '65', `children_count` int(11) DEFAULT '0', `limit_low_load` int(11) DEFAULT '30', `history_load` varchar(255) DEFAULT NULL, `status` int(11) DEFAULT '0', `group_id` int(11) DEFAULT '21576', `_alarm_types` varchar(64) DEFAULT NULL, `star_level` int(11) DEFAULT '0', `remark` text, `enable` tinyint(11) NOT NULL DEFAULT '1', UNIQUE KEY `busid` (`busid`), UNIQUE KEY `uid` (`uid`), KEY `core_business_6be37982` (`parent_id`), KEY `group_id` (`group_id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8 |+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+1?row?in?set?(0.01?sec)MySQL [test]>2.PostgreSQL表結(jié)構(gòu)akendb=# \d+ tab_testtext Table "public.tab_testtext" Column | Type | Collation | Nullable | Default | Storage | Stats target | Description ----------------+-----------------------------+-----------+----------+---------+----------+--------------+------------- id | integer | | not null | 0 | plain | | name | text | | | | extended | | owners | text | | | | extended | | parent_id | integer | | | | plain | | busid | integer | | | | plain | | uid | character varying(255) | | | | extended | | level | integer | | | | plain | | update_date | timestamp without time zone | | not null | | plain | | create_date | timestamp without time zone | | not null | | plain | | limit_load | integer | | | 65 | plain | | children_count | integer | | | 0 | plain | | limit_low_load | integer | | | 30 | plain | | history_load | character varying(255) | | | | extended | | status | integer | | | 0 | plain | | group_id | integer | | | 21576 | plain | | _alarm_types | character varying(64) | | | | extended | | star_level | integer | | | 0 | plain | | remark | text | | | | extended | | enable | smallint | | not null | 1 | plain | | Indexes: "tab_testtext_busid" UNIQUE, btree (busid) "tab_testtext_uid" UNIQUE, btree (uid) "tab_testtext_group_id" btree (group_id) "tab_testtext_parent_id" btree (parent_id)Access method: heapakendb=#l數(shù)據(jù)抽樣比對

1.MySQL數(shù)據(jù)

MySQL [test]> select count(*) from tab_testtext;+----------+| count(*) |+----------+| 4919 |+----------+1?row?in?set?(0.00?sec)MySQL [test]> select id,name from tab_testtext limit 3;+------+--------------------------------------+| id | name |+------+--------------------------------------+|?7306?|?[N][測試]????????????????????????||?7307?|?[N][數(shù)據(jù)遷移]??????????????????????|| 7308 | [客戶端接入][登陸] |+------+--------------------------------------+3 rows in set (0.00 sec)MySQL?[test]>2.PostgreSQL數(shù)據(jù)akendb=# select count(*) from tab_testtext; count ------- 4919(1?row)akendb=# select id,name from tab_testtext limit 3; id | name ------+--------------------------------------?7306?|?[N][測試]?7307?|?[N][數(shù)據(jù)遷移]?7308?|?[客戶端接入][登陸](3?rows)akendb=#>>>

參考資料

1.https://github.com/philipsoutham/py-mysql2pgsql

往期推薦

1.PostgreSQL等待事件-鎖等待分析

2.基于PG億級毫秒響應(yīng)實時推薦系統(tǒng)-解決方案探索

? ? ? ? ? ? ? ? ? ? ? ? ?------讓學(xué)習(xí)成為一種習(xí)慣-Aken

總結(jié)

以上是生活随笔為你收集整理的yii 执行指定迁移文件_MySQL迁移PG工具pymysql2pgsql的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。