PostgreSQL 最佳实践 - 水平分库(基于plproxy)
背景
我一直以來都比較推薦plproxy這個PostgreSQL代理軟件, 因為它小巧靈活好用, 效率高.
最近朋友邀請我給他們做個分布式的方案, 所以又把plproxy翻出來了.
本文講一講在單節(jié)點中如何快速的部署plproxy環(huán)境.
環(huán)境
PostgreSQL 9.3.1 plproxy 2.xplrpoxy節(jié)點
hostaddr 172.16.3.150 port 1921 user proxy password proxy dbname proxy schema digoal // 這個schema名和數(shù)據(jù)節(jié)點一致, 可以省去寫plproxy language target的步驟.數(shù)據(jù)節(jié)點
hostaddr 172.16.3.150 port 1921 user digoal // plproxy將使用digoal用戶連接數(shù)據(jù)節(jié)點. password digoal dbname db0 schema digoal dbname db1 schema digoal dbname db2 schema digoal dbname db3 schema digoal部署plproxy
首先在http://git.postgresql.org/gitweb/?p=plproxy.git;a=summary下載plproxy.
tar -zxvf plproxy-d703683.tar.gz mv plproxy-d703683 /opt/soft_bak/postgresql-9.3.1/contrib cd /opt/soft_bak/postgresql-9.3.1/contrib/plproxy-d703683 [root@db-172-16-3-150 plproxy-d703683]# export PATH=/home/pg93/pgsql9.3.1/bin:$PATH [root@db-172-16-3-150 plproxy-d703683]# which pg_config [root@db-172-16-3-150 plproxy-d703683]# gmake clean [root@db-172-16-3-150 plproxy-d703683]# gmake [root@db-172-16-3-150 plproxy-d703683]# gmake install創(chuàng)建proxy庫, proxy角色, 在proxy庫創(chuàng)建plproxy extension.
pg93@db-172-16-3-150-> psql psql (9.3.1) Type "help" for help. postgres=# create role proxy nosuperuser login encrypted password 'proxy'; CREATE ROLE digoal=# create database proxy; CREATE DATABASE digoal=# \c proxy You are now connected to database "proxy" as user "postgres". proxy=# create extension plproxy; CREATE EXTENSION調(diào)整proxy庫權(quán)限
proxy=# grant all on database proxy to proxy; GRANT proxy=# \c proxy proxy You are now connected to database "proxy" as user "digoal".創(chuàng)建digoal schema, 目的是和數(shù)據(jù)節(jié)點的schema匹配, 這樣的話可以省去在代理函數(shù)中寫target強行指定schema.
proxy=> create schema digoal; CREATE SCHEMA創(chuàng)建節(jié)點數(shù)據(jù)庫
proxy=> \c postgres postgres You are now connected to database "postgres" as user "postgres". postgres=# create role digoal nosuperuser login encrypted password 'digoal'; postgres=# create database db0; postgres=# create database db1; postgres=# create database db2; postgres=# create database db3;調(diào)整權(quán)限, 賦予給后面將要給user mapping中配置的option user權(quán)限.
postgres=# grant all on database db0 to digoal; postgres=# grant all on database db1 to digoal; postgres=# grant all on database db2 to digoal; postgres=# grant all on database db3 to digoal;使用超級用戶在proxy數(shù)據(jù)庫中創(chuàng)建server.
proxy=> \c proxy postgres You are now connected to database "proxy" as user "postgres". proxy=# CREATE SERVER cluster_srv1 FOREIGN DATA WRAPPER plproxy options (connection_lifetime '1800', p0 'dbname=db0 hostaddr=172.16.3.150 port=1921 application_name=test', p1 'dbname=db1 hostaddr=172.16.3.150 port=1921', p2 'dbname=db2 hostaddr=172.16.3.150 port=1921', p3 'dbname=db3 hostaddr=172.16.3.150 port=1921');創(chuàng)建server時可以使用libpq中的選項. 例如本例使用了application_name.
將server權(quán)限賦予給proxy用戶.
proxy=# grant usage on FOREIGN server cluster_srv1 to proxy; GRANT配置proxy用戶的連接cluster_srv1的選項.
proxy=# create user mapping for proxy server cluster_srv1 options (user 'digoal'); CREATE USER MAPPING用戶proxy連接到cluster_srv1時使用digoal用戶連接, 這里不需要配置password, 因為我們將使用trust認證.
修改數(shù)據(jù)節(jié)點的pg_hba.conf
從proxy節(jié)點使用digoal用戶連接數(shù)據(jù)庫db0, db1, db2, db3使用trust認證.
vi $PGDATA/pg_hba.conf host db0 digoal 172.16.3.150/32 trust host db1 digoal 172.16.3.150/32 trust host db2 digoal 172.16.3.150/32 trust host db3 digoal 172.16.3.150/32 trust pg_ctl reload在plproxy節(jié)點創(chuàng)建代理函數(shù)
使用超級用戶創(chuàng)建plproxy函數(shù), 然后把函數(shù)權(quán)限賦予給proxy權(quán)限.
proxy=# CREATE OR REPLACE FUNCTION digoal.dy(sql text) RETURNS SETOF record LANGUAGE plproxy STRICT AS $function$ cluster 'cluster_srv1'; run on all; $function$; proxy=# grant execute on function digoal.dy(text) to proxy; GRANT在數(shù)據(jù)節(jié)點創(chuàng)建實體函數(shù)
proxy=# \c db0 digoal db0=# CREATE OR REPLACE FUNCTION digoal.dy(sql text) RETURNS SETOF record LANGUAGE plpgsql STRICT AS $function$ declare rec record; begin for rec in execute sql loop return next rec; end loop; return; end; $function$; db0=# \c db1 digoal ... db1=# \c db2 digoal ... db2=# \c db3 digoal ...在proxy節(jié)點中就可以訪問數(shù)據(jù)節(jié)點了。
例如查詢這個動態(tài)SQL.
proxy=> select * from digoal.dy('select count(*) from pg_class') as t(i int8); i ----- 293 293 293 293 (4 rows) proxy=> select sum(i) from digoal.dy('select count(*) from pg_class') as t(i int8); sum ------ 1172 (1 row)plproxy節(jié)點測試
一. 修改foreign server測試, 觀察連接將重置.
前面那個會話不要斷開, 在另一個會話中觀察proxy發(fā)起的連接到數(shù)據(jù)節(jié)點的連接.
postgres=# select * from pg_stat_activity where usename='digoal'; datid | datname | pid | usesysid | usename | application_name | client_addr | client_hostname | client_port | backend_sta rt | xact_start | query_start | state_change | waiting | state | q uery -------+---------+------+----------+---------+------------------+--------------+-----------------+-------------+-------------------- -----------+------------+-------------------------------+-------------------------------+---------+-------+------------------------- ---------------------------- 91246 | db0 | 8171 | 91250 | digoal | test | 172.16.3.150 | | 47937 | 2013-11-22 17:23:26 .138425+08 | | 2013-11-22 17:27:05.539286+08 | 2013-11-22 17:27:05.539745+08 | f | idle | select i::int8 from digo al.dy($1::text) as (i int8) 91247 | db1 | 8172 | 91250 | digoal | | 172.16.3.150 | | 47938 | 2013-11-22 17:23:26 .138688+08 | | 2013-11-22 17:27:05.53938+08 | 2013-11-22 17:27:05.539874+08 | f | idle | select i::int8 from digo al.dy($1::text) as (i int8) 91248 | db2 | 8173 | 91250 | digoal | | 172.16.3.150 | | 47939 | 2013-11-22 17:23:26 .138957+08 | | 2013-11-22 17:27:05.53938+08 | 2013-11-22 17:27:05.539841+08 | f | idle | select i::int8 from digo al.dy($1::text) as (i int8) 91249 | db3 | 8174 | 91250 | digoal | | 172.16.3.150 | | 47940 | 2013-11-22 17:23:26 .139178+08 | | 2013-11-22 17:27:05.539366+08 | 2013-11-22 17:27:05.539793+08 | f | idle | select i::int8 from digo al.dy($1::text) as (i int8) (4 rows)再次在proxy的同一會話中查詢時, 這些會話會復用, 不會斷開. 前面已經(jīng)講了plproxy是使用長連接的.
如果修改了server, 那么這些連接會斷開, 重新連接. 所以不需要擔心修改server帶來的連接cache問題.
postgres=# \c proxy postgres You are now connected to database "proxy" as user "postgres". proxy=# alter server cluster_srv1 options (set p1 'dbname=db1 hostaddr=172.16.3.150 port=1921 application_name=abc'); ALTER SERVER再次在proxy的同一會話中查詢后, 我們發(fā)現(xiàn)4個連接都變了, 說明alter server后, 如果再次發(fā)起plproxy函數(shù)的查詢請求, 那么proxy會重置連接.
proxy=> select sum(i) from digoal.dy('select count(*) from pg_class') as t(i int8); sum ------ 1172 (1 row)在另一會話的查詢結(jié)果 :
proxy=# select * from pg_stat_activity where usename='digoal'; datid | datname | pid | usesysid | usename | application_name | client_addr | client_hostname | client_port | backend_sta rt | xact_start | query_start | state_change | waiting | state | q uery -------+---------+------+----------+---------+------------------+--------------+-----------------+-------------+-------------------- -----------+------------+-------------------------------+-------------------------------+---------+-------+------------------------- ---------------------------- 91246 | db0 | 8245 | 91250 | digoal | test | 172.16.3.150 | | 47941 | 2013-11-22 17:30:36 .933077+08 | | 2013-11-22 17:30:36.936784+08 | 2013-11-22 17:30:36.938837+08 | f | idle | select i::int8 from digo al.dy($1::text) as (i int8) 91248 | db2 | 8247 | 91250 | digoal | | 172.16.3.150 | | 47943 | 2013-11-22 17:30:36 .933502+08 | | 2013-11-22 17:30:36.936783+08 | 2013-11-22 17:30:36.938981+08 | f | idle | select i::int8 from digo al.dy($1::text) as (i int8) 91249 | db3 | 8248 | 91250 | digoal | | 172.16.3.150 | | 47944 | 2013-11-22 17:30:36 .933731+08 | | 2013-11-22 17:30:36.937147+08 | 2013-11-22 17:30:36.939015+08 | f | idle | select i::int8 from digo al.dy($1::text) as (i int8) 91247 | db1 | 8246 | 91250 | digoal | abc | 172.16.3.150 | | 47942 | 2013-11-22 17:30:36 .933288+08 | | 2013-11-22 17:30:36.93757+08 | 2013-11-22 17:30:36.939299+08 | f | idle | select i::int8 from digo al.dy($1::text) as (i int8) (4 rows)二. run on 的幾種形式, 表示數(shù)據(jù)路由方法.
在數(shù)據(jù)節(jié)點創(chuàng)建測試表.
proxy=# \c db0 digoal db0=> create table t(id int); CREATE TABLE db0=> \c db1 You are now connected to database "db1" as user "digoal". db1=> create table t(id int); CREATE TABLE db1=> \c db2 You are now connected to database "db2" as user "digoal". db2=> create table t(id int); CREATE TABLE db2=> \c db3 You are now connected to database "db3" as user "digoal". db3=> create table t(id int); CREATE TABLE在數(shù)據(jù)節(jié)點創(chuàng)建插入數(shù)據(jù)的實體函數(shù), 每個節(jié)點返回不一樣的數(shù)字.
\c db0 digoal db0=> create or replace function digoal.f_test4() returns int as $$declare begin insert into t(id) values (1); return 0; end; $$language plpgsql strict; db1=> create or replace function digoal.f_test4() returns int as $$declare begin insert into t(id) values (1); return 1; end; $$language plpgsql strict; db2=> create or replace function digoal.f_test4() returns int as $$declare begin insert into t(id) values (1); return 2; end; $$language plpgsql strict; db3=> create or replace function digoal.f_test4() returns int as $$declare begin insert into t(id) values (1); return 3; end; $$language plpgsql strict;在proxy節(jié)點創(chuàng)建代理函數(shù), 并且將執(zhí)行權(quán)限賦予給proxy用戶.
proxy=> \c proxy postgres create or replace function digoal.f_test4() returns int as $$cluster 'cluster_srv1'; run on 0; -- 在指定的數(shù)據(jù)節(jié)點上運行, 本例可以設(shè)置為0到3, 順序和創(chuàng)建的server中的配置順序一致. p0, p1, p2, p3 $$language plproxy strict; proxy=# grant execute on function digoal.f_test4() to proxy; GRANT proxy=# \c proxy proxy You are now connected to database "proxy" as user "proxy". proxy=> select * from digoal.f_test4(); f_test4 --------- 0 (1 row)如果run on 的數(shù)字改成0-3以外的數(shù)字, 運行時將報錯, 符合預期.
proxy=# create or replace function digoal.f_test4() returns int as $$cluster 'cluster_srv1'; run on 4; $$language plproxy strict; CREATE FUNCTION proxy=# \c proxy proxy You are now connected to database "proxy" as user "proxy". proxy=> select * from digoal.f_test4(); ERROR: PL/Proxy function digoal.f_test4(0): part number out of rangerun on any表示隨機的選擇一個數(shù)據(jù)節(jié)點運行.
proxy=> \c proxy postgres You are now connected to database "proxy" as user "postgres". proxy=# create or replace function digoal.f_test4() returns int as $$cluster 'cluster_srv1'; run on any; $$language plproxy strict; CREATE FUNCTION proxy=# \c proxy proxy You are now connected to database "proxy" as user "proxy". proxy=> select * from digoal.f_test4(); f_test4 --------- 0 (1 row) proxy=> select * from digoal.f_test4(); f_test4 --------- 3 (1 row) proxy=> select * from digoal.f_test4(); f_test4 --------- 2 (1 row) proxy=> select * from digoal.f_test4(); f_test4 --------- 3 (1 row)run on function() 則使用函數(shù)結(jié)果的hash值計算得到運行節(jié)點.
proxy=> create or replace function digoal.f(int) returns int as $$select $1; $$language sql strict; CREATE FUNCTION proxy=> \c proxy postgres You are now connected to database "proxy" as user "postgres". proxy=# create or replace function digoal.f_test4() returns int as $$cluster 'cluster_srv1'; run on digoal.f(10); $$language plproxy strict; CREATE FUNCTION proxy=> select digoal.f_test4(); f_test4 --------- 2 (1 row) proxy=> \c proxy postgres You are now connected to database "proxy" as user "postgres". proxy=# create or replace function digoal.f_test4() returns int as $$cluster 'cluster_srv1'; run on digoal.f(11); $$language plproxy strict; CREATE FUNCTION proxy=# \c proxy proxy You are now connected to database "proxy" as user "proxy". proxy=> select digoal.f_test4(); f_test4 --------- 3 (1 row) proxy=> \c proxy postgres You are now connected to database "proxy" as user "postgres". proxy=# create or replace function digoal.f_test4() returns int as $$cluster 'cluster_srv1'; run on digoal.f(-11); $$language plproxy strict; CREATE FUNCTION proxy=# \c proxy proxy You are now connected to database "proxy" as user "proxy". proxy=> select digoal.f_test4(); f_test4 --------- 1 (1 row)run on all表示所有數(shù)據(jù)節(jié)點運行. 代理函數(shù)必須使用returns setof返回.
proxy=> \c proxy postgres You are now connected to database "proxy" as user "postgres". proxy=# create or replace function digoal.f_test4() returns int as $$cluster 'cluster_srv1'; run on all; $$language plproxy strict; ERROR: PL/Proxy function digoal.f_test4(0): RUN ON ALL requires set-returning function proxy=# drop function digoal.f_test4(); DROP FUNCTION proxy=# create or replace function digoal.f_test4() returns setof int as $$cluster 'cluster_srv1'; run on all; $$language plproxy strict; CREATE FUNCTION proxy=# grant execute on function digoal.f_test4() to proxy; GRANT proxy=# \c proxy proxy You are now connected to database "proxy" as user "proxy". proxy=> select digoal.f_test4(); f_test4 --------- 0 1 2 3 (4 rows)注意事項
1. 設(shè)計時需要注意
plproxy函數(shù)所在的schema盡量和數(shù)據(jù)節(jié)點上實際函數(shù)的schema一致.
否則需要在plproxy函數(shù)中使用target指定 schema.functionname;
2. 數(shù)據(jù)節(jié)點的個數(shù)請保持2^n,
這么做有利于后期的節(jié)點擴展, 例如2個節(jié)點擴展到4個節(jié)點時, 數(shù)據(jù)不需要發(fā)生跨節(jié)點的重分布.
例如
mod(x,2)=0 那么mod(x,4)=0或2 mod(x,2)=1 那么mod(x,4)=1或3比較適合位運算的分布算法.
當然我們也可以使用一致性哈希的設(shè)計思路，參考
《一致性哈希在分布式數(shù)據(jù)庫中的應(yīng)用探索》
https://yq.aliyun.com/articles/57954
3. 如果業(yè)務(wù)為短連接的形式, 那么需要1層連接池, 在應(yīng)用程序和plproxy數(shù)據(jù)庫之間. 而不是plproxy和數(shù)據(jù)節(jié)點之間.
在應(yīng)用程序和plproxy之間加連接池后, 其實對于plproxy來說就是長連接了, 所以在plproxy和數(shù)據(jù)節(jié)點之間也就不需要連接池了.
4. 長連接不需要連接池, 因為plproxy和數(shù)據(jù)節(jié)點之間的連接是長連接.
5. plproxy語法非常簡潔，而且函數(shù)調(diào)用徹底避免了事務(wù)的問題
connect, cluster, run, select, split, target.
6. 關(guān)于連接密碼
出于安全考慮, 建議在任何配置中不要出現(xiàn)明文密碼, 所以最好是plproxy服務(wù)器到數(shù)據(jù)節(jié)點是trust驗證, 保護好plproxy即可.
假設(shè)plproxy在172.16.3.2上. 數(shù)據(jù)節(jié)點有4個, 庫名和用戶名都為digoal. 那么在4個節(jié)點上配置pg_hba.conf如下.
node0 host digoal digoal 172.16.3.2/32 trust node1 host digoal digoal 172.16.3.2/32 trust node2 host digoal digoal 172.16.3.2/32 trust node3 host digoal digoal 172.16.3.2/32 trust7. run 詳解:
run on <NR>, <NR>是數(shù)字常量, 范圍是0 到 nodes-1; 例如有4個節(jié)點 run on 0; (run on 4則報錯). run on ANY, run on function(...), 這里用到的函數(shù)返回結(jié)果必須是int2, int4 或 int8. run on ALL, 這種的plproxy函數(shù)必須是returns setof..., 實體函數(shù)沒有setof的要求.8. 一個plproxy中只能出現(xiàn)一條connect語句, 符合預期, 否則報錯.
digoal=# create or replace function f_test3() returns setof int8 as $$connect 'hostaddr=172.16.3.150 dbname=db0 user=digoal port=1921'; connect 'hostaddr=172.16.3.150 dbname=db1 user=digoal port=1921'; select count(*) from pg_class; $$language plproxy strict; ERROR: PL/Proxy function postgres.f_test3(0): Compile error at line 2: Only one CONNECT statement allowed9. 不要把plproxy語言的權(quán)限賦予給普通用戶, 因為開放了trust認證, 如果再開放plproxy語言的權(quán)限是非常危險的.
正確的做法是使用超級用戶創(chuàng)建plproxy函數(shù), 然后把函數(shù)的執(zhí)行權(quán)限賦予給普通用戶.
千萬不要這樣省事 :
update pg_language set lanpltrusted='t' where lanname='plproxy';10. 如果有全局唯一的序列需求, 可以將序列的步調(diào)調(diào)整一下, 每個數(shù)據(jù)節(jié)點使用不同的初始值.
例如
考慮到擴容, 可以將步調(diào)調(diào)比較大, 例如1024. 那么可以容納1024個節(jié)點.
參考
1. http://kaiv.wordpress.com/2007/07/27/postgresql-cluster-partitioning-with-plproxy-part-i/
2. http://kaiv.wordpress.com/2007/09/02/postgresql-cluster-partitioning-with-plproxy-part-ii/
3. http://blog.163.com/digoal@126/blog/static/163877040201041111304328/
4. http://blog.163.com/digoal@126/blog/static/1638770402010411113114315/
5. http://blog.163.com/digoal@126/blog/static/163877040201192535630895/
6. http://www.postgresql.org/docs/9.3/static/libpq-connect.html#LIBPQ-CONNSTRING
7. http://git.postgresql.org/gitweb/?p=plproxy.git;a=summary
總結(jié)
以上是生活随笔為你收集整理的PostgreSQL 最佳实践 - 水平分库(基于plproxy)的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： 总结之前做项目中要注意的一些书写规范
- 下一篇： MySQL存储过程相互调用