20220308

select format_datetime(create_time ,'yyyy-MM-dd')fromiceberg.ice_ods.ods_o_hz_b2b_tb_order_itemtimestamp轉(zhuǎn)日期

https://blog.csdn.net/u010711495/article/details/112195655
timestamp日期轉(zhuǎn)換重點

20220308

 date(order_day) > date_add('day',-30,current_date) 字符轉(zhuǎn)日期,日期加減

20220215

SQL 錯誤: Error executing query
服務(wù)器連不上
堡壘機(jī)出問題了

20220207

ModuleNotFoundError: No module named ‘trino.sqlalchemy’
trino和sqlalchemy的版本兼容問題
trino需要0.306.0
sqlalchemy 1.4.23

sqlalchemy.exc.NoSuchModuleError: Can‘t load plugin: sqlalchemy.dialects:presto
sqlalchemy.exc.NoSuchModuleError: Can‘t load plugin: sqlalchemy.dialects:trino
需要安裝PyHive 0.6.4

20220126

用python直連Trino不能delete表里面的數(shù)據(jù)只能刪表
presto也可以直接使用mysql語法？
presto也是數(shù)據(jù)倉庫

20220119

TrinoUserError(type=USER_ERROR, name=SYNTAX_ERROR, message="line 20:49: mismatched input '#'. Expecting: '(', ')',sql代碼里面還有井號,去掉井號換成 --

20220118

HTTPConnectionPool(host='192.168.1.55', port=8881): Max retries exceeded with url: /v1/statement (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001E783328278>: Failed to establish a new connection: [WinError 10061] 由于目標(biāo)計算機(jī)積極拒絕，無法連接。'))
服務(wù)掛了,重啟

20220114

TrinoUserError(type=USER_ERROR, name=SYNTAX_ERROR, message="line 9:8: mismatched input '100'. Expecting: '%', '*', '+', '-', '.', '/', 'AND', 'AT', 'EXCEPT', 'FETCH', 'GROUP', 'HAVING', 'INTERSECT', 'LIMIT', 'OFFSET', 'OR', 'ORDER', 'UNION', 'WINDOW', '[', '||', <EOF>, <predicate>", query_id=20220114_081216_00157_decgk)應(yīng)該是sql語法錯了   limit之前不能加and 

20220107

date(‘2021-10-01’)
cast(‘2021-10-01’ as date)
字符轉(zhuǎn)日期

https://www.cnblogs.com/lixiaozhi/p/11752483.html
窗口函數(shù) lead 和 lag的理解
partition by 分區(qū)之后 lead 取當(dāng)前行的前幾行,lag取當(dāng)前行的后幾行

20220106

date_diff('day',LEAD(uo.create_time,1,uo.create_time) over (partition by ui.user_id order by uo.create_time desc) ,uo.create_time)

20211223

第三方模塊trino讀寫數(shù)據(jù)（不推薦）
參考：https://github.com/trinodb/trino-python-client#########
# 20220301更新 批量插入,全部插入要報錯
def insert_trino(df):http.client._MAXLINE = 655360trino_engine = create_engine('trino://root@192.168.1.55:8881/iceberg/ice_ods')times = int(np.ceil(df.shape[0] / 10000))for i in tqdm(range(times)):df.iloc[i * 10000: (i + 1) * 10000, :].to_sql(con=trino_engine, schema="ice_ods",name="ods_o_hz_onekey_jkzj_goods_washed_da", method="multi", if_exists='append',index=False)logger.debug("存入數(shù)據(jù)庫成功")
#######import trino
from trino import transaction
with trino.dbapi.connect(host='192.168.1.55',port=8881,user='root',catalog='iceberg',schema='ice_dwt',# isolation_level=transaction.IsolationLevel.REPEATABLE_READ,
) as conn:cur = conn.cursor()cur.execute('SELECT * FROM iceberg.ice_dwt.dwt_dm_bi_b2b_company_name_wide')rows = cur.fetchall()
寫(不推薦，批量寫入速度太慢)
import trino
from trino import transaction
with trino.dbapi.connect(host='192.168.1.55',port=8881,user='root',catalog='iceberg',schema='ice_dwt',# isolation_level=transaction.IsolationLevel.REPEATABLE_READ,
) as conn:cur = conn.cursor()# 寫入cur.execute("insert into iceberg.ice_dwt.dwt_dm_bi_b2b_company_name_wide (original_name,standard_name,code,create_date)   values ('邛崍本地','無效',-2,date('2021-12-22'))")cur.fetchall()
pandas模塊的 to_sql,read_sql方法讀取寫入數(shù)據(jù)（推薦）參考：https://github.com/dungdm93/sqlalchemy-trinoSQLAlchemy 連接trino 的url格式：trino://<username>:<password>@<host>:<port>/catalog/[schema]
import pandas as pd
from pandas import DataFramefrom sqlalchemy.engine import Engine, Connection
from sqlalchemy.engine import create_enginetrino_engine = create_engine('trino://root@192.168.1.55:8881/iceberg/ice_dwt')def trino_pandas_write(engine: Engine):df: DataFrame = pd.read_csv('***.csv')df['create_date'] = pd.to_datetime(df['create_date'])df.to_sql(con=trino_engine, schema="ice_dwt",name="dwt_dm_bi_b2b_company_name_wide", method="multi", if_exists='append',index=False)def trino_pandas_read(engine: Engine):connection: Connection = engine.connect()df = pd.read_sql("SELECT original_name, standard_name, code, create_date, update_date, note FROM iceberg.ice_dwt.dwt_dm_bi_b2b_company_name_wide",connection)print(df.shape)trino_pandas_write(trino_engine)
trino_pandas_read(trino_engine)寫入數(shù)據(jù)報錯requests.exceptions.ConnectionError詳情：requests.exceptions.ConnectionError: ('Connection aborted.', LineTooLong('got more than 65536 bytes when reading header line'))原因分析：http.client模塊限制了傳輸數(shù)據(jù)量大小解決方案：
import http.client
http.client._MAXLINE = 655360
參考：https://stackoverflow.com/questions/63157046/python-http-request-exception-linetoolong

trino也可以批量直接寫入數(shù)據(jù)庫
https://github.com/trinodb/trino-python-client
engine從sqlchemey換到trino就行了
trino可以配合to_sql 參數(shù) 傳入 mulit

20211102

date_format(rq,'%Y-%m-%d')='2021-10-22'

日期轉(zhuǎn)字符

https://blog.csdn.net/u010711495/article/details/112290966
字符轉(zhuǎn)日期trino

20211102

select 常量
直接可以形成一列

20211015

CAST(order_nums AS decimal(10,4)
int轉(zhuǎn)浮點數(shù)

https://trino.io/docs/current/sql/create-table-as.html
建表 官方文檔

-- 建表測試
create table iceberg.ice_dwd.t2  (user_id) as  select user_id from iceberg.ice_dwd.dwd_dm_hz_b2b_new_user_register ;

連接數(shù)據(jù)庫
import trino
import pandas as pd
'''
python連接trino
'''
conn = trino.dbapi.connect(host='192.168.1.55',port=8881,user='root',catalog='iceberg',schema='ice_ods',
)
cur = conn.cursor()
cur.execute('SELECT * FROM ods_o_hz_b2b_tb_order_item limit 100' )
rows = cur.fetchall()

總結(jié)

以上是生活随笔為你收集整理的trinosql_prestosql问题的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。