需要生成1-2個(gè)標(biāo)志全表字段的字段,根據(jù)該字段進(jìn)行數(shù)據(jù)變化的判斷,如:MD5_FILED(存主鍵)和NON_MD5_FIELD(存主鍵以外的必要業(yè)務(wù)字段); MD5生成可以如下:select listagg(to_char(column_name)) from user_table_columns left join user_cons_columns on t2.position is not null where columnname not in (技術(shù)字段)
主鍵 業(yè)務(wù)主鍵和開(kāi)始時(shí)間;
輔助字段,跑批時(shí)間、跑批字段、跑批批次等
拉鏈表加工
重跑機(jī)制
刪除當(dāng)前批次之后開(kāi)鏈的數(shù)據(jù)
delete from 表 where start_date >=batch_date and jobname = '123'
回退閉鏈的數(shù)據(jù)
update 表 set end_date = date'29991231',load_date = sysdate,del_flag = '未刪除狀態(tài)'where end_date>= batch_date and end_date <> date'29991231'and jobname = '123'
邏輯加工
增量數(shù)據(jù)插入臨時(shí)表(邏輯加工階段)
更新臨時(shí)表加密字段(fn_md5()函數(shù)加密成32位,char類型)
update 表 set md5_field = fn_md5()
刪除臨時(shí)表沒(méi)有變化的數(shù)據(jù)
delete from 臨時(shí)表 t where t.del_flag = '未刪除標(biāo)示符' and exists(select 1from 表 tt where tt.md5 = t.md5 and tt.startdate <= batchdate and tt.enddate > batchdate)
閉鏈刪除的數(shù)據(jù)
update 表 tt set enddate =batchdate,loaddate = sysdate,delflag=‘刪除’where exists(select 1 from 臨時(shí)表 t where delflag=‘刪除’ tt.主鍵= t.主鍵)and tt.startdate <= batchdate and tt.enddate > batchdate)
閉鏈修改的數(shù)據(jù)
merge into 表 ttusing(select * from 臨時(shí)表 t delflag =‘未刪除’)ton(t.主鍵 = tt.主鍵)when matched then update set enddate =batchdate,loaddate = sysdate
開(kāi)鏈新增和修改的數(shù)據(jù)
insert into 表
select * from 臨時(shí)表 where delflag = ‘未刪除’