因?yàn)榻拥竭@個(gè)需求的時(shí)候，一開始想寫Sql語句直接實(shí)現(xiàn)，發(fā)現(xiàn)比較麻煩；后來想通過Mysql存儲(chǔ)過程來處理，但以前對(duì)Oracle存儲(chǔ)過程熟悉，MySql存儲(chǔ)過程代碼不熟悉，要使用起來有一定的學(xué)習(xí)成本；最后選擇用Python控制實(shí)現(xiàn)。謹(jǐn)此以記，供以后類似需要做參考。

需求描述：

一般的統(tǒng)計(jì)對(duì)格式如：”dim1,dim2,index1,index2,date“，如果維度對(duì)應(yīng)的指標(biāo)不存在，則結(jié)果表中就沒有這個(gè)記錄。現(xiàn)在要求，指標(biāo)為空的記錄給補(bǔ)上，且將其指標(biāo)置為0。

實(shí)現(xiàn)代碼:

/Users/nisj/PycharmProjects/EsDataProc/result_null_proc.py

# -*- coding=utf-8 -*-

import os

import re

def resultNullProc():

result_data = os.popen("""mysql -hMysqlHost -P6603 -uHadoop -pMysqlPass -N -e "use funnyai_data; \

select appkey,app_source,week_count \

from x_chushou_user_compare_week_stat \

where app_source <>'' \

group by appkey,app_source,week_count; \

" """ ).readlines();

nrpd_list = []

for nrp_list in result_data:

nrp = re.split('\t', nrp_list.replace('\n', ''))

nrpd_list.append(nrp)

for nrpd in nrpd_list:

appkey = nrpd[0]

appsource = nrpd[1]

week_count = nrpd[2]

result_data1 = os.popen("""mysql -hMysqlHost -P6603 -uhadoop -pMysqlPass -N -e "use funnyai_data; \

select pm,appkey,app_source,week_count,compare_week_count from \

(SELECT (SELECT COUNT(1) + 1 \

FROM (select distinct remain_week from bi_user_remain_pay_byweek ) A \

WHERE concat(substr(A.remain_week,1,4),lpad(substr(A.remain_week,6),2,0)) < concat(substr(B.remain_week,1,4),lpad(substr(B.remain_week,6),2,0))) pm \

FROM (select distinct remain_week from bi_user_remain_pay_byweek ) B \

ORDER BY PM) a1 \

left join ( \

select appkey,app_source,week_count,compare_week_count \

from x_chushou_user_compare_week_stat \

where appkey='%s' and app_source='%s' and week_count='%s' \

) a2 on a1.pm=a2.compare_week_count \

where a1.pm>'%s' and a2.appkey is null \

order by a1.pm;" """ % (appkey, appsource, week_count, week_count)).readlines();

nrpd_list = []

for nrp_list in result_data1:

nrp = re.split('\t', nrp_list.replace('\n', ''))

nrpd_list.append(nrp)

for nrpd in nrpd_list:

pm = nrpd[0]

# print pm,appkey,appsource,week_count

os.system("""mysql -hMysqlHost -P6603 -uhadoop -pMysqlPass -N -e "use funnyai_data; \

insert into x_chushou_user_compare_week_stat(appkey,app_source,week_count,compare_week_count,compare_identify_count,compare_register_user_count,compare_user_count,compare_pay_amount,compare_pay_user_count) \

select '%s','%s','%s','%s','%d','%d','%d','%d','%d'; \

" """ % (appkey, appsource, week_count, pm, 0, 0, 0, 0, 0))

resultNullProc()

總結(jié)

以上是生活随笔為你收集整理的python数据组织存在维度吗_用Python将统计数据不存在的记录按维度对应指标补齐...的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。