生活随笔
收集整理的這篇文章主要介紹了
请问：hive中avg聚合函数会使用到combiner功能吗？
小編覺得挺不錯的,現在分享給大家,幫大家做個參考.
例如下面這條SQL， 肯定是用上了combiner功能的
select deptno, sum(sal) as sum_sal from emp group by deptno
hive (test)> explain select deptno, sum(sal) as sum_sal from emp group by deptno;
OK
Explain
STAGE DEPENDENCIES:Stage-1 is a root stageStage-0 depends on stages: Stage-1STAGE PLANS:Stage: Stage-1Map ReduceMap Operator Tree:TableScanalias: empStatistics: Num rows: 5 Data size: 603 Basic stats: COMPLETE Column stats: NONESelect Operatorexpressions: deptno (type: int), sal (type: decimal(22,2))outputColumnNames: deptno, salStatistics: Num rows: 5 Data size: 603 Basic stats: COMPLETE Column stats: NONEGroup By Operatoraggregations: sum(sal)keys: deptno (type: int)mode: hashoutputColumnNames: _col0, _col1Statistics: Num rows: 5 Data size: 603 Basic stats: COMPLETE Column stats: NONEReduce Output Operatorkey expressions: _col0 (type: int)sort order: +Map-reduce partition columns: _col0 (type: int)Statistics: Num rows: 5 Data size: 603 Basic stats: COMPLETE Column stats: NONEvalue expressions: _col1 (type: decimal(32,2))Reduce Operator Tree:Group By Operatoraggregations: sum(VALUE._col0)keys: KEY._col0 (type: int)mode: mergepartialoutputColumnNames: _col0, _col1Statistics: Num rows: 2 Data size: 241 Basic stats: COMPLETE Column stats: NONEFile Output Operatorcompressed: falseStatistics: Num rows: 2 Data size: 241 Basic stats: COMPLETE Column stats: NONEtable:input format: org.apache.hadoop.mapred.SequenceFileInputFormatoutput format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormatserde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDeStage: Stage-0Fetch Operatorlimit: -1Processor Tree:ListSink
如果是這個SQL，是否能用上combiner功能？？ 之前學習的時候說combiner不能處理avg這種函數的
select deptno, avg(sal) as avg_sal from emp group by deptno
我看執行計劃和使用sum聚合函數無差異
hive (test)> explain select deptno, avg(sal) as avg_sal from emp group by deptno;
OK
Explain
STAGE DEPENDENCIES:Stage-1 is a root stageStage-0 depends on stages: Stage-1STAGE PLANS:Stage: Stage-1Map ReduceMap Operator Tree:TableScanalias: empStatistics: Num rows: 5 Data size: 603 Basic stats: COMPLETE Column stats: NONESelect Operatorexpressions: deptno (type: int), sal (type: decimal(22,2))outputColumnNames: deptno, salStatistics: Num rows: 5 Data size: 603 Basic stats: COMPLETE Column stats: NONEGroup By Operatoraggregations: avg(sal)keys: deptno (type: int)mode: hashoutputColumnNames: _col0, _col1Statistics: Num rows: 5 Data size: 603 Basic stats: COMPLETE Column stats: NONEReduce Output Operatorkey expressions: _col0 (type: int)sort order: +Map-reduce partition columns: _col0 (type: int)Statistics: Num rows: 5 Data size: 603 Basic stats: COMPLETE Column stats: NONEvalue expressions: _col1 (type: struct<count:bigint,sum:decimal(32,2),input:decimal(22,2)>)Reduce Operator Tree:Group By Operatoraggregations: avg(VALUE._col0)keys: KEY._col0 (type: int)mode: mergepartialoutputColumnNames: _col0, _col1Statistics: Num rows: 2 Data size: 241 Basic stats: COMPLETE Column stats: NONEFile Output Operatorcompressed: falseStatistics: Num rows: 2 Data size: 241 Basic stats: COMPLETE Column stats: NONEtable:input format: org.apache.hadoop.mapred.SequenceFileInputFormatoutput format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormatserde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDeStage: Stage-0Fetch Operatorlimit: -1Processor Tree:ListSink
總結
以上是生活随笔為你收集整理的请问：hive中avg聚合函数会使用到combiner功能吗？的全部內容，希望文章能夠幫你解決所遇到的問題。
如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。
