GaussDB(DWS)案例丨MERGE场景下语句不下推引起的性能瓶颈问题
本文分享自华为云社区《GaussDB(DWS)性能调优：MERGE场景下语句不下推引起的性能瓶颈问题案例》，作者：O泡果奶~。
1、【问题描述】
语句执行时间过长，且该语句performance执行计划中SQL Diagnostic Information显示SQL语句不下推，理由为：Type of Record in dual that is not a real table can not be shipped
2、【原始语句】
merge into fin_drt_act.rt_act_apd_npd_rmk_sto_his_dtl_f T using
(SELECT 'Y' del_flag FROM DUAL) S on
( T.PERIOD_ID = 202308 AND NOT EXISTS
(SELECT 1 FROM FIN_DRT_ACT.RT_ACT_APD_NPD_RMK_TO_STO_V S WHERE T.UNIQUE_ID = S.UNIQUE_ID AND T.PERIOD_ID = S.PERIOD_ID ))
WHEN MATCHED THEN UPDATE SET T.del_flag=S.del_flag
3、【性能分析】
改写前SQL语句performance执行计划
id | operation | A-time | A-rows | E-rows | E-distinct | Peak Memory | A-width | E-width | E-costs
----+---------------------------------------------------------------------------------------------------------------+------------+--------+--------+------------+-------------+---------+---------+---------
1 | -> Merge on fin_drt_act.rt_act_apd_npd_rmk_sto_his_dtl_f t | 806911.617 | 17948 | 17140 | | 74KB | | 498 | 501.74
2 | -> Nested Loop (3,4) | 3795.279 | 17948 | 17140 | | 66KB | | 498 | 501.74
3 | -> Result | 0.003 | 1 | 1 | | 24KB | | 0 | 0.01
4 | -> Hash Anti Join (5, 6) | 3741.930 | 17948 | 17140 | | 56KB | | 498 | 330.32
5 | -> Data Node Scan on rt_act_apd_npd_rmk_sto_his_dtl_f "_REMOTE_TABLE_QUERY_" | 3291.507 | 17948 | 17400 | | 24832KB | | 498 | 0.00
6 | -> Hash | 384.317 | 7 | 261 | | 320KB | | 554 | 54.23
7 | -> Hash Right Join (8, 14) | 384.287 | 7 | 261 | | 32KB | | 554 | 54.23
8 | -> Subquery Scan on t2 | 6.915 | 198 | 236 | | 32KB | | 18 | 26.41
9 | -> WindowAgg | 6.862 | 198 | 236 | | 64KB | | 18 | 23.46
10 | -> Sort | 6.730 | 198 | 236 | | 72KB | | 18 | 16.97
11 | -> Subquery Scan on t | 5.650 | 198 | 236 | | 64KB | | 18 | 7.08
12 | -> HashAggregate | 5.535 | 198 | 236 | | 104KB | | 28 | 4.72
13 | -> Data Node Scan on rt_act_mca_dc_user_privilege_dtl_f "_REMOTE_TABLE_QUERY_" | 5.378 | 198 | 236 | | 64KB | | 28 | 0.00
14 | -> Hash | 377.009 | 7 | 260 | | 312KB | | 2102 | 0.00
15 | -> Data Node Scan on apd_npd_rmk_to_sto_tmp "_REMOTE_TABLE_QUERY_" | 376.991 | 7 | 260 | | 64KB | | 2102 | 0.00 SQL Diagnostic Information
---------------------------------------------------------------------------
SQL is not plan-shipping
reason: Type of Record in dual that is not a real table can not be shipped Predicate Information (identified by plan id)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1 --Merge on fin_drt_act.rt_act_apd_npd_rmk_sto_his_dtl_f t
Merge Updated: 17948
4 --Hash Anti Join (5, 6)
Hash Cond: (((t.unique_id)::text = ((t1.period_id)::text || (t1.unique_id)::text)) AND (t.period_id = (t1.period_id)::numeric))
7 --Hash Right Join (8, 14)
Hash Cond: ((t2.ssc_code)::text = (t1.ssc_code)::text)
Join Filter: (((t1.company_code)::text = (CASE WHEN ((t2.company_code)::text = 'ALL'::text) THEN t1.company_code ELSE t2.company_code END)::text) AND ((t1.account_code)::text = (CASE WHEN ((t2.account_number)::text = 'ALL'::text) THEN t1.account_code ELSE t2.account_number END)::text))
Rows Removed by Join Filter: 119
8 --Subquery Scan on t2
Filter: (t2.rownumber = 1) Memory Information (identified by plan id)
---------------------------------------------------------------------------------
1 --Merge on fin_drt_act.rt_act_apd_npd_rmk_sto_his_dtl_f t
Peak Memory: 74KB, Estimate Memory: 2048MB
2 --Nested Loop (3,4)
Peak Memory: 66KB, Estimate Memory: 2048MB
3 --Result
Peak Memory: 24KB, Estimate Memory: 2048MB
4 --Hash Anti Join (5, 6)
Peak Memory: 56KB, Estimate Memory: 2048MB
5 --Data Node Scan on rt_act_apd_npd_rmk_sto_his_dtl_f "_REMOTE_TABLE_QUERY_"
Peak Memory: 24832KB, Estimate Memory: 2048MB
6 --Hash
Peak Memory: 320KB, Width: 56
Buckets: 32768 Batches: 1 Memory Usage: 1kB
7 --Hash Right Join (8, 14)
Peak Memory: 32KB, Estimate Memory: 2048MB
8 --Subquery Scan on t2
Peak Memory: 32KB, Estimate Memory: 2048MB
9 --WindowAgg
Peak Memory: 64KB, Estimate Memory: 2048MB
10 --Sort
Peak Memory: 72KB, Estimate Memory: 2048MB
11 --Subquery Scan on t
Peak Memory: 64KB, Estimate Memory: 2048MB
12 --HashAggregate
Peak Memory: 104KB, Estimate Memory: 2048MB
13 --Data Node Scan on rt_act_mca_dc_user_privilege_dtl_f "_REMOTE_TABLE_QUERY_"
Peak Memory: 64KB, Estimate Memory: 2048MB
14 --Hash
Peak Memory: 312KB, Width: 76
Buckets: 32768 Batches: 1 Memory Usage: 1kB
15 --Data Node Scan on apd_npd_rmk_to_sto_tmp "_REMOTE_TABLE_QUERY_"
Peak Memory: 64KB, Estimate Memory: 2048MB Targetlist Information (identified by plan id)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1 --Merge on fin_drt_act.rt_act_apd_npd_rmk_sto_his_dtl_f t
Node/s: All datanodes
Remote query: UPDATE ONLY fin_drt_act.rt_act_apd_npd_rmk_sto_his_dtl_f t SET seq_id = $47, batch_number = $48, approved_status = $49, reference_id = $50, enabled_flag = $51, operate_flag = $52, original_period = $53, carry_flag = $54, account_period_id = $55, period_id = $56, ssc_code = $57, ssc_cn_name = $58, ssc_en_name = $59, company_code = $60, source_name = $61, ic = $62, lc_dr_amt_je = $63, lc_cr_amt_je = $64, ptd_je = $65, lc_dr_tb = $66, lc_cr_tb = $67, ptd_tb = $68, dif_ptd = $69, account_code = $70, schedule_end_time = $71, remark = $72, status = $73, status_name = $74, exception_type = $75, exception_type_name = $76, approved_by = $77, approve_comment = $78, approve_date = $79, unique_id = $80, created_by = $81, creation_date = $82, last_updated_by = $83, last_update_date = $84, last_update_login = $85, error_message_cn = $86, error_message_en = $87, del_flag = $88 WHERE t.ctid = $44 AND t.xc_node_id = $45 AND t.tableoid = $46
2 --Nested Loop (3,4)
Output: 'Y'::text, t.seq_id, t.batch_number, t.approved_status, t.reference_id, t.enabled_flag, t.operate_flag, t.original_period, t.carry_flag, t.account_period_id, t.period_id, t.ssc_code, t.ssc_cn_name, t.ssc_en_name, t.company_code, t.source_name, t.ic, t.lc_dr_amt_je, t.lc_cr_amt_je, t.ptd_je, t.lc_dr_tb, t.lc_cr_tb, t.ptd_tb, t.dif_ptd, t.account_code, t.schedule_end_time, t.remark, t.status, t.status_name, t.exception_type, t.exception_type_name, t.approved_by, t.approve_comment, t.approve_date, t.unique_id, t.created_by, t.creation_date, t.last_updated_by, t.last_update_date, t.last_update_login, t.error_message_cn, t.error_message_en, t.del_flag, t.ctid, t.xc_node_id, t.tableoid, t.seq_id, t.batch_number, t.approved_status, t.reference_id, t.enabled_flag, t.operate_flag, t.original_period, t.carry_flag, t.account_period_id, t.period_id, t.ssc_code, t.ssc_cn_name, t.ssc_en_name, t.company_code, t.source_name, t.ic, t.lc_dr_amt_je, t.lc_cr_amt_je, t.ptd_je, t.lc_dr_tb, t.lc_cr_tb, t.ptd_tb, t.dif_ptd, t.account_code, t.schedule_end_time, t.remark, t.status, t.status_name, t.exception_type, t.exception_type_name, t.approved_by, t.approve_comment, t.approve_date, t.unique_id, t.created_by, t.creation_date, t.last_updated_by, t.last_update_date, t.last_update_login, t.error_message_cn, t.error_message_en, 'Y'::character varying(1)
4 --Hash Anti Join (5, 6)
Output: t.seq_id, t.batch_number, t.approved_status, t.reference_id, t.enabled_flag, t.operate_flag, t.original_period, t.carry_flag, t.account_period_id, t.period_id, t.ssc_code, t.ssc_cn_name, t.ssc_en_name, t.company_code, t.source_name, t.ic, t.lc_dr_amt_je, t.lc_cr_amt_je, t.ptd_je, t.lc_dr_tb, t.lc_cr_tb, t.ptd_tb, t.dif_ptd, t.account_code, t.schedule_end_time, t.remark, t.status, t.status_name, t.exception_type, t.exception_type_name, t.approved_by, t.approve_comment, t.approve_date, t.unique_id, t.created_by, t.creation_date, t.last_updated_by, t.last_update_date, t.last_update_login, t.error_message_cn, t.error_message_en, t.del_flag, t.ctid, t.xc_node_id, t.tableoid
5 --Data Node Scan on rt_act_apd_npd_rmk_sto_his_dtl_f "_REMOTE_TABLE_QUERY_"
Output: t.seq_id, t.batch_number, t.approved_status, t.reference_id, t.enabled_flag, t.operate_flag, t.original_period, t.carry_flag, t.account_period_id, t.period_id, t.ssc_code, t.ssc_cn_name, t.ssc_en_name, t.company_code, t.source_name, t.ic, t.lc_dr_amt_je, t.lc_cr_amt_je, t.ptd_je, t.lc_dr_tb, t.lc_cr_tb, t.ptd_tb, t.dif_ptd, t.account_code, t.schedule_end_time, t.remark, t.status, t.status_name, t.exception_type, t.exception_type_name, t.approved_by, t.approve_comment, t.approve_date, t.unique_id, t.created_by, t.creation_date, t.last_updated_by, t.last_update_date, t.last_update_login, t.error_message_cn, t.error_message_en, t.del_flag, t.ctid, t.xc_node_id, t.tableoid
Node/s: All datanodes
Remote query: SELECT seq_id, batch_number, approved_status, reference_id, enabled_flag, operate_flag, original_period, carry_flag, account_period_id, period_id, ssc_code, ssc_cn_name, ssc_en_name, company_code, source_name, ic, lc_dr_amt_je, lc_cr_amt_je, ptd_je, lc_dr_tb, lc_cr_tb, ptd_tb, dif_ptd, account_code, schedule_end_time, remark, status, status_name, exception_type, exception_type_name, approved_by, approve_comment, approve_date, unique_id, created_by, creation_date, last_updated_by, last_update_date, last_update_login, error_message_cn, error_message_en, del_flag, ctid, xc_node_id, tableoid FROM ONLY fin_drt_act.rt_act_apd_npd_rmk_sto_his_dtl_f t WHERE period_id = 202308::numeric
6 --Hash
Output: t1.period_id, t1.unique_id
7 --Hash Right Join (8, 14)
Output: t1.period_id, t1.unique_id
8 --Subquery Scan on t2
Output: t2.ssc_code, t2.company_code, t2.account_number, t2.level_key, t2.rownumber
9 --WindowAgg
Output: t.ssc_code, t.company_code, t.account_number, (CASE WHEN (((t.company_code)::text = 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 0 WHEN (((t.company_code)::text <> 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 1 ELSE 2 END), dense_rank() OVER (PARTITION BY t.ssc_code ORDER BY (CASE WHEN (((t.company_code)::text = 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 0 WHEN (((t.company_code)::text <> 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 1 ELSE 2 END) USING = NULLS LAST)
10 --Sort
Output: t.ssc_code, (CASE WHEN (((t.company_code)::text = 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 0 WHEN (((t.company_code)::text <> 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 1 ELSE 2 END), t.company_code, t.account_number
Sort Key: t.ssc_code, (CASE WHEN (((t.company_code)::text = 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 0 WHEN (((t.company_code)::text <> 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 1 ELSE 2 END)
11 --Subquery Scan on t
Output: t.ssc_code, CASE WHEN (((t.company_code)::text = 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 0 WHEN (((t.company_code)::text <> 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 1 ELSE 2 END, t.company_code, t.account_number
12 --HashAggregate
Output: t1.ssc_code, t1.company_code, t1.account_number, t1.user_id
Group By Key: t1.ssc_code, t1.company_code, t1.account_number, t1.user_id
13 --Data Node Scan on rt_act_mca_dc_user_privilege_dtl_f "_REMOTE_TABLE_QUERY_"
Output: t1.ssc_code, t1.company_code, t1.account_number, t1.user_id
Node/s: All datanodes
Remote query: SELECT ssc_code, company_code, account_number, user_id FROM ONLY fin_drt_act.rt_act_mca_dc_user_privilege_dtl_f t1 WHERE flow_type::text = 'EC_GL'::text AND chek_type::text = 'EC_GL_STO'::text AND begin_date <= pg_systimestamp()::timestamp(0) without time zone AND end_date >= pg_systimestamp()::timestamp(0) without time zone
14 --Hash
Output: t1.ssc_code, t1.company_code, t1.account_code, t1.period_id, t1.unique_id
15 --Data Node Scan on apd_npd_rmk_to_sto_tmp "_REMOTE_TABLE_QUERY_"
Output: t1.ssc_code, t1.company_code, t1.account_code, t1.period_id, t1.unique_id
Node/s: All datanodes
Remote query: SELECT ssc_code, company_code, account_code, period_id, unique_id FROM ONLY fin_drt_act.apd_npd_rmk_to_sto_tmp t1 WHERE true Datanode Information (identified by plan id)
-----------------------------------------------------------------------------------------------
1 --Merge on fin_drt_act.rt_act_apd_npd_rmk_sto_his_dtl_f t
(actual time=806911.616..806911.617 rows=17948 loops=1)
(Buffers: shared hit=6)
(CPU: ex c/r=116340958, ex row=17948, ex cyc=2088087526316, inc cyc=2097953048602)
2 --Nested Loop (3,4)
(actual time=561.052..3795.279 rows=17948 loops=1) (projection time=23.340)
(CPU: ex c/r=7752, ex row=17949, ex cyc=139143070, inc cyc=9865522286)
3 --Result
(actual time=0.001..0.003 rows=1 loops=1) (projection time=0.001)
(CPU: ex c/r=4680, ex row=1, ex cyc=4680, inc cyc=4680)
4 --Hash Anti Join (5, 6)
(actual time=561.044..3741.930 rows=17948 loops=1) (projection time=7.143)
(Buffers: shared hit=6)
(CPU: ex c/r=9584, ex row=17955, ex cyc=172088156, inc cyc=9726374536)
5 --Data Node Scan on rt_act_apd_npd_rmk_sto_his_dtl_f "_REMOTE_TABLE_QUERY_"
(actual time=176.472..3291.507 rows=17948 loops=1)
(Buffers: 0)
(CPU: ex c/r=476658, ex row=17948, ex cyc=8555072428, inc cyc=8555072428)
6 --Hash
(actual time=384.317..384.317 rows=7 loops=1)
(Buffers: shared hit=6)
(CPU: ex c/r=10954, ex row=7, ex cyc=76680, inc cyc=999213952)
7 --Hash Right Join (8, 14)
(actual time=384.028..384.287 rows=7 loops=1) (filter time=0.043 projection time=0.000)
(Buffers: shared hit=6)
(CPU: ex c/r=4613, ex row=205, ex cyc=945852, inc cyc=999137272)
8 --Subquery Scan on t2
(actual time=6.751..6.915 rows=198 loops=1) (filter time=0.018)
(CPU: ex c/r=682, ex row=198, ex cyc=135038, inc cyc=17975308)
9 --WindowAgg
(actual time=6.741..6.862 rows=198 loops=1) (projection time=0.016)
(Buffers: shared hit=6)
(CPU: ex c/r=1848, ex row=198, ex cyc=366028, inc cyc=17840270)
10 --Sort
(actual time=6.716..6.730 rows=198 loops=1)
(Buffers: shared hit=6)
(CPU: ex c/r=14173, ex row=198, ex cyc=2806302, inc cyc=17474242)
11 --Subquery Scan on t
(actual time=5.524..5.650 rows=198 loops=1) (projection time=0.067)
(CPU: ex c/r=1406, ex row=198, ex cyc=278492, inc cyc=14667940)
12 --HashAggregate
(actual time=5.490..5.535 rows=198 loops=1) (projection time=0.003)
(Buffers: 0)
(CPU: ex c/r=2111, ex row=198, ex cyc=418106, inc cyc=14389448)
13 --Data Node Scan on rt_act_mca_dc_user_privilege_dtl_f "_REMOTE_TABLE_QUERY_"
(actual time=2.249..5.378 rows=198 loops=1)
(Buffers: 0)
(CPU: ex c/r=70562, ex row=198, ex cyc=13971342, inc cyc=13971342)
14 --Hash
(actual time=377.009..377.009 rows=7 loops=1)
(Buffers: 0)
(CPU: ex c/r=8018, ex row=7, ex cyc=56128, inc cyc=980216112)
15 --Data Node Scan on apd_npd_rmk_to_sto_tmp "_REMOTE_TABLE_QUERY_"
(actual time=128.393..376.991 rows=7 loops=1)
(Buffers: 0)
(CPU: ex c/r=140022854, ex row=7, ex cyc=980159984, inc cyc=980159984) ====== Query Summary =====
-------------------------------------------------------------
Remote query poll time: 169.602 ms, Deserialze time: 7.760 ms
Remote query poll time: 4.858 ms, Deserialze time: 0.006 ms
Remote query poll time: 319.716 ms, Deserialze time: 0.000 ms
Enqueue time: 0.034 ms
Coordinator executor start time: 0.551 ms
Coordinator executor run time: 806912.768 ms
Coordinator executor end time: 17.822 ms
Parser runtime: 0.000 ms
Planner runtime: 4.840 ms
Query Id: 83598068858005619
Unique SQL Id: 4289851310
Total runtime: 806936.261 ms
业务使用时，会使用MERGE INTO语句实现类似UPSERT的功能。这种场景下MERGE INTO语句的USING部分的数据为VALUES子句，USING部分的书写方式可能导致MERGE INTO语句的执行不下推。
本文SQL语句中，可以看出不下推原因是dual表且VALUES子句中并没有使用到该表，因此对USING子句的SQL语句进行改写，以便整个SQL语句可以下推。
改写语句
merge into fin_drt_act.rt_act_apd_npd_rmk_sto_his_dtl_f T using
(SELECT 'Y' del_flag) S on
( T.PERIOD_ID = 202308 AND NOT EXISTS
(SELECT 1 FROM FIN_DRT_ACT.RT_ACT_APD_NPD_RMK_TO_STO_V S WHERE T.UNIQUE_ID = S.UNIQUE_ID AND T.PERIOD_ID = S.PERIOD_ID ))
WHEN MATCHED THEN UPDATE SET T.del_flag=S.del_flag
改写后语句verbose执行计划
id | operation | E-rows | E-distinct | E-memory | E-width | E-costs
----+----------------------------------------------------------------------------------------------------------------+--------+------------+----------+---------+---------
1 | -> Row Adapter | 1 | | | 572 | 2838.74
2 | -> Vector Streaming (type: GATHER) | 1 | | | 572 | 2838.74
3 | -> Vector Merge on fin_drt_act.rt_act_apd_npd_rmk_sto_his_dtl_f t | 17592 | | 48MB | 572 | 2834.97
4 | -> Vector Hash Anti Join (5, 9) | 17592 | | 16MB | 572 | 2834.97
5 | -> Vector Nest Loop (6,8) | 17852 | 686 | 1MB | 532 | 1873.40
6 | -> Vector Adapter | 26 | | 1MB | 0 | 0.01
7 | -> Result | 26 | | 1MB | 0 | 0.01
8 | -> CStore Scan on fin_drt_act.rt_act_apd_npd_rmk_sto_his_dtl_f t | 17852 | | 1MB | 500 | 1866.51
9 | -> Vector Streaming(type: BROADCAST) | 6838 | 52 | 2MB | 594 | 947.28
10 | -> Vector Hash Right Join (11, 20) | 263 | | 16MB | 594 | 706.57
11 | -> Vector Streaming(type: BROADCAST) | 6162 | 47 | 2MB | 30 | 674.47
12 | -> Vector Subquery Scan on t2 | 237 | | 1MB | 30 | 631.14
13 | -> Vector WindowAgg | 237 | | 16MB | 18 | 631.03
14 | -> Vector Sort | 237 | | 16MB | 18 | 630.56
15 | -> Vector Streaming(type: REDISTRIBUTE) | 234 | | 2MB | 18 | 629.85
16 | -> Vector Subquery Scan on t | 234 | | 1MB | 18 | 629.51
17 | -> Vector Sonic Hash Aggregate | 234 | | 16MB | 28 | 629.42
18 | -> Vector Streaming(type: REDISTRIBUTE) | 236 | | 2MB | 28 | 629.24
19 | -> CStore Scan on fin_drt_act.rt_act_mca_dc_user_privilege_dtl_f t1 | 236 | | 1MB | 28 | 628.94
20 | -> Vector Partition Iterator | 260 | 30 | 1MB | 2112 | 30.03
21 | -> Partitioned CStore Scan on fin_drt_act.apd_npd_rmk_to_sto_tmp t1 | 260 | | 1MB | 2112 | 30.03 Predicate Information (identified by plan id)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
4 --Vector Hash Anti Join (5, 9)
Hash Cond: (((t.unique_id)::text = ((t1.period_id)::text || (t1.unique_id)::text)) AND (t.period_id = (t1.period_id)::numeric))
8 --CStore Scan on fin_drt_act.rt_act_apd_npd_rmk_sto_his_dtl_f t
Filter: (t.period_id = 202308::numeric)
Pushdown Predicate Filter: (t.period_id = 202308::numeric)
10 --Vector Hash Right Join (11, 20)
Hash Cond: ((t2.ssc_code)::text = (t1.ssc_code)::text)
Join Filter: (((t1.company_code)::text = (CASE WHEN ((t2.company_code)::text = 'ALL'::text) THEN t1.company_code ELSE t2.company_code END)::text) AND ((t1.account_code)::text = (CASE WHEN ((t2.account_number)::text = 'ALL'::text) THEN t1.account_code ELSE t2.account_number END)::text))
12 --Vector Subquery Scan on t2
Filter: (t2.rownumber = 1)
17 --Vector Sonic Hash Aggregate
Skew Agg Optimized by Statistic
19 --CStore Scan on fin_drt_act.rt_act_mca_dc_user_privilege_dtl_f t1
Filter: (((t1.flow_type)::text = 'EC_GL'::text) AND ((t1.chek_type)::text = 'EC_GL_STO'::text) AND (t1.begin_date <= (pg_systimestamp())::timestamp(0) without time zone) AND (t1.end_date >= (pg_systimestamp())::timestamp(0) without time zone))
Pushdown Predicate Filter: (((t1.flow_type)::text = 'EC_GL'::text) AND ((t1.chek_type)::text = 'EC_GL_STO'::text) AND (t1.begin_date <= (pg_systimestamp())::timestamp(0) without time zone) AND (t1.end_date >= (pg_systimestamp())::timestamp(0) without time zone))
20 --Vector Partition Iterator
Iterations: 120
21 --Partitioned CStore Scan on fin_drt_act.apd_npd_rmk_to_sto_tmp t1
Partitions Selected by Static Prune: 1..120 Targetlist Information (identified by plan id)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2 --Vector Streaming (type: GATHER)
Node/s: All datanodes
4 --Vector Hash Anti Join (5, 9)
Output: ('Y'::text), t.seq_id, t.batch_number, t.approved_status, t.reference_id, t.enabled_flag, t.operate_flag, t.original_period, t.carry_flag, t.account_period_id, t.period_id, t.ssc_code, t.ssc_cn_name, t.ssc_en_name, t.company_code, t.source_name, t.ic, t.lc_dr_amt_je, t.lc_cr_amt_je, t.ptd_je, t.lc_dr_tb, t.lc_cr_tb, t.ptd_tb, t.dif_ptd, t.account_code, t.schedule_end_time, t.remark, t.status, t.status_name, t.exception_type, t.exception_type_name, t.approved_by, t.approve_comment, t.approve_date, t.unique_id, t.created_by, t.creation_date, t.last_updated_by, t.last_update_date, t.last_update_login, t.error_message_cn, t.error_message_en, t.del_flag, t.ctid, t.xc_node_id, t.tableoid, ('Y'::text), t1.ctid, t1.tableoid, t2.ssc_code, t2.company_code, t2.account_number, t2.level_key, t2.rownumber
5 --Vector Nest Loop (6,8)
Output: ('Y'::text), t.seq_id, t.batch_number, t.approved_status, t.reference_id, t.enabled_flag, t.operate_flag, t.original_period, t.carry_flag, t.account_period_id, t.period_id, t.ssc_code, t.ssc_cn_name, t.ssc_en_name, t.company_code, t.source_name, t.ic, t.lc_dr_amt_je, t.lc_cr_amt_je, t.ptd_je, t.lc_dr_tb, t.lc_cr_tb, t.ptd_tb, t.dif_ptd, t.account_code, t.schedule_end_time, t.remark, t.status, t.status_name, t.exception_type, t.exception_type_name, t.approved_by, t.approve_comment, t.approve_date, t.unique_id, t.created_by, t.creation_date, t.last_updated_by, t.last_update_date, t.last_update_login, t.error_message_cn, t.error_message_en, t.del_flag, t.ctid, t.xc_node_id, t.tableoid
6 --Vector Adapter
Output: 'Y'::text
7 --Result
Output: 'Y'::text
8 --CStore Scan on fin_drt_act.rt_act_apd_npd_rmk_sto_his_dtl_f t
Output: t.seq_id, t.batch_number, t.approved_status, t.reference_id, t.enabled_flag, t.operate_flag, t.original_period, t.carry_flag, t.account_period_id, t.period_id, t.ssc_code, t.ssc_cn_name, t.ssc_en_name, t.company_code, t.source_name, t.ic, t.lc_dr_amt_je, t.lc_cr_amt_je, t.ptd_je, t.lc_dr_tb, t.lc_cr_tb, t.ptd_tb, t.dif_ptd, t.account_code, t.schedule_end_time, t.remark, t.status, t.status_name, t.exception_type, t.exception_type_name, t.approved_by, t.approve_comment, t.approve_date, t.unique_id, t.created_by, t.creation_date, t.last_updated_by, t.last_update_date, t.last_update_login, t.error_message_cn, t.error_message_en, t.del_flag, t.ctid, t.xc_node_id, t.tableoid
Distribute Key: t.seq_id
9 --Vector Streaming(type: BROADCAST)
Output: t1.ctid, t1.tableoid, t1.period_id, t1.unique_id, t2.ssc_code, t2.company_code, t2.account_number, t2.level_key, t2.rownumber
Spawn on: All datanodes
Consumer Nodes: All datanodes
10 --Vector Hash Right Join (11, 20)
Output: t1.ctid, t1.tableoid, t1.period_id, t1.unique_id, t2.ssc_code, t2.company_code, t2.account_number, t2.level_key, t2.rownumber
11 --Vector Streaming(type: BROADCAST)
Output: t2.ssc_code, t2.company_code, t2.account_number, t2.level_key, t2.rownumber
Spawn on: All datanodes
Consumer Nodes: All datanodes
12 --Vector Subquery Scan on t2
Output: t2.ssc_code, t2.company_code, t2.account_number, t2.level_key, t2.rownumber
13 --Vector WindowAgg
Output: t.ssc_code, t.company_code, t.account_number, (CASE WHEN (((t.company_code)::text = 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 0 WHEN (((t.company_code)::text <> 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 1 ELSE 2 END), dense_rank() OVER (PARTITION BY t.ssc_code ORDER BY (CASE WHEN (((t.company_code)::text = 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 0 WHEN (((t.company_code)::text <> 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 1 ELSE 2 END) USING = NULLS LAST)
14 --Vector Sort
Output: t.ssc_code, (CASE WHEN (((t.company_code)::text = 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 0 WHEN (((t.company_code)::text <> 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 1 ELSE 2 END), t.company_code, t.account_number
Sort Key: t.ssc_code, (CASE WHEN (((t.company_code)::text = 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 0 WHEN (((t.company_code)::text <> 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 1 ELSE 2 END)
15 --Vector Streaming(type: REDISTRIBUTE)
Output: t.ssc_code, (CASE WHEN (((t.company_code)::text = 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 0 WHEN (((t.company_code)::text <> 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 1 ELSE 2 END), t.company_code, t.account_number
Distribute Key: t.ssc_code
Spawn on: All datanodes
Consumer Nodes: All datanodes
16 --Vector Subquery Scan on t
Output: t.ssc_code, CASE WHEN (((t.company_code)::text = 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 0 WHEN (((t.company_code)::text <> 'ALL'::text) AND ((t.account_number)::text = 'ALL'::text)) THEN 1 ELSE 2 END, t.company_code, t.account_number
17 --Vector Sonic Hash Aggregate
Output: t1.ssc_code, t1.company_code, t1.account_number, t1.user_id
Group By Key: t1.ssc_code, t1.company_code, t1.account_number, t1.user_id
18 --Vector Streaming(type: REDISTRIBUTE)
Output: t1.ssc_code, t1.company_code, t1.account_number, t1.user_id
Distribute Key: t1.account_number
Spawn on: All datanodes
Consumer Nodes: All datanodes
19 --CStore Scan on fin_drt_act.rt_act_mca_dc_user_privilege_dtl_f t1
Output: t1.ssc_code, t1.company_code, t1.account_number, t1.user_id
Distribute Key: t1.id
20 --Vector Partition Iterator
Output: t1.ctid, t1.tableoid, t1.ssc_code, t1.company_code, t1.account_code, t1.period_id, t1.unique_id
21 --Partitioned CStore Scan on fin_drt_act.apd_npd_rmk_to_sto_tmp t1
Output: t1.ctid, t1.tableoid, t1.ssc_code, t1.company_code, t1.account_code, t1.period_id, t1.unique_id
Distribute Key: t1.row_number ====== Query Summary =====
-------------------------------
System available mem: 2097152KB
Query Max mem: 2097152KB
Query estimated mem: 131072KB
Parser runtime: 0.102 ms
Planner runtime: 4.602 ms
Unique SQL Id: 1168204269
可以看出，SQL语句能够下推，且经过业务验证，语句执行时间下降为10s左右。
GaussDB(DWS)案例丨MERGE场景下语句不下推引起的性能瓶颈问题的更多相关文章
- GaussDB(DWS)运维：导致SQL执行不下推的改写方案
摘要:本文就针对因USING子句的书写方式可能导致MERGE INTO语句的执行不下推的场景,对USING子句的SQL语句进行改写一遍,整个SQL语句可以下推. 本文分享自华为云社区<Gauss ...
- GaussDB(DWS)应用实践丨负载管理与作业排队处理方法
摘要:本文用来总结一些GaussDB(DWS)在实际应用过程中,可能出现的各种作业排队的情况,以及出现排队时,我们应该怎么去判断是否正常,调整一些参数,让资源分配与负载管理更符合当前的业务:或者在作业 ...
- GaussDB(DWS)现网案例：collation报错
摘要:用户创建hash分布表,使用pbe方式执行使用分布列作为查询条件的语句时报错 本文分享自华为云社区<GaussDB(DWS)现网案例之collation报错>,作者: 你是猴子请来的 ...
- 从数据仓库双集群系统模式探讨，看GaussDB(DWS)的容灾设计
摘要:本文主要是探讨OLAP关系型数据库框架的数据仓库平台如何设计双集群系统,即增强系统高可用的保障水准,然后讨论一下GaussDB(DWS)的容灾应该如何设计. 当前社会.企业运行当中,大数据分析. ...
- 十八般武艺玩转GaussDB(DWS)性能调优：路径干预
摘要:路径生成是表关联方式确定的主要阶段,本文介绍了几个影响路径生成的要素:cost_param, scan方式,join方式,stream方式,并从原理上分析如何干预路径的生成. 一.cost模型选 ...
- 一文详解GaussDB(DWS) 的并发管控和内存管控
摘要:DWS的负载管理分为两层,第一层为cn的全局并发控制,第二层为资源池级别的并发控制. 本文分享自华为云社区<GaussDB(DWS) 并发管控&内存管控>,作者: fight ...
- GaussDB(DWS)应用实战：对被视图引用的表进行DDL操作
摘要:GaussDB(DWS)是从Postgres演进过来的,像Postgres一样,如果表被视图引用的话,特定场景下,部分DDL操作是不能直接执行的. 背景说明 GaussDB(DWS)是从Post ...
- 详解GaussDB(DWS) explain分布式执行计划
摘要:本文主要介绍如何详细解读GaussDB(DWS)产生的分布式执行计划,从计划中发现性能调优点. 前言 执行计划(又称解释计划)是数据库执行SQL语句的具体步骤,例如通过索引还是全表扫描访问表中的 ...
- 十八般武艺玩转GaussDB(DWS)性能调优（三）：好味道表定义
摘要:表结构设计是数据库建模的一个关键环节,表定义好坏直接决定了集群的有效容量以及业务查询性能,本文从产品架构.功能实现以及业务特征的角度阐述在GaussDB(DWS)的中表定义时需要关注的一些关键因 ...
- 十八般武艺玩转GaussDB(DWS)性能调优：SQL改写
摘要:本文将系统介绍在GaussDB(DWS)系统中影响性能的坏味道SQL及SQL模式,帮助大家能够从原理层面尽快识别这些坏味道SQL,在调优过程中及时发现问题,进行整改. 数据库的应用中,充斥着坏味 ...
随机推荐
- kubernetes 概述
云原生的发展 云原生是一条最佳路径或者最佳实践.更详细的说,云原生为用户指定了一条低心智负担的.敏捷的.能够以可扩展.可复制的方式最大化地利用云的能力.发挥云的价值的最佳路径.因此,云原生其实是一套指 ...
- ExtJS的使用方法汇总(1)——配置和表格控件使用
在网上差一些关于ExtJS的相关资料,看到这篇博客写的不错,拿出来分享一下! 博客文章:ExtJS的使用方法汇总(1)--配置和表格控件使用 ExtJS的使用方法汇总(2)- ...
- javascript继承的 6 种方法
1. 原型链继承 2. 借用构造函数继承 3. 组合继承(原型+借用构造) 4. 原型式继承 5. 寄生式继承 6. 寄生组合式继承
- wireshark数据包时间戳修改
1.数据包格式 两种数据包格式,pcap和pcapng两种.pcapng为升级版,时间戳细粒度更高. 2.时间戳位置(arrival time) 时间戳在数据包中表现为物理层的arrival time ...
- Gmail如何开启SMTP/POP
1. 登录Gmail账号,右上角点击设置图标 -> 查看所有设置,如图 2. 点击"转发和POP/IMAP",如图 3. 开启IMAP和POP,选择"对所有邮件启用 ...
- CSS必学：元素之间的空白与行内块的幽灵空白问题
作者:WangMin 格言:努力做好自己喜欢的每一件事 CSDN原创文章 博客地址 WangMin 我们在开发的过程中,难免会出现一些难以预料的问题.那么其中,CSS空白现象就是非常常见的问题之一.虽 ...
- js判断字符串数据类型
mounted() { this.isJSON('{"key":1232,"a":2},{"key":1232,"a": ...
- 文心一言 VS 讯飞星火 VS chatgpt （150）-- 算法导论12.2 6题
六.用go语言,考虑一棵二叉搜索树 T ,其关键字互不相同.证明:如果 T 中一个结点 x 的右子树为空,且 x 有一个后继 y ,那么 y 一定是 x 的最底层祖先,并且其左孩子也是 x 的祖先.( ...
- 如何用java的虚拟线程连接数据库
我觉得这个很简单 首先确保你idea支持jdk21. 然后把idea编译成的目标字节码设置为21版本的 然后编写代码. 创建虚拟线程的方式有: Runnable runnable = () -> ...
- 如何对连续型数据进行离散化处理，并进行OneHot编码？
如何对连续型数据进行离散化处理,并进行OneHot编码,最终将OneHot编码作为特征因子输入模型? 什么是OneHot编码 One-Hot编码是分类变量作为二进制向量的表示.这首先要求将分类值映射到 ...