mysql depended_query 优化案例一则

月度利息统计sql优化

原因：写的sql语句复杂，理解起来有难度，另一方面，查询性能比较低

原来的语句如下：

SELECT

tp.year,

tp.month,

tp.bid_id,

b.`title`,

DATE(b.`time`) `time`,

tp.receive_date,

u.`name`,

u.`reality_name`,

b.`amount`,

b.`apr`,

b.`period_unit`,

b.`period`,

tp.receive_interest,

tp.bid_invest_count,

IFNULL((SELECT SUM(s.`amount`) FROM t_invests s WHERE s.`bid_id`=tp.bid_id AND s.user_id IN (SELECT r.user_id FROM `t_user_repair` r)), 0)  AS bid_amount1,

IFNULL((SELECT SUM(s.`receive_corpus`) FROM t_bill_invests s WHERE s.`bid_id`=tp.bid_id AND DATE(IFNULL(s.`real_receive_time`, s.`receive_time`))=tp.`receive_date` AND s.user_id IN (SELECT r.user_id FROM `t_user_repair` r)), 0) AS bid_corpus1,

IFNULL((SELECT SUM(s.`receive_interest`) FROM t_bill_invests s WHERE s.`bid_id`=tp.bid_id AND DATE(IFNULL(s.`real_receive_time`, s.`receive_time`))=tp.`receive_date` AND s.user_id IN (SELECT r.user_id FROM `t_user_repair` r)), 0) AS bid_interest1,

IFNULL((SELECT COUNT(1) FROM t_bill_invests s WHERE s.`bid_id`=tp.bid_id AND DATE(IFNULL(s.`real_receive_time`, s.`receive_time`))=tp.`receive_date` AND s.user_id IN (SELECT r.user_id FROM `t_user_repair` r)), 0) AS bid_invest_count1, 

IFNULL((SELECT SUM(s.`receive_corpus`) FROM t_bill_invests s WHERE s.`bid_id`=tp.bid_id AND DATE(IFNULL(s.`real_receive_time`, s.`receive_time`))=tp.`receive_date` AND s.user_id NOT IN (SELECT r.user_id FROM `t_user_repair` r)), 0) AS bid_corpus2

FROM

(

SELECT

t.`bid_id`,

DATE(IFNULL(t.`real_receive_time`, t.`receive_time`)) AS `receive_date`,

YEAR(IFNULL(t.`real_receive_time`, t.`receive_time`)) AS `year`,

MONTH(IFNULL(t.`real_receive_time`, t.`receive_time`)) AS `month`,

 

IFNULL(SUM(t.`receive_interest`),0) receive_interest,

COUNT(1) AS bid_invest_count

FROM t_bill_invests t

WHERE 1=1 AND DATE(IFNULL(t.`real_receive_time`, t.`receive_time`)) >= '2015-09-01' AND DATE(IFNULL(t.`real_receive_time`, t.`receive_time`)) <= '2015-10-31'

GROUP BY `year`, `month`, t.`bid_id`, DATE(IFNULL(t.`real_receive_time`, t.`receive_time`))

)tp

LEFT JOIN t_bids b ON tp.bid_id=b.`id`

LEFT JOIN t_users u ON b.`user_id`=u.`id`;

执行结果所需时间信息如下：

/* Affected rows: 0  已找到记录: 129  警告: 0  持续时间 1 query: 28.704 sec. (+ 10.031 sec. network) */

最终优化语句：

SELECT

YEAR(IFNULL(tp.`real_receive_time`, tp.`receive_time`)) AS `year`,

MONTH(IFNULL(tp.`real_receive_time`, tp.`receive_time`)) AS `month`,

tp.bid_id,

b.`title`,

DATE(b.`time`) `time`,

DATE(IFNULL(tp.`real_receive_time`, tp.`receive_time`)) AS `receive_date`,

u.`name`, 

u.`reality_name`, 

b.`amount`,

b.`apr`, 

b.`period_unit`, 

b.`period`, 

sum(if(tp.real_receive_time,tp.`receive_interest`,0)) receive_interest, 

COUNT(1) AS bid_invest_count , 

sum(if(tp.user_id in (SELECT r.user_id FROM `t_user_repair` r) and tp.real_receive_time is not null,ti.amount,0 )) bid_amount1,

sum(if(tp.user_id in (SELECT r.user_id FROM `t_user_repair` r) and tp.real_receive_time is not null,tp.receive_corpus,0 )) bid_corpus1,

sum(if(tp.user_id in (SELECT r.user_id FROM `t_user_repair` r) and tp.real_receive_time is not null,tp.receive_interest,0 )) bid_interest1, 

sum(if(tp.user_id in (SELECT r.user_id FROM `t_user_repair` r),1,0 )) bid_invest_count1, 

sum(if(tp.user_id not in (SELECT r.user_id FROM `t_user_repair` r) and tp.real_receive_time is not null,tp.receive_corpus,0 )) bid_corpus2

 

FROM t_bill_invests tp

LEFT JOIN t_bids b ON tp.bid_id=b.`id`

LEFT JOIN t_users u ON b.`user_id`=u.`id`

left join (select id,amount from t_invests) ti on ti.id=tp.invest_id

 

WHERE 1=1 AND DATE(IFNULL(tp.`real_receive_time`, tp.`receive_time`)) >= '2015-09-01'  AND DATE(IFNULL(tp.`real_receive_time`, tp.`receive_time`)) <= '2015-10-31'

GROUP BY tp.`bid_id`, DATE(tp.`receive_time`);

 

执行时间:

/* Affected rows: 0  已找到记录: 129  警告: 0  持续时间 1 query: 0.671 sec. */

结果一样，但最终结果只需0.67s

优化思路，原来语句explain中：

在select_type中有大量的dependent_subquery,此种类型的查询极耗性能，在sql的编写中应该极力避免。

1. 结合业务需求，查看group by写法，发现没有必要写那么多的group by，只需要抓住核心的tp.`bid_id`, DATE(tp.`receive_time`)便可确定一组帐单信息。

2.  尽量改写查询列，让其为subquery,如下：

SELECT

YEAR(IFNULL(tp.`real_receive_time`, tp.`receive_time`)) AS `year`,

MONTH(IFNULL(tp.`real_receive_time`, tp.`receive_time`)) AS `month`,

tp.bid_id,

b.`title`,

DATE(b.`time`) `time`, 

DATE(IFNULL(tp.`real_receive_time`, tp.`receive_time`)) AS `receive_date`, 

u.`name`,

u.`reality_name`, 

b.`amount`,

b.`apr`, 

b.`period_unit`, 

b.`period`, 

sum(if(tp.real_receive_time,tp.`receive_interest`,0)) receive_interest, 

COUNT(1) AS bid_invest_count ,

IFNULL((SELECT SUM(s.`amount`) FROM t_invests s    WHERE s.`bid_id`=tp.bid_id and s.user_id IN (SELECT r.user_id FROM `t_user_repair` r)), 0) as bid_amount1, 

sum(if(tp.user_id in (SELECT r.user_id FROM `t_user_repair` r) and tp.real_receive_time is not null,tp.receive_corpus,0 )) bid_corpus1, 

sum(if(tp.user_id in (SELECT r.user_id FROM `t_user_repair` r) and tp.real_receive_time is not null,tp.receive_interest,0 )) bid_interest1,

sum(if(tp.user_id in (SELECT r.user_id FROM `t_user_repair` r),1,0 )) bid_invest_count1, 

sum(if(tp.user_id not in (SELECT r.user_id FROM `t_user_repair` r) and tp.real_receive_time is not null,tp.receive_corpus,0 )) bid_corpus2 

FROM t_bill_invests tp

LEFT JOIN t_bids b ON tp.bid_id=b.`id`

LEFT JOIN t_users u ON b.`user_id`=u.`id`

 

WHERE 1=1 AND DATE(IFNULL(tp.`real_receive_time`, tp.`receive_time`)) >= '2015-04-01'  AND DATE(IFNULL(tp.`real_receive_time`, tp.`receive_time`)) <= '2015-10-31'

GROUP BY tp.`bid_id`, DATE(tp.`receive_time`);

在其explain中还有残留的依赖外查询：

主要是由框中的语句引起，于是再改写，让其改成临时表连接，就是最后的sql语句：

只剩临时表与了查询，性能有了大幅度的提升，优化结束。

总结：统计类查询经常不小心就会写成depended_subquery的形式，可以结合sum count case if条件判断来改写，让其可以达到subquery的形式，另一方面也可以改成临时表的方式并通过连接来提高性能。