函数调用次数与性能

在查询语句中，如果 Select 子句调用了较为耗时的函数或子查询，需要特别考虑函数调用次数对于SQL整体执行时间的影响。

一、数据准备，SQL 语句

  • 模拟较耗时的用户函数

确保执行子查询的时长是1秒。

create or replace function f001()

returns int stable language sql

as

$$

  select 1 from pg_sleep(1);

$$;
  • 模拟返回多行数据的子查询

结果集中，关联条件列含有重复值。

test=# create table t1 as select sn, id from (select generate_series(1, 3) sn),    (select generate_series(3, 8) id);

SELECT 18

test=# create table t2 as select generate_series(1, 5) id;

SELECT 5

test=# select * from t1;

 sn | id

----+----

  1 |  3

  2 |  3

  3 |  3

  1 |  4

  2 |  4

  3 |  4

  1 |  5

  2 |  5

  3 |  5

  1 |  6

  2 |  6

  3 |  6

  1 |  7

  2 |  7

  3 |  7

  1 |  8

  2 |  8

  3 |  8

(18 rows)

test=# select * from t2;

 id

----

  1

  2

  3

  4

  5

(5 rows)　　

二、查询全体数据的优化方案

1、初始SQL

with a as (select sn, id  from t1 )

select a.*, (select  id + f001()  sq_sum from t2 b where b.id = a.id)

from a

where 1 = 1;

 sn | id | sq_sum

----+----+--------

  1 |  3 |      4

  2 |  3 |      4

  3 |  3 |      4

  1 |  4 |      5

  2 |  4 |      5

  3 |  4 |      5

  1 |  5 |      6

  2 |  5 |      6

  3 |  5 |      6

  1 |  6 |

  2 |  6 |

  3 |  6 |

  1 |  7 |

  2 |  7 |

  3 |  7 |

  1 |  8 |

  2 |  8 |

  3 |  8 |

                                                 QUERY PLAN

-------------------------------------------------------------------------------------------------------------

 Seq Scan on t1  (cost=0.00..101806.05 rows=2260 width=12) (actual time=1000.638..9008.584 rows=18 loops=1)

   SubPlan 1

     ->  Seq Scan on t2 b  (cost=0.00..45.03 rows=13 width=4) (actual time=500.465..500.468 rows=0 loops=18)

           Filter: (id = t1.id)

           Rows Removed by Filter: 4

 Planning Time: 0.149 ms

 Execution Time: 9008.667 ms

(7 rows)

对于t1 表的每条记录，都要访问一次t2 ，如果 t2 表有对应的满足条件的记录，就要调用一次函数。

2、CTE

使用临时表的结果进行连接，避免循环。注意，这里CTE 有materilaized 选项，主要是把耗时的部分先执行出结果，避免与其他部分查询合并，引发多次执行。

with a as (select sn, id  from t1),

     b as materialized (select id, id + f001() sq_sum from t2 )

select a.*, (select sq_sum from b where b.id = a.id)

from a

where 1 = 1;

                                                  QUERY PLAN

---------------------------------------------------------------------------------------------------------------

 Seq Scan on t1  (cost=676.75..129868.35 rows=2260 width=12) (actual time=5004.377..5004.392 rows=18 loops=1)

   CTE b

     ->  Seq Scan on t2  (cost=0.00..676.75 rows=2540 width=8) (actual time=1000.562..5004.348 rows=5 loops=1)

   SubPlan 2

     ->  CTE Scan on b  (cost=0.00..57.15 rows=13 width=4) (actual time=166.821..278.021 rows=0 loops=18)

           Filter: (id = t1.id)

           Rows Removed by Filter: 4

 Planning Time: 0.107 ms

 Execution Time: 5004.441 ms

(9 rows)

3、LEFT JOIN 子查询

与上例一样，还是利用materialize 特性，避免了函数的多次调用。

with a as (select sn, id from t1)

select a.*, b.sq_sum

from a  left join (select id, id + f001() sq_sum  from t2 ) b on b.id = a.id

where 1 = 1 ;

                                                    QUERY PLAN

-------------------------------------------------------------------------------------------------------------------

 Nested Loop Left Join  (cost=0.00..86821.70 rows=28702 width=12) (actual time=3002.905..5005.627 rows=18 loops=1)

   Join Filter: (t2.id = t1.id)

   Rows Removed by Join Filter: 81

   ->  Seq Scan on t1  (cost=0.00..32.60 rows=2260 width=8) (actual time=0.007..0.011 rows=18 loops=1)

   ->  Materialize  (cost=0.00..689.45 rows=2540 width=8) (actual time=55.621..278.088 rows=5 loops=18)

         ->  Seq Scan on t2  (cost=0.00..676.75 rows=2540 width=8) (actual time=1001.176..5005.568 rows=5 loops=1)

 Planning Time: 0.180 ms

 Execution Time: 5005.650 ms

(8 rows)

三、查询局部数据（过滤主表的条件）的优化方案

1、初始SQL

with a as (select sn, id from t1)

select a.*, (select  id + f001() sq_sum from  t2  b where b.id = a.id)

from a

where 1 = 1 and a.id = 3;

 sn | id | sq_sum

----+----+--------

  1 |  3 |      4

  2 |  3 |      4

  3 |  3 |      4

(3 rows)

                                                  QUERY PLAN

--------------------------------------------------------------------------------------------------------------

 Seq Scan on t1  (cost=0.00..533.61 rows=11 width=12) (actual time=1000.356..3002.635 rows=3 loops=1)

   Filter: (id = 3)

   Rows Removed by Filter: 15

   SubPlan 1

     ->  Seq Scan on t2 b  (cost=0.00..45.03 rows=13 width=4) (actual time=1000.859..1000.866 rows=1 loops=3)

           Filter: (id = t1.id)

           Rows Removed by Filter: 4

 Planning Time: 0.164 ms

 Execution Time: 3002.657 ms

(9 rows)

2、CTE

with a as (select sn, id from t1),

     b as materialized (select id, id + f001() sq_sum from t2)

select a.*, (select sq_sum from b where b.id = a.id) sq_sum

from a

where 1 = 1 and a.id=3 ;

                                                  QUERY PLAN

---------------------------------------------------------------------------------------------------------------

 Seq Scan on t1  (cost=676.75..1343.65 rows=11 width=12) (actual time=5004.958..5004.965 rows=3 loops=1)

   Filter: (id = 3)

   Rows Removed by Filter: 15

   CTE b

     ->  Seq Scan on t2  (cost=0.00..676.75 rows=2540 width=8) (actual time=1001.387..5004.930 rows=5 loops=1)

   SubPlan 2

     ->  CTE Scan on b  (cost=0.00..57.15 rows=13 width=4) (actual time=1001.132..1668.316 rows=1 loops=3)

           Filter: (id = t1.id)

           Rows Removed by Filter: 4

 Planning Time: 0.105 ms

 Execution Time: 5004.983 ms

(11 rows)

3、LEFT JOIN 子查询

explain analyse

with a as (select sn, id from t1)

select a.*, b.sq_sum

from a left  join (select id, id + f001() sq_sum from t2) b on b.id = a.id

where 1 = 1 and a.id = 3 ;

                                                   QUERY PLAN

----------------------------------------------------------------------------------------------------------------

 Nested Loop Left Join  (cost=0.00..85.46 rows=143 width=12) (actual time=1000.972..1000.980 rows=3 loops=1)

   Join Filter: (t2.id = t1.id)

   ->  Seq Scan on t1  (cost=0.00..38.25 rows=11 width=8) (actual time=0.008..0.009 rows=3 loops=1)

         Filter: (id = 3)

         Rows Removed by Filter: 15

   ->  Materialize  (cost=0.00..45.10 rows=13 width=8) (actual time=333.654..333.656 rows=1 loops=3)

         ->  Seq Scan on t2  (cost=0.00..45.03 rows=13 width=8) (actual time=1000.959..1000.964 rows=1 loops=1)

               Filter: (id = 3)

               Rows Removed by Filter: 4

 Planning Time: 0.107 ms

 Execution Time: 1000.996 ms

(11 rows)

四、查询局部数据（过滤从表的条件）的优化方案

1、初始SQL

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id))

select a.*, (select  id + f001()  sq_sum from  (select  generate_series(1, 5) id) b where b.id = a.id) sq_sum

from a

where 1 = 1 and sq_sum = 6 ;


 sn | id | sq_sum

----+----+--------

  1 |  5 |      6

  2 |  5 |      6

  3 |  5 |      6

(3 行记录)

时间：6007.526 ms (00:06.008)

                                                         QUERY PLAN

----------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=0.00..3.43 rows=3 width=12) (actual time=4004.455..6006.681 rows=3 loops=1)

   ->  Subquery Scan on "SYSINTERNAL-4-1"  (cost=0.00..2.27 rows=1 width=4) (actual time=3003.273..3003.289 rows=1 loops=1)

         Filter: ((SubPlan 2) = 6)

         Rows Removed by Filter: 5

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.001..0.005 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

         SubPlan 2

           ->  Subquery Scan on b_1  (cost=0.00..0.36 rows=1 width=4) (actual time=500.541..500.544 rows=0 loops=6)

                 Filter: (b_1.id = "SYSINTERNAL-4-1".id)

                 Rows Removed by Filter: 4

                 ->  ProjectSet  (cost=0.00..0.04 rows=5 width=4) (actual time=0.001..0.003 rows=5 loops=6)

                       ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=6)

   ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.002..0.006 rows=3 loops=1)

         ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

   SubPlan 1

     ->  Subquery Scan on b  (cost=0.00..0.36 rows=1 width=4) (actual time=1001.116..1001.122 rows=1 loops=3)

           Filter: (b.id = "SYSINTERNAL-4-1".id)

           Rows Removed by Filter: 4

           ->  ProjectSet  (cost=0.00..0.04 rows=5 width=4) (actual time=0.001..0.006 rows=5 loops=3)

                 ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=3)

 Planning Time: 0.121 ms

 Execution Time: 6006.709 ms

(22 行记录)

时间：6007.303 ms (00:06.007)

2、CTE

explain analyse

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id)),

     b as (select id, id + f001() sq_sum from generate_series(1, 5) id)

select a.*, (select sq_sum from b where b.id = a.id) sq_sum

from a

where 1 = 1  and sq_sum = 6 ;

                                                          QUERY PLAN

-------------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=1.31..2.54 rows=3 width=12) (actual time=5005.414..5005.422 rows=3 loops=1)

   CTE b

     ->  Function Scan on generate_series id  (cost=0.00..1.31 rows=5 width=8) (actual time=1001.093..5005.376 rows=5 loops=1)

   ->  Subquery Scan on "SYSINTERNAL-4-1"  (cost=0.00..0.80 rows=1 width=4) (actual time=5005.406..5005.410 rows=1 loops=1)

         Filter: ((SubPlan 3) = 6)

         Rows Removed by Filter: 5

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.002..0.005 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

         SubPlan 3

           ->  CTE Scan on b b_1  (cost=0.00..0.11 rows=1 width=4) (actual time=500.540..834.232 rows=0 loops=6)

                 Filter: (id = "SYSINTERNAL-4-1".id)

                 Rows Removed by Filter: 4

   ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.002..0.003 rows=3 loops=1)

         ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.000 rows=1 loops=1)

   SubPlan 2

     ->  CTE Scan on b  (cost=0.00..0.11 rows=1 width=4) (actual time=0.001..0.001 rows=1 loops=3)

           Filter: (id = "SYSINTERNAL-4-1".id)

           Rows Removed by Filter: 4

 Planning Time: 0.143 ms

 Execution Time: 5005.449 ms

(20 行记录)

时间：5006.115 ms (00:05.006)

3、LEFT JOIN

explain analyse

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id))

select a.*, b.sq_sum

from a  left  join (select id, id + f001() sq_sum  from generate_series(1, 5) id) b on b.id = a.id

where 1 = 1 and sq_sum = 6 ;

                                                                  QUERY PLAN

-----------------------------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=1.39..1.62 rows=3 width=12) (actual time=5005.285..5005.291 rows=3 loops=1)

   ->  Hash Join  (cost=1.39..1.53 rows=1 width=8) (actual time=5005.281..5005.285 rows=1 loops=1)

         Hash Cond: ((generate_series(3, 8)) = b.id)

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.002..0.005 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.001..0.001 rows=1 loops=1)

         ->  Hash  (cost=1.38..1.38 rows=1 width=8) (actual time=5005.270..5005.270 rows=1 loops=1)

               Buckets: 1024  Batches: 1  Memory Usage: 9kB

               ->  Subquery Scan on b  (cost=0.00..1.38 rows=1 width=8) (actual time=5005.264..5005.266 rows=1 loops=1)

                     Filter: (b.sq_sum = 6)

                     Rows Removed by Filter: 4

                     ->  Function Scan on generate_series id  (cost=0.00..1.31 rows=5 width=8) (actual time=1001.089..5005.254 rows=5 loops=1)

   ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.002..0.003 rows=3 loops=1)

         ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

 Planning Time: 0.113 ms

 Execution Time: 5005.320 ms

(15 行记录)

时间：5005.873 ms (00:05.006)

4、LATERAL 连接 CTE

explain analyse

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id)),

     b as (select id, id + f001() sq_sum from generate_series(1, 5) id)

select a.*, b.sq_sum

from a join lateral (select * from b where b.id = a.id ) b on true

where 1 = 1  and sq_sum = 6;

                                                          QUERY PLAN

-------------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=1.44..1.67 rows=3 width=12) (actual time=5005.492..5005.499 rows=3 loops=1)

   CTE b

     ->  Function Scan on generate_series id  (cost=0.00..1.31 rows=5 width=8) (actual time=1001.103..5005.440 rows=5 loops=1)

   ->  Hash Join  (cost=0.12..0.27 rows=1 width=8) (actual time=5005.485..5005.489 rows=1 loops=1)

         Hash Cond: ((generate_series(3, 8)) = b.id)

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.002..0.006 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

         ->  Hash  (cost=0.11..0.11 rows=1 width=8) (actual time=5005.472..5005.472 rows=1 loops=1)

               Buckets: 1024  Batches: 1  Memory Usage: 9kB

               ->  CTE Scan on b  (cost=0.00..0.11 rows=1 width=8) (actual time=5005.457..5005.459 rows=1 loops=1)

                     Filter: (sq_sum = 6)

                     Rows Removed by Filter: 4

   ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.004..0.005 rows=3 loops=1)

         ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.001..0.001 rows=1 loops=1)

 Planning Time: 0.205 ms

 Execution Time: 5005.531 ms

(16 行记录)

时间：5006.765 ms (00:05.007)

5、LATERAL 连接子查询

explain analyse

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id))

select a.*, b.sq_sum

from a left join lateral (select id, id + f001() sq_sum  from generate_series(1, 5) id where id = a.id ) b on true

where 1 = 1 and sq_sum = 6 ;

                                                              QUERY PLAN

---------------------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=0.00..2.44 rows=18 width=12) (actual time=3003.267..3003.280 rows=3 loops=1)

   ->  Nested Loop  (cost=0.00..2.15 rows=6 width=8) (actual time=3003.257..3003.268 rows=1 loops=1)

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.002..0.007 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.002 rows=1 loops=1)

         ->  Subquery Scan on b  (cost=0.00..0.33 rows=1 width=4) (actual time=500.541..500.542 rows=0 loops=6)

               Filter: (b.sq_sum = 6)

               Rows Removed by Filter: 0

               ->  Function Scan on generate_series id  (cost=0.00..0.32 rows=1 width=8) (actual time=500.539..500.540 rows=0 loops=6)

                     Filter: (id = (generate_series(3, 8)))

                     Rows Removed by Filter: 4

   ->  Materialize  (cost=0.00..0.08 rows=3 width=4) (actual time=0.007..0.009 rows=3 loops=1)

         ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.005..0.006 rows=3 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.001..0.002 rows=1 loops=1)

 Planning Time: 0.125 ms

 Execution Time: 3003.306 ms

(15 行记录)

时间：3003.950 ms (00:03.004)

总结

  • Select 子句中，表达式会逐行运算，总时长与结果集成正比。
  • CTE子句，先计算全部结果，然后关联主表，总时长是可控。
  • 使用子查询和 LATERAL 连接，可以避免结果集的重复计算。

SQL优化篇之-如何减少耗时查询的调用次数的更多相关文章

  1. 【MySQL】SQL优化系列之 in与range 查询

    首先我们来说下in()这种方式的查询 在<高性能MySQL>里面提及用in这种方式可以有效的替代一定的range查询,提升查询效率,因为在一条索引里面,range字段后面的部分是不生效的. ...

  2. 聊聊数据库~4.SQL优化篇

    1.5.查询的艺术 上期回顾:https://www.cnblogs.com/dotnetcrazy/p/10399838.html 本节脚本:https://github.com/lotapp/Ba ...

  3. MyBatis动态SQL第一篇之实现多条件查询（if、where、trim标签）

    一.动态SQL概述 以前在使用JDBC操作数据时,如果查询条件特别多,将条件串联成SQL字符串是一件痛苦的事情.通常的解决方法是写很多的if-else条件语句对字符串进行拼接,并确保不能忘了空格或在字 ...

  4. MySQL优化篇（一），我可以和面试官多聊几句吗？——SQL优化流程与优化数据库对象

    我可以和面试官多聊几句吗?只是想偷点技能过来.MySQL优化篇(基于MySQL8.0测试验证),上部分:优化SQL语句.数据库对象,MyISAM表锁和InnoDB锁问题. MyISAM表锁和InnoD ...

  5. sql优化个人总结（全）

    sql优化总结--博客 第一次自己写博客,以后要坚持每掌握一个技能点,就要写一篇博客出来,做一个不满足于一个只会写if...else的程序员. 最近三个月入职了一家新的公司,做的是CRM系统,将公司多 ...

  6. MySQL中的sql优化

    目标: 掌握SQL调优的原则 掌握SQL调优的基本逻辑 掌握优秀SQL的编写方案 掌握何为慢SQL以及检测方案 SQL优化原则 1.减少数据量(表中数据太多可以分表,例如超过500万数据  双11一个 ...

  7. 常见SQL优化方法

    SQL优化的一些方法 1.对查询进行优化,应尽量避免全表扫描,首先应考虑在 where 及 order by 涉及的列上建立索引. 2.应尽量避免在 where 子句中对字段进行 null 值判断,否 ...

  8. sql优化的几种方式

    一.为什么要对SQL进行优化 我们开发项目上线初期,由于业务数据量相对较少,一些SQL的执行效率对程序运行效率的影响不太明显,而开发和运维人员也无法判断SQL对程序的运行效率有多大,故很少针对SQL进 ...

  9. SQL优化原理

    SQL优化过程: 1,捕获高负荷的SQL语句-->2得到SQL语句的执行计划和统计信息--->3分析SQL语句的执行计划和统计信息--->4采取措施,对SQL语句进行调整.1找出高负 ...

  10. SQL Server数据库性能优化之SQL语句篇【转】

    SQL Server数据库性能优化之SQL语句篇http://www.blogjava.net/allen-zhe/archive/2010/07/23/326927.html 近期项目需要, 做了一 ...

随机推荐

  1. Springboot+Freemarker+Boostrap实现用户增删改查实战

    说明 做java web用的2大模板语言分别是:thymeleaf和freemarker,thymeleaf前面已经用了很多了,所以今天用一下这个freemarker. 技术栈 springboot ...

  2. spring boot+layui分页实战

    项目用了layui,做了个简单的图书搜索页,分享出来. 喜欢的朋友给点个赞!!! 实现效果 开发步骤 1.前端页面和JS <!DOCTYPE html> <html xmlns=&q ...

  3. Java判断是否为闰年

    题目: 判断一个输入的整数是否为闰年? 1.需要对输入的数据类型进行验证 2.支持多次输入和结束符号判断,例如输入q代表退出程序. 分析: 闰年的判断规则如下: (1)若某个年份能被4整除但不能被10 ...

  4. Java并发编程实例--5.线程睡眠

    有时候我们需要让线程在一段时间内不做任何事.例如某线程每个一小时检测一下传感器,剩余的时间不做任何事. 我们可以使用sleep()方法使线程睡眠,此期间不占用计算机资源. 这个方法接受一个整数表示睡眠 ...

  5. Direct2D CreateHwndRenderTarget 和 CreateDCRenderTarget

    前段时间稍微看了点Direct3D, 觉得挺有意思的,但是想着要有3D得先从2D开始.故开始了D2D旅行. 如标题所示,CreateHwndRenderTarget 是在用来创建一个渲染到窗口的渲染目 ...

  6. win32-CreateDIBSection的使用

    使用CreateDIBSection 可以创建一个设备无关位图 #include <windows.h> using namespace std; int main() { HDC hdc ...

  7. 【Android 逆向】【攻防世界】人民的名义-抓捕赵德汉1-200

    1. 这一题下载下来是个jar文件,感觉很android关系不大,但还是放在了mobile这个分类下了 2. 直接java jar运行,提示需要输入密码 # java -jar 169e139f152 ...

  8. 以二进制文件安装K8S之高可用部署架构

    在Kubernetes系统中,Master节点扮演着总控中心的角色,通过不间断地与各个工作节点(Node)通信来维护整个集群的健康工作状态,集群中各资源对象的状态则被保存在etcd数据库中. 在正式环 ...

  9. cmake安装及报错解决办法

    安装 yum install cmake 报错 centOS8(x86_64 或 aarch64) 系统下 yum或dnf 默认安装的 cmake-3.18.2-11.el8版本,安装后无法使用,出现 ...

  10. jupyter notebook更改默认工作目录

    jupyter notebook默认配置路径:C:\Users\Administrator\.jupyter\jupyter_notebook_config.py 如果找不到配置文件,可以生成一个 j ...