函数调用次数与性能

在查询语句中，如果 Select 子句调用了较为耗时的函数或子查询，需要特别考虑函数调用次数对于SQL整体执行时间的影响。

一、数据准备，SQL 语句

  • 模拟较耗时的用户函数

确保执行子查询的时长是1秒。

create or replace function f001()

returns int stable language sql

as

$$

  select 1 from pg_sleep(1);

$$;
  • 模拟返回多行数据的子查询

结果集中，关联条件列含有重复值。

test=# create table t1 as select sn, id from (select generate_series(1, 3) sn),    (select generate_series(3, 8) id);

SELECT 18

test=# create table t2 as select generate_series(1, 5) id;

SELECT 5

test=# select * from t1;

 sn | id

----+----

  1 |  3

  2 |  3

  3 |  3

  1 |  4

  2 |  4

  3 |  4

  1 |  5

  2 |  5

  3 |  5

  1 |  6

  2 |  6

  3 |  6

  1 |  7

  2 |  7

  3 |  7

  1 |  8

  2 |  8

  3 |  8

(18 rows)

test=# select * from t2;

 id

----

  1

  2

  3

  4

  5

(5 rows)　　

二、查询全体数据的优化方案

1、初始SQL

with a as (select sn, id  from t1 )

select a.*, (select  id + f001()  sq_sum from t2 b where b.id = a.id)

from a

where 1 = 1;

 sn | id | sq_sum

----+----+--------

  1 |  3 |      4

  2 |  3 |      4

  3 |  3 |      4

  1 |  4 |      5

  2 |  4 |      5

  3 |  4 |      5

  1 |  5 |      6

  2 |  5 |      6

  3 |  5 |      6

  1 |  6 |

  2 |  6 |

  3 |  6 |

  1 |  7 |

  2 |  7 |

  3 |  7 |

  1 |  8 |

  2 |  8 |

  3 |  8 |

                                                 QUERY PLAN

-------------------------------------------------------------------------------------------------------------

 Seq Scan on t1  (cost=0.00..101806.05 rows=2260 width=12) (actual time=1000.638..9008.584 rows=18 loops=1)

   SubPlan 1

     ->  Seq Scan on t2 b  (cost=0.00..45.03 rows=13 width=4) (actual time=500.465..500.468 rows=0 loops=18)

           Filter: (id = t1.id)

           Rows Removed by Filter: 4

 Planning Time: 0.149 ms

 Execution Time: 9008.667 ms

(7 rows)

对于t1 表的每条记录，都要访问一次t2 ，如果 t2 表有对应的满足条件的记录，就要调用一次函数。

2、CTE

使用临时表的结果进行连接，避免循环。注意，这里CTE 有materilaized 选项，主要是把耗时的部分先执行出结果，避免与其他部分查询合并，引发多次执行。

with a as (select sn, id  from t1),

     b as materialized (select id, id + f001() sq_sum from t2 )

select a.*, (select sq_sum from b where b.id = a.id)

from a

where 1 = 1;

                                                  QUERY PLAN

---------------------------------------------------------------------------------------------------------------

 Seq Scan on t1  (cost=676.75..129868.35 rows=2260 width=12) (actual time=5004.377..5004.392 rows=18 loops=1)

   CTE b

     ->  Seq Scan on t2  (cost=0.00..676.75 rows=2540 width=8) (actual time=1000.562..5004.348 rows=5 loops=1)

   SubPlan 2

     ->  CTE Scan on b  (cost=0.00..57.15 rows=13 width=4) (actual time=166.821..278.021 rows=0 loops=18)

           Filter: (id = t1.id)

           Rows Removed by Filter: 4

 Planning Time: 0.107 ms

 Execution Time: 5004.441 ms

(9 rows)

3、LEFT JOIN 子查询

与上例一样，还是利用materialize 特性，避免了函数的多次调用。

with a as (select sn, id from t1)

select a.*, b.sq_sum

from a  left join (select id, id + f001() sq_sum  from t2 ) b on b.id = a.id

where 1 = 1 ;

                                                    QUERY PLAN

-------------------------------------------------------------------------------------------------------------------

 Nested Loop Left Join  (cost=0.00..86821.70 rows=28702 width=12) (actual time=3002.905..5005.627 rows=18 loops=1)

   Join Filter: (t2.id = t1.id)

   Rows Removed by Join Filter: 81

   ->  Seq Scan on t1  (cost=0.00..32.60 rows=2260 width=8) (actual time=0.007..0.011 rows=18 loops=1)

   ->  Materialize  (cost=0.00..689.45 rows=2540 width=8) (actual time=55.621..278.088 rows=5 loops=18)

         ->  Seq Scan on t2  (cost=0.00..676.75 rows=2540 width=8) (actual time=1001.176..5005.568 rows=5 loops=1)

 Planning Time: 0.180 ms

 Execution Time: 5005.650 ms

(8 rows)

三、查询局部数据（过滤主表的条件）的优化方案

1、初始SQL

with a as (select sn, id from t1)

select a.*, (select  id + f001() sq_sum from  t2  b where b.id = a.id)

from a

where 1 = 1 and a.id = 3;

 sn | id | sq_sum

----+----+--------

  1 |  3 |      4

  2 |  3 |      4

  3 |  3 |      4

(3 rows)

                                                  QUERY PLAN

--------------------------------------------------------------------------------------------------------------

 Seq Scan on t1  (cost=0.00..533.61 rows=11 width=12) (actual time=1000.356..3002.635 rows=3 loops=1)

   Filter: (id = 3)

   Rows Removed by Filter: 15

   SubPlan 1

     ->  Seq Scan on t2 b  (cost=0.00..45.03 rows=13 width=4) (actual time=1000.859..1000.866 rows=1 loops=3)

           Filter: (id = t1.id)

           Rows Removed by Filter: 4

 Planning Time: 0.164 ms

 Execution Time: 3002.657 ms

(9 rows)

2、CTE

with a as (select sn, id from t1),

     b as materialized (select id, id + f001() sq_sum from t2)

select a.*, (select sq_sum from b where b.id = a.id) sq_sum

from a

where 1 = 1 and a.id=3 ;

                                                  QUERY PLAN

---------------------------------------------------------------------------------------------------------------

 Seq Scan on t1  (cost=676.75..1343.65 rows=11 width=12) (actual time=5004.958..5004.965 rows=3 loops=1)

   Filter: (id = 3)

   Rows Removed by Filter: 15

   CTE b

     ->  Seq Scan on t2  (cost=0.00..676.75 rows=2540 width=8) (actual time=1001.387..5004.930 rows=5 loops=1)

   SubPlan 2

     ->  CTE Scan on b  (cost=0.00..57.15 rows=13 width=4) (actual time=1001.132..1668.316 rows=1 loops=3)

           Filter: (id = t1.id)

           Rows Removed by Filter: 4

 Planning Time: 0.105 ms

 Execution Time: 5004.983 ms

(11 rows)

3、LEFT JOIN 子查询

explain analyse

with a as (select sn, id from t1)

select a.*, b.sq_sum

from a left  join (select id, id + f001() sq_sum from t2) b on b.id = a.id

where 1 = 1 and a.id = 3 ;

                                                   QUERY PLAN

----------------------------------------------------------------------------------------------------------------

 Nested Loop Left Join  (cost=0.00..85.46 rows=143 width=12) (actual time=1000.972..1000.980 rows=3 loops=1)

   Join Filter: (t2.id = t1.id)

   ->  Seq Scan on t1  (cost=0.00..38.25 rows=11 width=8) (actual time=0.008..0.009 rows=3 loops=1)

         Filter: (id = 3)

         Rows Removed by Filter: 15

   ->  Materialize  (cost=0.00..45.10 rows=13 width=8) (actual time=333.654..333.656 rows=1 loops=3)

         ->  Seq Scan on t2  (cost=0.00..45.03 rows=13 width=8) (actual time=1000.959..1000.964 rows=1 loops=1)

               Filter: (id = 3)

               Rows Removed by Filter: 4

 Planning Time: 0.107 ms

 Execution Time: 1000.996 ms

(11 rows)

四、查询局部数据（过滤从表的条件）的优化方案

1、初始SQL

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id))

select a.*, (select  id + f001()  sq_sum from  (select  generate_series(1, 5) id) b where b.id = a.id) sq_sum

from a

where 1 = 1 and sq_sum = 6 ;


 sn | id | sq_sum

----+----+--------

  1 |  5 |      6

  2 |  5 |      6

  3 |  5 |      6

(3 行记录)

时间：6007.526 ms (00:06.008)

                                                         QUERY PLAN

----------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=0.00..3.43 rows=3 width=12) (actual time=4004.455..6006.681 rows=3 loops=1)

   ->  Subquery Scan on "SYSINTERNAL-4-1"  (cost=0.00..2.27 rows=1 width=4) (actual time=3003.273..3003.289 rows=1 loops=1)

         Filter: ((SubPlan 2) = 6)

         Rows Removed by Filter: 5

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.001..0.005 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

         SubPlan 2

           ->  Subquery Scan on b_1  (cost=0.00..0.36 rows=1 width=4) (actual time=500.541..500.544 rows=0 loops=6)

                 Filter: (b_1.id = "SYSINTERNAL-4-1".id)

                 Rows Removed by Filter: 4

                 ->  ProjectSet  (cost=0.00..0.04 rows=5 width=4) (actual time=0.001..0.003 rows=5 loops=6)

                       ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=6)

   ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.002..0.006 rows=3 loops=1)

         ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

   SubPlan 1

     ->  Subquery Scan on b  (cost=0.00..0.36 rows=1 width=4) (actual time=1001.116..1001.122 rows=1 loops=3)

           Filter: (b.id = "SYSINTERNAL-4-1".id)

           Rows Removed by Filter: 4

           ->  ProjectSet  (cost=0.00..0.04 rows=5 width=4) (actual time=0.001..0.006 rows=5 loops=3)

                 ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=3)

 Planning Time: 0.121 ms

 Execution Time: 6006.709 ms

(22 行记录)

时间：6007.303 ms (00:06.007)

2、CTE

explain analyse

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id)),

     b as (select id, id + f001() sq_sum from generate_series(1, 5) id)

select a.*, (select sq_sum from b where b.id = a.id) sq_sum

from a

where 1 = 1  and sq_sum = 6 ;

                                                          QUERY PLAN

-------------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=1.31..2.54 rows=3 width=12) (actual time=5005.414..5005.422 rows=3 loops=1)

   CTE b

     ->  Function Scan on generate_series id  (cost=0.00..1.31 rows=5 width=8) (actual time=1001.093..5005.376 rows=5 loops=1)

   ->  Subquery Scan on "SYSINTERNAL-4-1"  (cost=0.00..0.80 rows=1 width=4) (actual time=5005.406..5005.410 rows=1 loops=1)

         Filter: ((SubPlan 3) = 6)

         Rows Removed by Filter: 5

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.002..0.005 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

         SubPlan 3

           ->  CTE Scan on b b_1  (cost=0.00..0.11 rows=1 width=4) (actual time=500.540..834.232 rows=0 loops=6)

                 Filter: (id = "SYSINTERNAL-4-1".id)

                 Rows Removed by Filter: 4

   ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.002..0.003 rows=3 loops=1)

         ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.000 rows=1 loops=1)

   SubPlan 2

     ->  CTE Scan on b  (cost=0.00..0.11 rows=1 width=4) (actual time=0.001..0.001 rows=1 loops=3)

           Filter: (id = "SYSINTERNAL-4-1".id)

           Rows Removed by Filter: 4

 Planning Time: 0.143 ms

 Execution Time: 5005.449 ms

(20 行记录)

时间：5006.115 ms (00:05.006)

3、LEFT JOIN

explain analyse

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id))

select a.*, b.sq_sum

from a  left  join (select id, id + f001() sq_sum  from generate_series(1, 5) id) b on b.id = a.id

where 1 = 1 and sq_sum = 6 ;

                                                                  QUERY PLAN

-----------------------------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=1.39..1.62 rows=3 width=12) (actual time=5005.285..5005.291 rows=3 loops=1)

   ->  Hash Join  (cost=1.39..1.53 rows=1 width=8) (actual time=5005.281..5005.285 rows=1 loops=1)

         Hash Cond: ((generate_series(3, 8)) = b.id)

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.002..0.005 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.001..0.001 rows=1 loops=1)

         ->  Hash  (cost=1.38..1.38 rows=1 width=8) (actual time=5005.270..5005.270 rows=1 loops=1)

               Buckets: 1024  Batches: 1  Memory Usage: 9kB

               ->  Subquery Scan on b  (cost=0.00..1.38 rows=1 width=8) (actual time=5005.264..5005.266 rows=1 loops=1)

                     Filter: (b.sq_sum = 6)

                     Rows Removed by Filter: 4

                     ->  Function Scan on generate_series id  (cost=0.00..1.31 rows=5 width=8) (actual time=1001.089..5005.254 rows=5 loops=1)

   ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.002..0.003 rows=3 loops=1)

         ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

 Planning Time: 0.113 ms

 Execution Time: 5005.320 ms

(15 行记录)

时间：5005.873 ms (00:05.006)

4、LATERAL 连接 CTE

explain analyse

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id)),

     b as (select id, id + f001() sq_sum from generate_series(1, 5) id)

select a.*, b.sq_sum

from a join lateral (select * from b where b.id = a.id ) b on true

where 1 = 1  and sq_sum = 6;

                                                          QUERY PLAN

-------------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=1.44..1.67 rows=3 width=12) (actual time=5005.492..5005.499 rows=3 loops=1)

   CTE b

     ->  Function Scan on generate_series id  (cost=0.00..1.31 rows=5 width=8) (actual time=1001.103..5005.440 rows=5 loops=1)

   ->  Hash Join  (cost=0.12..0.27 rows=1 width=8) (actual time=5005.485..5005.489 rows=1 loops=1)

         Hash Cond: ((generate_series(3, 8)) = b.id)

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.002..0.006 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

         ->  Hash  (cost=0.11..0.11 rows=1 width=8) (actual time=5005.472..5005.472 rows=1 loops=1)

               Buckets: 1024  Batches: 1  Memory Usage: 9kB

               ->  CTE Scan on b  (cost=0.00..0.11 rows=1 width=8) (actual time=5005.457..5005.459 rows=1 loops=1)

                     Filter: (sq_sum = 6)

                     Rows Removed by Filter: 4

   ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.004..0.005 rows=3 loops=1)

         ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.001..0.001 rows=1 loops=1)

 Planning Time: 0.205 ms

 Execution Time: 5005.531 ms

(16 行记录)

时间：5006.765 ms (00:05.007)

5、LATERAL 连接子查询

explain analyse

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id))

select a.*, b.sq_sum

from a left join lateral (select id, id + f001() sq_sum  from generate_series(1, 5) id where id = a.id ) b on true

where 1 = 1 and sq_sum = 6 ;

                                                              QUERY PLAN

---------------------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=0.00..2.44 rows=18 width=12) (actual time=3003.267..3003.280 rows=3 loops=1)

   ->  Nested Loop  (cost=0.00..2.15 rows=6 width=8) (actual time=3003.257..3003.268 rows=1 loops=1)

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.002..0.007 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.002 rows=1 loops=1)

         ->  Subquery Scan on b  (cost=0.00..0.33 rows=1 width=4) (actual time=500.541..500.542 rows=0 loops=6)

               Filter: (b.sq_sum = 6)

               Rows Removed by Filter: 0

               ->  Function Scan on generate_series id  (cost=0.00..0.32 rows=1 width=8) (actual time=500.539..500.540 rows=0 loops=6)

                     Filter: (id = (generate_series(3, 8)))

                     Rows Removed by Filter: 4

   ->  Materialize  (cost=0.00..0.08 rows=3 width=4) (actual time=0.007..0.009 rows=3 loops=1)

         ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.005..0.006 rows=3 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.001..0.002 rows=1 loops=1)

 Planning Time: 0.125 ms

 Execution Time: 3003.306 ms

(15 行记录)

时间：3003.950 ms (00:03.004)

总结

  • Select 子句中，表达式会逐行运算，总时长与结果集成正比。
  • CTE子句，先计算全部结果，然后关联主表，总时长是可控。
  • 使用子查询和 LATERAL 连接，可以避免结果集的重复计算。

SQL优化篇之-如何减少耗时查询的调用次数的更多相关文章

  1. 【MySQL】SQL优化系列之 in与range 查询

    首先我们来说下in()这种方式的查询 在<高性能MySQL>里面提及用in这种方式可以有效的替代一定的range查询,提升查询效率,因为在一条索引里面,range字段后面的部分是不生效的. ...

  2. 聊聊数据库~4.SQL优化篇

    1.5.查询的艺术 上期回顾:https://www.cnblogs.com/dotnetcrazy/p/10399838.html 本节脚本:https://github.com/lotapp/Ba ...

  3. MyBatis动态SQL第一篇之实现多条件查询（if、where、trim标签）

    一.动态SQL概述 以前在使用JDBC操作数据时,如果查询条件特别多,将条件串联成SQL字符串是一件痛苦的事情.通常的解决方法是写很多的if-else条件语句对字符串进行拼接,并确保不能忘了空格或在字 ...

  4. MySQL优化篇（一），我可以和面试官多聊几句吗？——SQL优化流程与优化数据库对象

    我可以和面试官多聊几句吗?只是想偷点技能过来.MySQL优化篇(基于MySQL8.0测试验证),上部分:优化SQL语句.数据库对象,MyISAM表锁和InnoDB锁问题. MyISAM表锁和InnoD ...

  5. sql优化个人总结（全）

    sql优化总结--博客 第一次自己写博客,以后要坚持每掌握一个技能点,就要写一篇博客出来,做一个不满足于一个只会写if...else的程序员. 最近三个月入职了一家新的公司,做的是CRM系统,将公司多 ...

  6. MySQL中的sql优化

    目标: 掌握SQL调优的原则 掌握SQL调优的基本逻辑 掌握优秀SQL的编写方案 掌握何为慢SQL以及检测方案 SQL优化原则 1.减少数据量(表中数据太多可以分表,例如超过500万数据  双11一个 ...

  7. 常见SQL优化方法

    SQL优化的一些方法 1.对查询进行优化,应尽量避免全表扫描,首先应考虑在 where 及 order by 涉及的列上建立索引. 2.应尽量避免在 where 子句中对字段进行 null 值判断,否 ...

  8. sql优化的几种方式

    一.为什么要对SQL进行优化 我们开发项目上线初期,由于业务数据量相对较少,一些SQL的执行效率对程序运行效率的影响不太明显,而开发和运维人员也无法判断SQL对程序的运行效率有多大,故很少针对SQL进 ...

  9. SQL优化原理

    SQL优化过程: 1,捕获高负荷的SQL语句-->2得到SQL语句的执行计划和统计信息--->3分析SQL语句的执行计划和统计信息--->4采取措施,对SQL语句进行调整.1找出高负 ...

  10. SQL Server数据库性能优化之SQL语句篇【转】

    SQL Server数据库性能优化之SQL语句篇http://www.blogjava.net/allen-zhe/archive/2010/07/23/326927.html 近期项目需要, 做了一 ...

随机推荐

  1. Rtmp 开发学习

    参考文章:视频传输协议详解(RTMP.RTSP.HLS) RTMP--Real Time Messaging Protocol(实时消息传输协议) RTMP 是由 Adobe 公司提出的,在互联网 T ...

  2. win32 - IFolderView2::GetCurrentFolderFlags的使用

    网上关于IFolderView2接口调用的示例有点少. 下面的例子是将桌面的图标隐藏起来,使用了FWF_NOICONS样式. #include <ShlObj.h> // Shell AP ...

  3. zookeeper源码(09)follower处理客户端请求

    在zookeeper中,follower也可以接收客户端连接,处理客户端请求,本文将分析follower处理客户端请求的流程: 读请求处理 写请求转发与响应 follower接收转发客户端请求 网络层 ...

  4. 【LeetCode栈与队列#03】删除字符串中所有的相邻重复项

    删除字符串中所有的相邻重复项 力扣题目链接(opens new window) 给出由小写字母组成的字符串 S,重复项删除操作会选择两个相邻且相同的字母,并删除它们. 在 S 上反复执行重复项删除操作 ...

  5. Windows 实例如何开放端口

    矩池云 Windows 实例相比于 Linux 实例,除了在租用机器的时候自定义端口外,还需要在 Windows防火墙中添加入口规则.接下来将教大家如何设置 Windows 防火墙,启用端口. 租用成 ...

  6. 可以取代宝塔和Nginx的Web服务器：Caddy

    一.安装 官网文章:https://caddyserver.com/docs/install 在左边选择:Install 我的服务器是Ubuntu,所以选第二行 我的服务器是Ubuntu,官方给出的就 ...

  7. HttpClient实现https调用

    在HttpClient 4.x版本中引入了大量的构造器设计模式 https请求建立详解 首先建立一个信任任何密钥的策略.代码很简单,不去考虑证书链和授权类型,均认为是受信任的: class AnyTr ...

  8. 【Azure K8S | AKS】在AKS中创建 StatefulSet 示例

    问题描述 [Azure K8S | AKS]在AKS集群中创建 PVC(PersistentVolumeClaim)和 PV(PersistentVolume) 示例 [Azure K8S|AKS]进 ...

  9. 【Azure API 管理】APIM的容量指标（Capacity）数据异常高的情况记录

    问题描述 APIM从标准版降级到基础版,在没有用户使用的情况,Capacity的指标平均显示在80%以上. 这是什么异常情况呢? 问题分析 APIM的容量指标(Capacity)是 API 管理实例中 ...

  10. 结构化思维助力Prompt创作：专业化技术讲解和实践案例

    结构化思维助力Prompt创作:专业化技术讲解和实践案例 最早接触 Prompt engineering 时, 学到的 Prompt 技巧都是: 你是一个 XX 角色- 你是一个有着 X 年经验的 X ...