函数调用次数与性能

在查询语句中，如果 Select 子句调用了较为耗时的函数或子查询，需要特别考虑函数调用次数对于SQL整体执行时间的影响。

一、数据准备，SQL 语句

  • 模拟较耗时的用户函数

确保执行子查询的时长是1秒。

create or replace function f001()

returns int stable language sql

as

$$

  select 1 from pg_sleep(1);

$$;
  • 模拟返回多行数据的子查询

结果集中，关联条件列含有重复值。

test=# create table t1 as select sn, id from (select generate_series(1, 3) sn),    (select generate_series(3, 8) id);

SELECT 18

test=# create table t2 as select generate_series(1, 5) id;

SELECT 5

test=# select * from t1;

 sn | id

----+----

  1 |  3

  2 |  3

  3 |  3

  1 |  4

  2 |  4

  3 |  4

  1 |  5

  2 |  5

  3 |  5

  1 |  6

  2 |  6

  3 |  6

  1 |  7

  2 |  7

  3 |  7

  1 |  8

  2 |  8

  3 |  8

(18 rows)

test=# select * from t2;

 id

----

  1

  2

  3

  4

  5

(5 rows)　　

二、查询全体数据的优化方案

1、初始SQL

with a as (select sn, id  from t1 )

select a.*, (select  id + f001()  sq_sum from t2 b where b.id = a.id)

from a

where 1 = 1;

 sn | id | sq_sum

----+----+--------

  1 |  3 |      4

  2 |  3 |      4

  3 |  3 |      4

  1 |  4 |      5

  2 |  4 |      5

  3 |  4 |      5

  1 |  5 |      6

  2 |  5 |      6

  3 |  5 |      6

  1 |  6 |

  2 |  6 |

  3 |  6 |

  1 |  7 |

  2 |  7 |

  3 |  7 |

  1 |  8 |

  2 |  8 |

  3 |  8 |

                                                 QUERY PLAN

-------------------------------------------------------------------------------------------------------------

 Seq Scan on t1  (cost=0.00..101806.05 rows=2260 width=12) (actual time=1000.638..9008.584 rows=18 loops=1)

   SubPlan 1

     ->  Seq Scan on t2 b  (cost=0.00..45.03 rows=13 width=4) (actual time=500.465..500.468 rows=0 loops=18)

           Filter: (id = t1.id)

           Rows Removed by Filter: 4

 Planning Time: 0.149 ms

 Execution Time: 9008.667 ms

(7 rows)

对于t1 表的每条记录，都要访问一次t2 ，如果 t2 表有对应的满足条件的记录，就要调用一次函数。

2、CTE

使用临时表的结果进行连接，避免循环。注意，这里CTE 有materilaized 选项，主要是把耗时的部分先执行出结果，避免与其他部分查询合并，引发多次执行。

with a as (select sn, id  from t1),

     b as materialized (select id, id + f001() sq_sum from t2 )

select a.*, (select sq_sum from b where b.id = a.id)

from a

where 1 = 1;

                                                  QUERY PLAN

---------------------------------------------------------------------------------------------------------------

 Seq Scan on t1  (cost=676.75..129868.35 rows=2260 width=12) (actual time=5004.377..5004.392 rows=18 loops=1)

   CTE b

     ->  Seq Scan on t2  (cost=0.00..676.75 rows=2540 width=8) (actual time=1000.562..5004.348 rows=5 loops=1)

   SubPlan 2

     ->  CTE Scan on b  (cost=0.00..57.15 rows=13 width=4) (actual time=166.821..278.021 rows=0 loops=18)

           Filter: (id = t1.id)

           Rows Removed by Filter: 4

 Planning Time: 0.107 ms

 Execution Time: 5004.441 ms

(9 rows)

3、LEFT JOIN 子查询

与上例一样，还是利用materialize 特性，避免了函数的多次调用。

with a as (select sn, id from t1)

select a.*, b.sq_sum

from a  left join (select id, id + f001() sq_sum  from t2 ) b on b.id = a.id

where 1 = 1 ;

                                                    QUERY PLAN

-------------------------------------------------------------------------------------------------------------------

 Nested Loop Left Join  (cost=0.00..86821.70 rows=28702 width=12) (actual time=3002.905..5005.627 rows=18 loops=1)

   Join Filter: (t2.id = t1.id)

   Rows Removed by Join Filter: 81

   ->  Seq Scan on t1  (cost=0.00..32.60 rows=2260 width=8) (actual time=0.007..0.011 rows=18 loops=1)

   ->  Materialize  (cost=0.00..689.45 rows=2540 width=8) (actual time=55.621..278.088 rows=5 loops=18)

         ->  Seq Scan on t2  (cost=0.00..676.75 rows=2540 width=8) (actual time=1001.176..5005.568 rows=5 loops=1)

 Planning Time: 0.180 ms

 Execution Time: 5005.650 ms

(8 rows)

三、查询局部数据（过滤主表的条件）的优化方案

1、初始SQL

with a as (select sn, id from t1)

select a.*, (select  id + f001() sq_sum from  t2  b where b.id = a.id)

from a

where 1 = 1 and a.id = 3;

 sn | id | sq_sum

----+----+--------

  1 |  3 |      4

  2 |  3 |      4

  3 |  3 |      4

(3 rows)

                                                  QUERY PLAN

--------------------------------------------------------------------------------------------------------------

 Seq Scan on t1  (cost=0.00..533.61 rows=11 width=12) (actual time=1000.356..3002.635 rows=3 loops=1)

   Filter: (id = 3)

   Rows Removed by Filter: 15

   SubPlan 1

     ->  Seq Scan on t2 b  (cost=0.00..45.03 rows=13 width=4) (actual time=1000.859..1000.866 rows=1 loops=3)

           Filter: (id = t1.id)

           Rows Removed by Filter: 4

 Planning Time: 0.164 ms

 Execution Time: 3002.657 ms

(9 rows)

2、CTE

with a as (select sn, id from t1),

     b as materialized (select id, id + f001() sq_sum from t2)

select a.*, (select sq_sum from b where b.id = a.id) sq_sum

from a

where 1 = 1 and a.id=3 ;

                                                  QUERY PLAN

---------------------------------------------------------------------------------------------------------------

 Seq Scan on t1  (cost=676.75..1343.65 rows=11 width=12) (actual time=5004.958..5004.965 rows=3 loops=1)

   Filter: (id = 3)

   Rows Removed by Filter: 15

   CTE b

     ->  Seq Scan on t2  (cost=0.00..676.75 rows=2540 width=8) (actual time=1001.387..5004.930 rows=5 loops=1)

   SubPlan 2

     ->  CTE Scan on b  (cost=0.00..57.15 rows=13 width=4) (actual time=1001.132..1668.316 rows=1 loops=3)

           Filter: (id = t1.id)

           Rows Removed by Filter: 4

 Planning Time: 0.105 ms

 Execution Time: 5004.983 ms

(11 rows)

3、LEFT JOIN 子查询

explain analyse

with a as (select sn, id from t1)

select a.*, b.sq_sum

from a left  join (select id, id + f001() sq_sum from t2) b on b.id = a.id

where 1 = 1 and a.id = 3 ;

                                                   QUERY PLAN

----------------------------------------------------------------------------------------------------------------

 Nested Loop Left Join  (cost=0.00..85.46 rows=143 width=12) (actual time=1000.972..1000.980 rows=3 loops=1)

   Join Filter: (t2.id = t1.id)

   ->  Seq Scan on t1  (cost=0.00..38.25 rows=11 width=8) (actual time=0.008..0.009 rows=3 loops=1)

         Filter: (id = 3)

         Rows Removed by Filter: 15

   ->  Materialize  (cost=0.00..45.10 rows=13 width=8) (actual time=333.654..333.656 rows=1 loops=3)

         ->  Seq Scan on t2  (cost=0.00..45.03 rows=13 width=8) (actual time=1000.959..1000.964 rows=1 loops=1)

               Filter: (id = 3)

               Rows Removed by Filter: 4

 Planning Time: 0.107 ms

 Execution Time: 1000.996 ms

(11 rows)

四、查询局部数据（过滤从表的条件）的优化方案

1、初始SQL

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id))

select a.*, (select  id + f001()  sq_sum from  (select  generate_series(1, 5) id) b where b.id = a.id) sq_sum

from a

where 1 = 1 and sq_sum = 6 ;


 sn | id | sq_sum

----+----+--------

  1 |  5 |      6

  2 |  5 |      6

  3 |  5 |      6

(3 行记录)

时间：6007.526 ms (00:06.008)

                                                         QUERY PLAN

----------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=0.00..3.43 rows=3 width=12) (actual time=4004.455..6006.681 rows=3 loops=1)

   ->  Subquery Scan on "SYSINTERNAL-4-1"  (cost=0.00..2.27 rows=1 width=4) (actual time=3003.273..3003.289 rows=1 loops=1)

         Filter: ((SubPlan 2) = 6)

         Rows Removed by Filter: 5

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.001..0.005 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

         SubPlan 2

           ->  Subquery Scan on b_1  (cost=0.00..0.36 rows=1 width=4) (actual time=500.541..500.544 rows=0 loops=6)

                 Filter: (b_1.id = "SYSINTERNAL-4-1".id)

                 Rows Removed by Filter: 4

                 ->  ProjectSet  (cost=0.00..0.04 rows=5 width=4) (actual time=0.001..0.003 rows=5 loops=6)

                       ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=6)

   ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.002..0.006 rows=3 loops=1)

         ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

   SubPlan 1

     ->  Subquery Scan on b  (cost=0.00..0.36 rows=1 width=4) (actual time=1001.116..1001.122 rows=1 loops=3)

           Filter: (b.id = "SYSINTERNAL-4-1".id)

           Rows Removed by Filter: 4

           ->  ProjectSet  (cost=0.00..0.04 rows=5 width=4) (actual time=0.001..0.006 rows=5 loops=3)

                 ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=3)

 Planning Time: 0.121 ms

 Execution Time: 6006.709 ms

(22 行记录)

时间：6007.303 ms (00:06.007)

2、CTE

explain analyse

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id)),

     b as (select id, id + f001() sq_sum from generate_series(1, 5) id)

select a.*, (select sq_sum from b where b.id = a.id) sq_sum

from a

where 1 = 1  and sq_sum = 6 ;

                                                          QUERY PLAN

-------------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=1.31..2.54 rows=3 width=12) (actual time=5005.414..5005.422 rows=3 loops=1)

   CTE b

     ->  Function Scan on generate_series id  (cost=0.00..1.31 rows=5 width=8) (actual time=1001.093..5005.376 rows=5 loops=1)

   ->  Subquery Scan on "SYSINTERNAL-4-1"  (cost=0.00..0.80 rows=1 width=4) (actual time=5005.406..5005.410 rows=1 loops=1)

         Filter: ((SubPlan 3) = 6)

         Rows Removed by Filter: 5

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.002..0.005 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

         SubPlan 3

           ->  CTE Scan on b b_1  (cost=0.00..0.11 rows=1 width=4) (actual time=500.540..834.232 rows=0 loops=6)

                 Filter: (id = "SYSINTERNAL-4-1".id)

                 Rows Removed by Filter: 4

   ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.002..0.003 rows=3 loops=1)

         ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.000 rows=1 loops=1)

   SubPlan 2

     ->  CTE Scan on b  (cost=0.00..0.11 rows=1 width=4) (actual time=0.001..0.001 rows=1 loops=3)

           Filter: (id = "SYSINTERNAL-4-1".id)

           Rows Removed by Filter: 4

 Planning Time: 0.143 ms

 Execution Time: 5005.449 ms

(20 行记录)

时间：5006.115 ms (00:05.006)

3、LEFT JOIN

explain analyse

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id))

select a.*, b.sq_sum

from a  left  join (select id, id + f001() sq_sum  from generate_series(1, 5) id) b on b.id = a.id

where 1 = 1 and sq_sum = 6 ;

                                                                  QUERY PLAN

-----------------------------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=1.39..1.62 rows=3 width=12) (actual time=5005.285..5005.291 rows=3 loops=1)

   ->  Hash Join  (cost=1.39..1.53 rows=1 width=8) (actual time=5005.281..5005.285 rows=1 loops=1)

         Hash Cond: ((generate_series(3, 8)) = b.id)

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.002..0.005 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.001..0.001 rows=1 loops=1)

         ->  Hash  (cost=1.38..1.38 rows=1 width=8) (actual time=5005.270..5005.270 rows=1 loops=1)

               Buckets: 1024  Batches: 1  Memory Usage: 9kB

               ->  Subquery Scan on b  (cost=0.00..1.38 rows=1 width=8) (actual time=5005.264..5005.266 rows=1 loops=1)

                     Filter: (b.sq_sum = 6)

                     Rows Removed by Filter: 4

                     ->  Function Scan on generate_series id  (cost=0.00..1.31 rows=5 width=8) (actual time=1001.089..5005.254 rows=5 loops=1)

   ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.002..0.003 rows=3 loops=1)

         ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

 Planning Time: 0.113 ms

 Execution Time: 5005.320 ms

(15 行记录)

时间：5005.873 ms (00:05.006)

4、LATERAL 连接 CTE

explain analyse

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id)),

     b as (select id, id + f001() sq_sum from generate_series(1, 5) id)

select a.*, b.sq_sum

from a join lateral (select * from b where b.id = a.id ) b on true

where 1 = 1  and sq_sum = 6;

                                                          QUERY PLAN

-------------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=1.44..1.67 rows=3 width=12) (actual time=5005.492..5005.499 rows=3 loops=1)

   CTE b

     ->  Function Scan on generate_series id  (cost=0.00..1.31 rows=5 width=8) (actual time=1001.103..5005.440 rows=5 loops=1)

   ->  Hash Join  (cost=0.12..0.27 rows=1 width=8) (actual time=5005.485..5005.489 rows=1 loops=1)

         Hash Cond: ((generate_series(3, 8)) = b.id)

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.002..0.006 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.001 rows=1 loops=1)

         ->  Hash  (cost=0.11..0.11 rows=1 width=8) (actual time=5005.472..5005.472 rows=1 loops=1)

               Buckets: 1024  Batches: 1  Memory Usage: 9kB

               ->  CTE Scan on b  (cost=0.00..0.11 rows=1 width=8) (actual time=5005.457..5005.459 rows=1 loops=1)

                     Filter: (sq_sum = 6)

                     Rows Removed by Filter: 4

   ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.004..0.005 rows=3 loops=1)

         ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.001..0.001 rows=1 loops=1)

 Planning Time: 0.205 ms

 Execution Time: 5005.531 ms

(16 行记录)

时间：5006.765 ms (00:05.007)

5、LATERAL 连接子查询

explain analyse

with a as (select sn, id from (select generate_series(1, 3) sn), (select generate_series(3, 8) id))

select a.*, b.sq_sum

from a left join lateral (select id, id + f001() sq_sum  from generate_series(1, 5) id where id = a.id ) b on true

where 1 = 1 and sq_sum = 6 ;

                                                              QUERY PLAN

---------------------------------------------------------------------------------------------------------------------------------------

 Nested Loop  (cost=0.00..2.44 rows=18 width=12) (actual time=3003.267..3003.280 rows=3 loops=1)

   ->  Nested Loop  (cost=0.00..2.15 rows=6 width=8) (actual time=3003.257..3003.268 rows=1 loops=1)

         ->  ProjectSet  (cost=0.00..0.05 rows=6 width=4) (actual time=0.002..0.007 rows=6 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.000..0.002 rows=1 loops=1)

         ->  Subquery Scan on b  (cost=0.00..0.33 rows=1 width=4) (actual time=500.541..500.542 rows=0 loops=6)

               Filter: (b.sq_sum = 6)

               Rows Removed by Filter: 0

               ->  Function Scan on generate_series id  (cost=0.00..0.32 rows=1 width=8) (actual time=500.539..500.540 rows=0 loops=6)

                     Filter: (id = (generate_series(3, 8)))

                     Rows Removed by Filter: 4

   ->  Materialize  (cost=0.00..0.08 rows=3 width=4) (actual time=0.007..0.009 rows=3 loops=1)

         ->  ProjectSet  (cost=0.00..0.03 rows=3 width=4) (actual time=0.005..0.006 rows=3 loops=1)

               ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.001..0.002 rows=1 loops=1)

 Planning Time: 0.125 ms

 Execution Time: 3003.306 ms

(15 行记录)

时间：3003.950 ms (00:03.004)

总结

  • Select 子句中，表达式会逐行运算，总时长与结果集成正比。
  • CTE子句，先计算全部结果，然后关联主表，总时长是可控。
  • 使用子查询和 LATERAL 连接，可以避免结果集的重复计算。

SQL优化篇之-如何减少耗时查询的调用次数的更多相关文章

  1. 【MySQL】SQL优化系列之 in与range 查询

    首先我们来说下in()这种方式的查询 在<高性能MySQL>里面提及用in这种方式可以有效的替代一定的range查询,提升查询效率,因为在一条索引里面,range字段后面的部分是不生效的. ...

  2. 聊聊数据库~4.SQL优化篇

    1.5.查询的艺术 上期回顾:https://www.cnblogs.com/dotnetcrazy/p/10399838.html 本节脚本:https://github.com/lotapp/Ba ...

  3. MyBatis动态SQL第一篇之实现多条件查询（if、where、trim标签）

    一.动态SQL概述 以前在使用JDBC操作数据时,如果查询条件特别多,将条件串联成SQL字符串是一件痛苦的事情.通常的解决方法是写很多的if-else条件语句对字符串进行拼接,并确保不能忘了空格或在字 ...

  4. MySQL优化篇（一），我可以和面试官多聊几句吗？——SQL优化流程与优化数据库对象

    我可以和面试官多聊几句吗?只是想偷点技能过来.MySQL优化篇(基于MySQL8.0测试验证),上部分:优化SQL语句.数据库对象,MyISAM表锁和InnoDB锁问题. MyISAM表锁和InnoD ...

  5. sql优化个人总结（全）

    sql优化总结--博客 第一次自己写博客,以后要坚持每掌握一个技能点,就要写一篇博客出来,做一个不满足于一个只会写if...else的程序员. 最近三个月入职了一家新的公司,做的是CRM系统,将公司多 ...

  6. MySQL中的sql优化

    目标: 掌握SQL调优的原则 掌握SQL调优的基本逻辑 掌握优秀SQL的编写方案 掌握何为慢SQL以及检测方案 SQL优化原则 1.减少数据量(表中数据太多可以分表,例如超过500万数据  双11一个 ...

  7. 常见SQL优化方法

    SQL优化的一些方法 1.对查询进行优化,应尽量避免全表扫描,首先应考虑在 where 及 order by 涉及的列上建立索引. 2.应尽量避免在 where 子句中对字段进行 null 值判断,否 ...

  8. sql优化的几种方式

    一.为什么要对SQL进行优化 我们开发项目上线初期,由于业务数据量相对较少,一些SQL的执行效率对程序运行效率的影响不太明显,而开发和运维人员也无法判断SQL对程序的运行效率有多大,故很少针对SQL进 ...

  9. SQL优化原理

    SQL优化过程: 1,捕获高负荷的SQL语句-->2得到SQL语句的执行计划和统计信息--->3分析SQL语句的执行计划和统计信息--->4采取措施,对SQL语句进行调整.1找出高负 ...

  10. SQL Server数据库性能优化之SQL语句篇【转】

    SQL Server数据库性能优化之SQL语句篇http://www.blogjava.net/allen-zhe/archive/2010/07/23/326927.html 近期项目需要, 做了一 ...

随机推荐

  1. Stream 总结

    1 前言 Stream 是 Java 8 中为方便操作集合及其元素而定制的接口,它将要处理的元素集合看作一种流,对流中的元素进行过滤.排序.映射.聚合等操作.使用 Stream API,就好像使用 S ...

  2. look命令

    look命令 look命令用于查询单词,仅需指定欲查询的字首字符串,它会显示所有开头字符串符合该条件的单词. 语法 look [-bdf] [-t char] string [file ...] 参数 ...

  3. Java操作EasyExcel实现导入导出入门

    介绍 EasyExcel是阿里巴巴开源的一个excel处理框架,以使用简单.节省内存著称.EasyExcel能大大减少占用内存的主要原因是在解析Excel时没有将文件数据一次性全部加载到内存中,而是从 ...

  4. RedHat5 安装中文输入法

    为了学习linux命令,虚拟机里搞了个RedHat 5.4 .下面是安装中文输入法过程: 1.下载并安装小企鹅输入法 fcitx-3.0.0-1.i386.rpm 2.安装后执行以下命令: # cd ...

  5. Java序列化（Serializable）与反序列化详解

    什么是序列化? Java序列化是在JDK 1.1中引入的,是Java内核的重要特性之一. Java序列化API允许我们将一个对象转换为流,并通过网络发送,或将其存入文件或数据库以便未来使用, 反序列化 ...

  6. 使用webgl(three.js)创建自动化抽象化3D机房，3D机房模块详细介绍(抽象版一)

    目前市面上有两种机房 一种是普通机房 一种是由微模块组成的机房,本文主要介绍普通机房的抽象化体现模式. 抽象机房模式:机房展示过程中,我们需要对机房进行建模,当遇到大量机房需要建模时,这无疑是巨大工作 ...

  7. Linux驱动开发笔记（一）：helloworld驱动源码编写、makefile编写以及驱动编译基本流程

    前言   基于linux的驱动开发学习笔记,本篇是描述了一个字符驱动的基础开发流程,以便做嵌入式开发多年的应用或者系统学习驱动开发.   笔者自身情况   笔者拥有硬件基础,单片机软硬基础,linux ...

  8. mysql进阶优化2---day41

    # ### part1 索引树高度 # 1.表的数据量 数据量越大,树的高度就会变高,理论上三层索引树的高度最为理想,可以支持百万级别的数据量 解决:可以使用分表(横切,竖切),分库,增加缓存,解决数 ...

  9. Flask遇到的坑及解决办法

    flask_script 在使用第三方包flask_script时,报一下错误 ModuleNotFoundError: No module named 'flask._compat' 问题时flas ...

  10. kotlin协程小记

    转载请标明出处:https://www.cnblogs.com/tangZH/p/16849169.html -[kotlin协程小记]-[协程的async使用]- [kotlin协程异常处理之-tr ...