exist和left join 性能对比
今天遇到一个性能问题，再调优过程中发现耗时最久的计划是exist 部分涉及的三个表。
然后计划用left join 来替换exist，然后查询了很多资料，大部分都说exist和left join 性能差不多。 为了验证这一结论进行了如下实验
步骤如下
1、创建测试表
drop table app_family;
CREATE TABLE app_family (
"family_id" character varying(32 char) NOT NULL,
"application_id" character varying(32 char) NULL,
"family_number" character varying(50 char) ,
"household_register_number" character varying(50 char),
"poverty_reason" character varying(32 char),
CONSTRAINT "pk_app_family_idpk" PRIMARY KEY (family_id));
insert into app_family select generate_series(1,1000000),generate_series(1,1000000),'aaaa','aaa','bbb' from dual ;
create table app_family2 as select * from app_family;
create table app_memeber as select * from app_family;
2、验证两张表join和exist 性能对比
语句1、两张表exist
explain analyze select a1.application_id,a1.family_id from app_family a1 where
a1.family_id >1000 and
EXISTS(
SELECT
1
FROM
app_family2 a2
WHERE
a2.application_id=a1.application_id
and a2.family_id > 500000
)
总计用时646.203 ms
----------------------------------------------------------------------------------------------------------------------------------------------------
Gather (cost=16927.11..44466.84 rows=111111 width=12) (actual time=354.314..621.714 rows=500000 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Parallel Hash Semi Join (cost=15927.11..32355.74 rows=46296 width=12) (actual time=355.657..512.049 rows=166667 loops=3)
Hash Cond: ((a1.application_id)::text = (a2.application_id)::text)
-> Parallel Seq Scan on app_family a1 (cost=0.00..13648.00 rows=138889 width=12) (actual time=0.222..111.618 rows=333000 loops=3)
Filter: ((family_id)::integer > 1000)
Rows Removed by Filter: 333
-> Parallel Hash (cost=13648.00..13648.00 rows=138889 width=6) (actual time=149.203..149.204 rows=166667 loops=3)
Buckets: 131072 Batches: 8 Memory Usage: 3520kB
-> Parallel Seq Scan on app_family2 a2 (cost=0.00..13648.00 rows=138889 width=6) (actual time=48.576..109.251 rows=166667 loops=3)
Filter: ((family_id)::integer > 500000)
Rows Removed by Filter: 166667
Planning Time: 0.145 ms
Execution Time: 645.095 ms
(15 rows)
Time: 646.203 ms
kingbase=#
语句2 两张表join
explain analyze select a1.application_id,a1.family_id from app_family a1 LEFT JOIN app_family2 a2 ON a2.application_id=a1.application_id
WHERE a1.family_id >1000 AND a2.family_id > 500000
总计执行时间624.211 ms
---------------------------------------------------------------------------------------------------------------------------------------------------
Gather (cost=16927.11..44300.95 rows=111111 width=12) (actual time=349.752..601.304 rows=500000 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Parallel Hash Join (cost=15927.11..32189.85 rows=46296 width=12) (actual time=337.548..508.139 rows=166667 loops=3)
Hash Cond: ((a1.application_id)::text = (a2.application_id)::text)
-> Parallel Seq Scan on app_family a1 (cost=0.00..13648.00 rows=138889 width=12) (actual time=0.087..111.949 rows=333000 loops=3)
Filter: ((family_id)::integer > 1000)
Rows Removed by Filter: 333
-> Parallel Hash (cost=13648.00..13648.00 rows=138889 width=6) (actual time=131.718..131.719 rows=166667 loops=3)
Buckets: 131072 Batches: 8 Memory Usage: 3488kB
-> Parallel Seq Scan on app_family2 a2 (cost=0.00..13648.00 rows=138889 width=6) (actual time=31.730..90.917 rows=166667 loops=3)
Filter: ((family_id)::integer > 500000)
Rows Removed by Filter: 166667
Planning Time: 0.093 ms
Execution Time: 623.465 ms
(15 rows)
Time: 624.211 ms
两张表场景总结
针对两张表的对比可以发现join还相对满了10几ms但是总的来说两边 差异不大。所以再两张表的关联情况下 join和exist 性能相近。
3、验证3张表join和exist 性能对比
语句1 三张表exist
本场景最开始执行时 exit 用户6 s多，原因时用到了内存排序，后来调整了work_mem 排除了内存排序的影响，最终执行时间
2911.146 ms
explain analyze select a1.application_id,a1.family_id from app_family a1 ,app_family2 a2 where
a1.family_id >1000 and a2.family_id < 900000 and
EXISTS(
SELECT
1
FROM
app_memeber m
WHERE
m.application_id=a1.application_id
and m.family_id=a2.family_id
)
------------------------------------------------------------------------------------------------------------------------------------------------------
--
Gather (cost=61282.11..88664.67 rows=111111 width=12) (actual time=2112.079..2847.233 rows=898999 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Parallel Hash Join (cost=60282.11..76553.57 rows=46296 width=12) (actual time=2119.345..2705.935 rows=299666 loops=3)
Hash Cond: ((m.family_id)::text = (a2.family_id)::text)
-> Hash Join (cost=44898.00..60455.72 rows=138889 width=18) (actual time=1885.923..2264.850 rows=333000 loops=3)
Hash Cond: ((a1.application_id)::text = (m.application_id)::text)
-> Parallel Seq Scan on app_family a1 (cost=0.00..13648.00 rows=138889 width=12) (actual time=0.091..109.196 rows=333000 loops=3)
Filter: ((family_id)::integer > 1000)
Rows Removed by Filter: 333
-> Hash (cost=32398.00..32398.00 rows=1000000 width=12) (actual time=1880.027..1880.028 rows=1000000 loops=3)
Buckets: 1048576 Batches: 1 Memory Usage: 52897kB
-> HashAggregate (cost=22398.00..32398.00 rows=1000000 width=12) (actual time=957.973..1382.683 rows=1000000 loops=3)
Group Key: (m.application_id)::text, (m.family_id)::text
-> Seq Scan on app_memeber m (cost=0.00..17398.00 rows=1000000 width=12) (actual time=0.047..247.902 rows=1000000 loops=3
)
-> Parallel Hash (cost=13648.00..13648.00 rows=138889 width=6) (actual time=231.705..231.706 rows=300000 loops=3)
Buckets: 1048576 (originally 524288) Batches: 1 (originally 1) Memory Usage: 47552kB
-> Parallel Seq Scan on app_family2 a2 (cost=0.00..13648.00 rows=138889 width=6) (actual time=0.039..100.756 rows=300000 loops=3)
Filter: ((family_id)::integer < 900000)
Rows Removed by Filter: 33334
Planning Time: 0.359 ms
Execution Time: 2911.146 ms
(22 rows)
语句2 三张表join
为了保证语句的一致性，三张表的join顺序保持和语句1的执行计划中的顺序一致，join总计用时1476.651 ms
explain analyze select a1.application_id,a1.family_id from app_family a1
left join app_memeber m on a1.application_id = m.application_id LEFT JOIN app_family2 a2 ON m.family_id = a2.family_id
WHERE a1.family_id >1000 AND a2.family_id < 900000
Gather (cost=32990.22..64898.93 rows=111111 width=12) (actual time=993.681..1436.895 rows=898999 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Parallel Hash Join (cost=31990.22..52787.83 rows=46296 width=12) (actual time=982.512..1241.385 rows=299666 loops=3)
Hash Cond: ((m.application_id)::text = (a1.application_id)::text)
-> Parallel Hash Join (cost=15927.11..34245.98 rows=138889 width=6) (actual time=377.411..635.945 rows=300000 loops=3)
Hash Cond: ((m.family_id)::text = (a2.family_id)::text)
-> Parallel Seq Scan on app_memeber m (cost=0.00..11564.67 rows=416667 width=12) (actual time=0.034..59.470 rows=333333 loops=3)
-> Parallel Hash (cost=13648.00..13648.00 rows=138889 width=6) (actual time=232.286..232.287 rows=300000 loops=3)
Buckets: 131072 (originally 131072) Batches: 16 (originally 8) Memory Usage: 3296kB
-> Parallel Seq Scan on app_family2 a2 (cost=0.00..13648.00 rows=138889 width=6) (actual time=0.030..104.370 rows=300000 loops=
3)
Filter: ((family_id)::integer < 900000)
Rows Removed by Filter: 33334
-> Parallel Hash (cost=13648.00..13648.00 rows=138889 width=12) (actual time=271.185..271.185 rows=333000 loops=3)
Buckets: 131072 (originally 131072) Batches: 16 (originally 8) Memory Usage: 4032kB
-> Parallel Seq Scan on app_family a1 (cost=0.00..13648.00 rows=138889 width=12) (actual time=0.091..129.188 rows=333000 loops=3)
Filter: ((family_id)::integer > 1000)
Rows Removed by Filter: 333
Planning Time: 0.140 ms
Execution Time: 1475.305 ms
(20 rows)
Time: 1476.651 ms (00:01.477)
总结三张表场景
在三张表的场景下exist用时2911.146 ms ，join用时1476.651 ms 可见 join的顺序明显优于exist。
在三张表的场景下可以看到，针对中间表appmember扫描时， exist语句用到HashAggregate 并做了 Group Key，所以导致exist 执行时间增加。如果work_mem 配置不合适时间会更长。
exist和left join 性能对比的更多相关文章
- Go 字符串连接+=与strings.Join性能对比
Go字符串连接 对于字符串的连接大致有两种方式: 1.通过+号连接 func StrPlus1(a []string) string { var s, sep string for i := 0; i ...
- 自己写的轻量级PHP框架trig与laravel5.1,yii2性能对比
看了下当前最热门的php开发框架,想对比一下自己写的框架与这些框架的性能对比.先看下当前流行框架的投票情况. 看结果对比,每个测试脚本做了一个数据库的联表查询并进行print_r输出,查询的sql语句 ...
- SQL点滴10—使用with语句来写一个稍微复杂sql语句，附加和子查询的性能对比
原文:SQL点滴10-使用with语句来写一个稍微复杂sql语句,附加和子查询的性能对比 今天偶尔看到sql中也有with关键字,好歹也写了几年的sql语句,居然第一次接触,无知啊.看了一位博主的文章 ...
- python3下multiprocessing、threading和gevent性能对比----暨进程池、线程池和协程池性能对比
python3下multiprocessing.threading和gevent性能对比----暨进程池.线程池和协程池性能对比 标签: python3 / 线程池 / multiprocessi ...
- SQL Server-聚焦IN VS EXISTS VS JOIN性能分析（十九）
前言 本节我们开始讲讲这一系列性能比较的终极篇IN VS EXISTS VS JOIN的性能分析,前面系列有人一直在说场景不够,这里我们结合查询索引列.非索引列.查询小表.查询大表来综合分析,简短的内 ...
- [原] KVM 环境下MySQL性能对比
KVM 环境下MySQL性能对比 标签(空格分隔): Cloud2.0 [TOC] 测试目的 对比MySQL在物理机和KVM环境下性能情况 压测标准 压测遵循单一变量原则,所有的对比都是只改变一个变量 ...
- 浅谈C++之冒泡排序、希尔排序、快速排序、插入排序、堆排序、基数排序性能对比分析之后续补充说明（有图有真相）
如果你觉得我的有些话有点唐突,你不理解可以想看看前一篇<C++之冒泡排序.希尔排序.快速排序.插入排序.堆排序.基数排序性能对比分析>. 这几天闲着没事就写了一篇<C++之冒泡排序. ...
- Java--Stream，NIO ByteBuffer，NIO MappedByteBuffer性能对比
目前Java中最IO有多种文件读取的方法,本文章对比Stream,NIO ByteBuffer,NIO MappedByteBuffer的性能,让我们知道到底怎么能写出性能高的文件读取代码. pack ...
- C正则库做DNS域名验证时的性能对比
C正则库做DNS域名验证时的性能对比 本文对C的正则库regex和pcre在做域名验证的场景下做评测. 验证DNS域名的正则表达式为: "^[0-9a-zA-Z_-]+(\\.[0-9a ...
- 开发语言性能对比，C++、Java、Python、LUA、TCC
一直想做开发语言性能对比,刚好有时间都做了给大家参考一下, 编译类:C++和Java表现还不错 脚本类:TCC脚本动态运行C语言,性能比其他脚本快好多... 想玩TCC的同学下载测试包,TCC目录下修 ...
随机推荐
- 【Lua】xLua逻辑热更新
1 前言 Lua基础语法 中系统介绍了 Lua 的语法体系,ToLua逻辑热更新 中介绍了 ToLua 的应用,本文将进一步介绍 Unity3D 中基于 xLua 实现逻辑热更新. 逻辑热更新 ...
- Swift —— 一、架构解析
一.简介 OpenStack 对象存储 (swift) 用于冗余.可扩展的数据 使用标准化服务器集群存储PB的存储 可访问的数据.它是一种长期存储系统,可存储大量 可以检索和更新的静态数据.对象存储使 ...
- 使用GDI时如何确定是否有内存泄漏
在创建GDI对象时,比如创建笔,画刷等对象时,在调用完之后忘记删除对象了,会造成内存泄漏 我们可以通过任务管理器来快速的查看 启动任务管理器(右键单击Windows任务栏以选择任务管理器) 在Wind ...
- kafka学习笔记02-kafka消息存储
kafka消息存储 broker.topic.partition kafka 的数据分布是一个 3 级结构,依次为 broker.topic.partition. 也可以理解为数据库的分库分表,然后还 ...
- 【Android逆向】破解黑宝宝apk，绕过签名校验
这是52pojie的一道题,实现输入任何密码都可以登录成功 他知道你最近在学习Android逆向 他想在游戏上线前让你测试一下他新加的签名验证是否能防住别人的破解. 下面是李华编写的黑宝宝apk 链接 ...
- .NET Core 引发的异常:“sqlsugar.sqlsugarexception” 位于 system.private.corelib.dll 中
运行一个.NET Core 项目 报错:引发的异常:"sqlsugar.sqlsugarexception" 位于 system.private.corelib.dll 中 . 我 ...
- 阿尔萨斯(Arthas)入门
目录 简介 Arthas(阿尔萨斯) 能为你做什么 安装 快速安装 全量安装 卸载 使用 启动arthas 查看dashboard 通过thread命令来获取到arthas-demo进程的Main C ...
- 【Azure APIM】解决APIM Self-hosted Gateway在AKS上，最开始访问时候遇见的404问题
问题描述 根据APIM官方文档,创建Self-hosted 网关在AKS中( 使用 YAML 将自承载网关部署到 Kubernetes :https://docs.azure.cn/zh-cn/api ...
- 【Azure 媒体服务】Azure Media Service上传的视频资产，如何保证在Transfer编码后音频文件和视频文件不分成两个文件？保持在一个可以直接播放的MP4文件中呢？
问题描述 Azure Media Service上传的视频资产,如何保证在Transfer编码后音频文件和视频文件不分成两个文件?保持在一个可以直接播放的MP4文件中呢? 问题解答 Azure Med ...
- 【Azure Redis 缓存】Redis Geo-replication(异地复制)的问题
问题描述 在Azure官网中,已列出了一系列的常规问题:https://docs.azure.cn/zh-cn/azure-cache-for-redis/cache-how-to-geo-repli ...