转自: https://docs.citusdata.com/en/v7.5/articles/outer_joins.html

SQL is a very powerful language for analyzing and reporting against data. At the core of SQL is the idea of joins and how you combine various tables together. One such type of join: outer joins are useful when we need to retain rows, even if it has no match on the other side.

And while the most common type of join, inner join, against tables A and B would bring only the tuples that have a match for both A and B, outer joins give us the ability to bring together from say all of table A even if they don’t have a corresponding match in table B. For example, let’s say you keep customers in one table and purchases in another table. When you want to see all purchases of customers, you may want to see all customers in the result even if they did not do any purchases yet. Then, you need an outer join. Within this post we’ll analyze a bit on what outer joins are, and then how we support them in a distributed fashion on Citus.

Let’s say we have two tables, customer and purchase:

customer table:
customer_id | name
-------------+-----------------
1 | Corra Ignacio
3 | Warren Brooklyn
2 | Jalda Francis purchase table:
purchase_id | customer_id | category | comment
-------------+-------------+----------+------------------------------
1000 | 1 | books | Nice to Have!
1001 | 1 | chairs | Comfortable
1002 | 2 | books | Good Read, cheap price
1003 | -1 | hardware | Not very cheap
1004 | -1 | laptops | Good laptop but expensive...

The following queries and results help clarifying the inner and outer join behaviors:

SELECT customer.name, purchase.comment
FROM customer JOIN purchase ON customer.customer_id = purchase.customer_id
ORDER BY purchase.comment; name | comment
---------------+------------------------
Corra Ignacio | Comfortable
Jalda Francis | Good Read, cheap price
Corra Ignacio | Nice to Have!

SELECT customer.name, purchase.comment
FROM customer INNER JOIN purchase ON customer.customer_id = purchase.customer_id
ORDER BY purchase.comment; name | comment
---------------+------------------------
Corra Ignacio | Comfortable
Jalda Francis | Good Read, cheap price
Corra Ignacio | Nice to Have!

SELECT customer.name, purchase.comment
FROM customer LEFT JOIN purchase ON customer.customer_id = purchase.customer_id
ORDER BY purchase.comment; name | comment
-----------------+------------------------
Corra Ignacio | Comfortable
Jalda Francis | Good Read, cheap price
Corra Ignacio | Nice to Have!
Warren Brooklyn |

SELECT customer.name, purchase.comment
FROM customer RIGHT JOIN purchase ON customer.customer_id = purchase.customer_id
ORDER BY purchase.comment; name | comment
---------------+------------------------------
Corra Ignacio | Comfortable
Jalda Francis | Good Read, cheap price
| Good laptop but expensive...
Corra Ignacio | Nice to Have!
| Not very cheap

SELECT customer.name, purchase.comment
FROM customer FULL JOIN purchase ON customer.customer_id = purchase.customer_id
ORDER BY purchase.comment; name | comment
-----------------+------------------------------
Corra Ignacio | Comfortable
Jalda Francis | Good Read, cheap price
| Good laptop but expensive...
Corra Ignacio | Nice to Have!
| Not very cheap
Warren Brooklyn |

Distributed Outer Joins with Citus

The Citus extension allows PostgreSQL to distribute big tables into smaller fragments called “shards” and performing outer joins on these distributed tables becomes a bit more challenging, since the union of outer joins between individual shards does not always give the correct result. Currently, Citus support distributed outer joins under some criteria:

  • Outer joins should be between distributed(sharded) tables only, i.e. it is not possible to outer join a sharded table with a regular PostgreSQL table.
  • Join criteria should be on partition columns of the distributed tables.
  • The query should join the distributed tables on the equality of partition columns (table1.a = table2.a)
  • Shards of the distributed table should match one to one, i.e. each shard of table A should overlap with one and only one shard from table B.

For example lets assume we 3 hash distributed tables X, Y and Z and let X and Y have 4 shards while Z has 8 shards.

CREATE TABLE user (user_id int, name text);
SELECT create_distributed_table('user', 'user_id'); CREATE TABLE purchase (user_id int, amount int);
SELECT create_distributed_table('purchase', 'user_id'); CREATE TABLE comment (user_id int, comment text, rating int);
SELECT create_distributed_table('comment', 'user_id');

The following query would work since distributed tables user and purchase have the same number of shards and the join criteria is equality of partition columns:

SELECT * FROM user OUTER JOIN purchase ON user.user_id = purchase.user_id;

The following queries are not supported out of the box:

-- user and comment tables doesn’t have the same number of shards:
SELECT * FROM user OUTER JOIN comment ON user.user_id = comment.user_id; -- join condition is not on the partition columns:
SELECT * FROM user OUTER JOIN purchase ON user.user_id = purchase.amount; -- join condition is not equality:
SELECT * FROM user OUTER JOIN purchase ON user.user_id < purchase.user_id;

How Citus Processes OUTER JOINs When one-to-one matching between shards exists, then performing an outer join on large tables is equivalent to combining outer join results of corresponding shards.

Let’s look at how Citus handles an outer join query:

SELECT table1.a, table1.b AS b1, table2.b AS b2, table3.b AS b3, table4.b AS b4
FROM table1
FULL JOIN table2 ON table1.a = table2.a
FULL JOIN table3 ON table1.a = table3.a
FULL JOIN table4 ON table1.a = table4.a;

First, the query goes through the standard PostgreSQL planner and Citus uses this plan to generate a distributed plan where various checks about Citus’ support of the query are performed. Then individual queries that will go to workers for distributed table fragments are generated.

SELECT table1.a, table1.b AS b1, table2.b AS b2, table3.b AS b3, table4.b AS b4
FROM (((table1_102359 table1
FULL JOIN table2_102363 table2 ON ((table1.a = table2.a)))
FULL JOIN table3_102367 table3 ON ((table1.a = table3.a)))
FULL JOIN table4_102371 table4 ON ((table1.a = table4.a))) WHERE true
SELECT table1.a, table1.b AS b1, table2.b AS b2, table3.b AS b3, table4.b AS b4
FROM (((table1_102360 table1
FULL JOIN table2_102364 table2 ON ((table1.a = table2.a)))
FULL JOIN table3_102368 table3 ON ((table1.a = table3.a)))
FULL JOIN table4_102372 table4 ON ((table1.a = table4.a))) WHERE true
SELECT table1.a, table1.b AS b1, table2.b AS b2, table3.b AS b3, table4.b AS b4
FROM (((table1_102361 table1
FULL JOIN table2_102365 table2 ON ((table1.a = table2.a)))
FULL JOIN table3_102369 table3 ON ((table1.a = table3.a)))
FULL JOIN table4_102373 table4 ON ((table1.a = table4.a))) WHERE true
SELECT table1.a, table1.b AS b1, table2.b AS b2, table3.b AS b3, table4.b AS b4
FROM (((table1_102362 table1
FULL JOIN table2_102366 table2 ON ((table1.a = table2.a)))
FULL JOIN table3_102370 table3 ON ((table1.a = table3.a)))
FULL JOIN table4_102374 table4 ON ((table1.a = table4.a))) WHERE true

The resulting queries may seem complex at first but you can see that they are actually the same with the original query with just the table names are a bit different. This is because Citus stores the data in standard postgres tables called shards with the name as _. With 1-1 matching of shards, the distributed outer join is equivalent to the union of all outer joins of individual matching shards. In many cases you don’t even have to think about this as Citus simply takes care of you. If you’re sharding on some shared id, as is common in certain use cases, then Citus will do the join on the appropriate node without any inter-worker communication.

We hope you found the insight into how we perform distributed outer joins valuable. If you’re curious about trying Citus or learning how more works we encourage you to join the conversation with us on Slack.

 
 
 
 

How Distributed Outer Joins on PostgreSQL with Citus Work的更多相关文章

  1. 转载: C#: Left outer joins with LINQ

    I always considered Left Outer Join in LINQ to be complex until today when I had to use it in my app ...

  2. Lerning Entity Framework 6 ------ Joins and Left outer Joins

    Joins allow developers to combine data from multiple tables into a sigle query. Let's have a look at ...

  3. 分布式 PostgreSQL 集群(Citus)官方示例 - 多租户应用程序实战

    如果您正在构建软件即服务 (SaaS) 应用程序,您可能已经在数据模型中内置了租赁的概念. 通常,大多数信息与租户/客户/帐户相关,并且数据库表捕获这种自然关系. 对于 SaaS 应用程序,每个租户的 ...

  4. 分布式 PostgreSQL 集群(Citus),分布式表中的分布列选择最佳实践

    确定应用程序类型 在 Citus 集群上运行高效查询要求数据在机器之间正确分布.这因应用程序类型及其查询模式而异. 大致上有两种应用程序在 Citus 上运行良好.数据建模的第一步是确定哪些应用程序类 ...

  5. 分布式 PostgreSQL 集群(Citus)官方示例 - 实时仪表盘

    Citus 提供对大型数据集的实时查询.我们在 Citus 常见的一项工作负载涉及为事件数据的实时仪表板提供支持. 例如,您可以是帮助其他企业监控其 HTTP 流量的云服务提供商.每次您的一个客户端收 ...

  6. 分布式 PostgreSQL 集群(Citus)官方安装指南

    单节点 Citus Docker (Mac 与 Linux) Docker 镜像仅用于开发/测试目的, 并且尚未准备好用于生产用途. 您可以使用一个命令在 Docker 中启动 Citus: # st ...

  7. Citus 分布式 PostgreSQL 集群 - SQL Reference(查询分布式表 SQL)

    如前几节所述,Citus 是一个扩展,它扩展了最新的 PostgreSQL 以进行分布式执行.这意味着您可以在 Citus 协调器上使用标准 PostgreSQL SELECT 查询进行查询. Cit ...

  8. 跟我一起读postgresql源码(十三)——Executor(查询执行模块之——Join节点(上))

    Join节点 JOIN节点有以下三种: T_NestLoopState, T_MergeJoinState, T_HashJoinState, 连接类型节点对应于关系代数中的连接操作,PostgreS ...

  9. 跟我一起读postgresql源码(十四)——Executor(查询执行模块之——Join节点(下))

    3.HashJoin 节点 postgres=# explain select a.*,b.* from test_dm a join test_dm2 b on a.xxx = b.xxx; QUE ...

随机推荐

  1. spoj The Next Palindrome

    题意:比给出的数大的最小回文数. 先用前n/2长对称到后面,如果没变大,在中间加1,进位,再对称. //#pragma comment(linker,"/STACK:1024000000,1 ...

  2. Confluence 6 教程:在 Confluence 中导航

    当你对 Confluence 有所了解后,你会发现 Confluence 使用起来非常简单.这个教程主要是针对你使用的 Confluence 界面进行一些说明,同时向你展示在那里可以进行一些通用的任务 ...

  3. Java基础-String和StringBuilder类型(11)

    String类概述 字符串是由多个字符组成的一串数据字符串可以看成是字符数组 构造方法 public String(String original)public String(char[] value ...

  4. Black Widow CodeForces - 704C (dp)

    大意: 给定一个m个bool变量的方程, 求方程解的个数 给定方程的形式类似于这样 每个括号是一个子式, 每个子式里变量数不超过2, 每个变量出现次数不超过2, 方程右侧一直是1 对每个变量出现的式子 ...

  5. python-day48--mysql之视图、触发器、事务、存储过程、函数

    一.视图(不推荐使用) 1).视图是一个虚拟表(非真实存在) 2).使用视图我们可以把查询过程中的临时表摘出来,用视图去实现,这样以后再想操作该临时表的数据时就无需重写复杂的sql了,直接去视图中查找 ...

  6. python 爬取妹子图

    作为一个python还没入门的小白,搞懂这段代码实在是很不容易,还要去学html的知识(#黑脸) 因此我加上了注释,比较好读懂点 #coding=utf-8 import time import re ...

  7. java 中利用异或实现两个变量互换

    一般实现两个变量之间的互换要用第三个变量,这样做可以,但创建新变量,增加了系统开销.如果要交换的变量时两个整数型变量,可以用更高效的方法.例如:^(异或)操作,举例如下: package chapte ...

  8. 简话Angular 04 Angular过滤器详解

    一句话: filter是万能的数据处理器,可以过滤数据,排序数据,删除数据,扩展数据 1. 内置filter大全 url: https://docs.angularjs.org/api/ng/filt ...

  9. MVC,重写AuthorizeAttribute实现自己的权限验证(一)

    我们要实现下面的效果,某个controller,只允许某几个角色访问(admin,user,document controller) [MyAuthorize(Roles = "Admin, ...

  10. java并发编程:线程安全管理类--原子操作类--AtomicReferenceArray<E>

    1.类 AtomicReferenceArray<E> public class AtomicReferenceArray<E>extends Objectimplements ...