Full join issue: When use full join, the below join condition should not occur:

1, OR statement
2, an obvious false or true condition, like 1 = 0 or 1=1
3, the datatype of the join column should not be timestamp, this case is very common in pcp join, when use the report date as the join key.

ERROR: 42846: could not convert type "unknown" to numeric because of modifier

When the datatype of a column is supposed to be numeric in a UNION ALL context(basically there are more than three sub query parts), 
If there exists NULL value for some sub query, it is needed to cast the NULL value to the numeric datatype.

Error info: [Assert error]

This means the Redshift performance is bad, usually multiple large table join, more CTAS is needed.

Error info: query is automatically killed saying cancelled on user's request

Error info: could not devise a query plan for the given query

The issue is prabably due to there is an join condition in the full join clause. The join column is report_date, which is a date datatype, but there is function applied to report_date, which convert the datatype to timestamp. Unluckily redshift seems not supported this datatype in the join condition.

To solve this issue, we need to cast the join column to date again, like below:
DATEADD(MONTH,-12,CUR.REPORT_DATE) :: DATE = RESULT_PCP.REPORT_DATE ::DATE

Tips: Avoid to use the string concatenation when doing a join, create a temporary table for that particular column.

Redshift related documents

https://blogs.aws.amazon.com/bigdata/post/Tx31034QG0G3ED1/Top-10-Performance-Tuning-Techniques-for-Amazon-Redshift

http://docs.aws.amazon.com/redshift/latest/dg/c_Byte_dictionary_encoding.html

http://content.infotrustllc.com/interleaved-sorting-a-novel-solution-for-optimizing-multiple-query-patterns/

http://docs.aws.amazon.com/redshift/latest/dg/welcome.html

https://aws.amazon.com/redshift/pricing/

Sort key testing
Currently test on a table about 10millon rows, no significant difference, Need a larger data set to do the testing.

Previously, the table with sort key is not applied column compression, so this could be an potential reason why there is no significant improvement.

AWS Redshift typical error and potential root cause:的更多相关文章

  1. mount: error mounting /dev/root on /sysroot as ext3: Invalid argument

    /************************************************************************ * mount: error mounting /d ...

  2. [ERROR] error: error while loading <root>, error in opening zip file error: scala.reflect.internal.MissingRequirementError: object scala.runtime in compiler mirror not found.

    在家编译一个Apache的开源项目,在编译时遇到错误如下: error: error while loading <root>, error in opening zip file [ER ...

  3. AWS Redshift summary

    https://blogs.aws.amazon.com/bigdata/post/Tx31034QG0G3ED1/Top-10-Performance-Tuning-Techniques-for-A ...

  4. 亚马逊的PuTTY连接AWS出现network error connection refused,终极解决方案。

    使用PuTTY连接AWS的时候,一直出现network error connection refused.百度了这个问题,大家都说是SSH要设置成22.但是我已经设置过了,为什么还是遇到这个问题呢? ...

  5. AWS EC2实例Linux系统创建root用户并更改为root用户登录

    对于刚创建AWS EC2实例,或者经常使用AWS 实例的小伙伴们来说,刚创建的EC2实例是没有ROOT权限的,因此不能直接使用ROOT用户去登陆实例,也无法获取到root权限.一般情况下,EC2实例默 ...

  6. vue error:The template root requires exactly one element.

    error:[vue/valid-template-root] The template root requires exactly one element. 原因: 因为vue的模版中只有能一个根节 ...

  7. ERROR 1130: Host 'root@localhost' is not allowed to connect to MySQL server

    连接mysql时遇到的错误. 原因:该用户没有权限连接访问mysql数据库 解决方法:网站上搜了好多,试了都没有用.最终在登陆的信息页面用root用户登陆时不输入root密码即可.

  8. aws linux主机root帐号登录

    默认情况下,aws主机必须使用pem密码文件并且以ec2-user用户登录系统,之后很多操作都必须用sudo来以root权限执行操作,显得比较麻烦. 以下来自知乎的一个问答,亲测ok ## AWS E ...

  9. ERROR 1130: Host ’...′ is not allowed to connect to this MySQL server

    /******************************************************************** * ERROR 1130: Host ’...′ is no ...

随机推荐

  1. 批处理学习之Bat命令——获取当前盘符、当前目录、上级目录

    命令 当前盘符:%~d0 当前路径:%cd% 当前执行命令行:%0 当前bat文件路径:%~dp0 当前bat文件短路径:%~sdp0 测试 下载testBatPath.bat测试文件,双击.bat运 ...

  2. bzoj2124: 等差子序列线段树+hash

    bzoj2124: 等差子序列线段树+hash 链接 https://www.lydsy.com/JudgeOnline/problem.php?id=2124 思路 找大于3的等差数列其实就是找等于 ...

  3. CSS基础【1】:体验CSS

    CSS起源 web的衰落:在 web 早期(1990-1993),html是一个很局限的语言.几乎完全由用于描述段落,超链接,列表和标题的结构化元素组成.随着万维网的出现(用户交互体验的加强),对 h ...

  4. WEB 前端插件整理

    Vs Code 系统插件 #1 Bracket Pair Colorizer 让括号拥有独立的颜色,易于区分.可以配合任意主题使用. #2 Code Runner 非常强大的一款插件,能够运行多种语言 ...

  5. hduoj 1002 A + B Problem II

    原题链接:http://acm.hdu.edu.cn/showproblem.php?pid=1002 题目描述如下: A + B Problem II Time Limit: 2000/1000 M ...

  6. CSS布局学习(三) - position属性定义及解释(官网直译)

    static ①元素的位置是在文档正常布局流中的位置. ②设置top right bottom left与z-index无效. ③在未指定position时,static是默认值 以下例子进行说明: ...

  7. 将nginx添加至service服务

    一.问题描述: 无法用service命令启动nginx 二.问题分析: /etc/init.d/目录下缺少nginx默认启动脚本 三.问题解决: 在/etc/init.d/路径下添加脚本文件,名称为n ...

  8. CentOS7.3上如何安装Apache/2.4.34

    1)卸载系统自带的httpd Centos可能自带了httpd,但是版本可能会较低,执行下面的命令检测是否已经安装了httpd rpm -qa | grep httpd 如果检测已经安装了二进制的ht ...

  9. 雷林鹏分享:jQuery EasyUI 数据网格 - 启用行内编辑

    jQuery EasyUI 数据网格 - 启用行内编辑 可编辑的功能是最近添加到数据网格(datagrid)的.它可以使用户添加一个新行到数据网格(datagrid).用户也可以更新一个或多个行. 本 ...

  10. Practical Node.js (2018版) 14章, async code in Node

    Asynchronous Code in Node 历史上,Node开发者只能用回调和事件emitters. 现在可以使用一些异步的语法: async module Promises Async/aw ...