Full join issue: When use full join, the below join condition should not occur:

1, OR statement
2, an obvious false or true condition, like 1 = 0 or 1=1
3, the datatype of the join column should not be timestamp, this case is very common in pcp join, when use the report date as the join key.

ERROR: 42846: could not convert type "unknown" to numeric because of modifier

When the datatype of a column is supposed to be numeric in a UNION ALL context(basically there are more than three sub query parts), 
If there exists NULL value for some sub query, it is needed to cast the NULL value to the numeric datatype.

Error info: [Assert error]

This means the Redshift performance is bad, usually multiple large table join, more CTAS is needed.

Error info: query is automatically killed saying cancelled on user's request

Error info: could not devise a query plan for the given query

The issue is prabably due to there is an join condition in the full join clause. The join column is report_date, which is a date datatype, but there is function applied to report_date, which convert the datatype to timestamp. Unluckily redshift seems not supported this datatype in the join condition.

To solve this issue, we need to cast the join column to date again, like below:
DATEADD(MONTH,-12,CUR.REPORT_DATE) :: DATE = RESULT_PCP.REPORT_DATE ::DATE

Tips: Avoid to use the string concatenation when doing a join, create a temporary table for that particular column.

Redshift related documents

https://blogs.aws.amazon.com/bigdata/post/Tx31034QG0G3ED1/Top-10-Performance-Tuning-Techniques-for-Amazon-Redshift

http://docs.aws.amazon.com/redshift/latest/dg/c_Byte_dictionary_encoding.html

http://content.infotrustllc.com/interleaved-sorting-a-novel-solution-for-optimizing-multiple-query-patterns/

http://docs.aws.amazon.com/redshift/latest/dg/welcome.html

https://aws.amazon.com/redshift/pricing/

Sort key testing
Currently test on a table about 10millon rows, no significant difference, Need a larger data set to do the testing.

Previously, the table with sort key is not applied column compression, so this could be an potential reason why there is no significant improvement.

AWS Redshift typical error and potential root cause:的更多相关文章

  1. mount: error mounting /dev/root on /sysroot as ext3: Invalid argument

    /************************************************************************ * mount: error mounting /d ...

  2. [ERROR] error: error while loading <root>, error in opening zip file error: scala.reflect.internal.MissingRequirementError: object scala.runtime in compiler mirror not found.

    在家编译一个Apache的开源项目,在编译时遇到错误如下: error: error while loading <root>, error in opening zip file [ER ...

  3. AWS Redshift summary

    https://blogs.aws.amazon.com/bigdata/post/Tx31034QG0G3ED1/Top-10-Performance-Tuning-Techniques-for-A ...

  4. 亚马逊的PuTTY连接AWS出现network error connection refused,终极解决方案。

    使用PuTTY连接AWS的时候,一直出现network error connection refused.百度了这个问题,大家都说是SSH要设置成22.但是我已经设置过了,为什么还是遇到这个问题呢? ...

  5. AWS EC2实例Linux系统创建root用户并更改为root用户登录

    对于刚创建AWS EC2实例,或者经常使用AWS 实例的小伙伴们来说,刚创建的EC2实例是没有ROOT权限的,因此不能直接使用ROOT用户去登陆实例,也无法获取到root权限.一般情况下,EC2实例默 ...

  6. vue error:The template root requires exactly one element.

    error:[vue/valid-template-root] The template root requires exactly one element. 原因: 因为vue的模版中只有能一个根节 ...

  7. ERROR 1130: Host 'root@localhost' is not allowed to connect to MySQL server

    连接mysql时遇到的错误. 原因:该用户没有权限连接访问mysql数据库 解决方法:网站上搜了好多,试了都没有用.最终在登陆的信息页面用root用户登陆时不输入root密码即可.

  8. aws linux主机root帐号登录

    默认情况下,aws主机必须使用pem密码文件并且以ec2-user用户登录系统,之后很多操作都必须用sudo来以root权限执行操作,显得比较麻烦. 以下来自知乎的一个问答,亲测ok ## AWS E ...

  9. ERROR 1130: Host ’...′ is not allowed to connect to this MySQL server

    /******************************************************************** * ERROR 1130: Host ’...′ is no ...

随机推荐

  1. C++类的大小计算汇总

    C++中类涉及到虚函数成员.静态成员.虚继承.多继承.空类等. 类,作为一种类型定义,是没有大小可言的. 类的大小,指的是类的对象所占的大小.因此,用sizeof对一个类型名操作,得到的是具有该类型实 ...

  2. Lintcode105 Copy List with Random Pointer solution 题解

    [题目描述] A linked list is given such that each node contains an additional random pointer which could ...

  3. Webpack + vue 搭建

    前言: 为何使用webpack? 为何相对于gulp&grunt更有优势 WebPack(前往官网)可以看做是模块打包机:直接分析项目结构,找到JavaScript模块以及其它的一些浏览器不能 ...

  4. 线程(四)之Queue

    SynchronousQueue SynchronousQueue是无界的,是一种无缓冲的等待队列,但是由于该Queue本身的特性,在某次添加元素后必须等待其他线程取走后才能继续添加:可以认为Sync ...

  5. POJ 3278 抓奶牛(BFS入门题)

    描述 农夫约翰已被告知逃亡牛的位置,并希望立即抓住她.他开始于一个点Ñ(0≤ Ñ ≤100,000)上的数线和牛是在点ķ(0≤ ķ上相同数目的线≤100,000).农夫约翰有两种交通方式:步行和传送. ...

  6. mysql 常用sql语句

    权限 撤销权限revoke all on *.* from 'root'@'192.168.0.197' ; 撤销权限revoke all on *.* from 'xx_db' @'%'; 给指定用 ...

  7. 【分布式搜索引擎】Elasticsearch写入和读取数据过程

    一.Elasticsearch写人数据的过程 1)客户端选择一个node发送请求过去,这个node就是coordinating node(协调节点)2)coordinating node,对docum ...

  8. python学习(四)

  9. day037 mysql之单表查询

    一.单表查询语法 select distinct 字段1,字段2,... from 库名.表名 where 条件 group by 字段 having 筛选条件 order by 字段 limit 限 ...

  10. flask配置文件的几种方法

    配置文件的参数 flask中的配置文件是一个flask.config.Config对象(继承字典),默认配置为: { 'DEBUG': get_debug_flag(default=False), 是 ...