AWS Redshift typical error and potential root cause:
Full join issue: When use full join, the below join condition should not occur:
1, OR statement
2, an obvious false or true condition, like 1 = 0 or 1=1
3, the datatype of the join column should not be timestamp, this case is very common in pcp join, when use the report date as the join key.
ERROR: 42846: could not convert type "unknown" to numeric because of modifier
When the datatype of a column is supposed to be numeric in a UNION ALL context(basically there are more than three sub query parts), 
If there exists NULL value for some sub query, it is needed to cast the NULL value to the numeric datatype.
Error info: [Assert error]
This means the Redshift performance is bad, usually multiple large table join, more CTAS is needed.
Error info: query is automatically killed saying cancelled on user's request
Error info: could not devise a query plan for the given query
The issue is prabably due to there is an join condition in the full join clause. The join column is report_date, which is a date datatype, but there is function applied to report_date, which convert the datatype to timestamp. Unluckily redshift seems not supported this datatype in the join condition.
To solve this issue, we need to cast the join column to date again, like below:
DATEADD(MONTH,-12,CUR.REPORT_DATE) :: DATE = RESULT_PCP.REPORT_DATE ::DATE
Tips: Avoid to use the string concatenation when doing a join, create a temporary table for that particular column.
Redshift related documents
http://docs.aws.amazon.com/redshift/latest/dg/c_Byte_dictionary_encoding.html
http://docs.aws.amazon.com/redshift/latest/dg/welcome.html
https://aws.amazon.com/redshift/pricing/
Sort key testing
Currently test on a table about 10millon rows, no significant difference, Need a larger data set to do the testing.
Previously, the table with sort key is not applied column compression, so this could be an potential reason why there is no significant improvement.
AWS Redshift typical error and potential root cause:的更多相关文章
- mount: error mounting /dev/root on /sysroot as ext3: Invalid argument
		
/************************************************************************ * mount: error mounting /d ...
 - [ERROR] error: error while loading <root>, error in opening zip file error: scala.reflect.internal.MissingRequirementError: object scala.runtime in compiler mirror not found.
		
在家编译一个Apache的开源项目,在编译时遇到错误如下: error: error while loading <root>, error in opening zip file [ER ...
 - AWS Redshift summary
		
https://blogs.aws.amazon.com/bigdata/post/Tx31034QG0G3ED1/Top-10-Performance-Tuning-Techniques-for-A ...
 - 亚马逊的PuTTY连接AWS出现network error connection refused,终极解决方案。
		
使用PuTTY连接AWS的时候,一直出现network error connection refused.百度了这个问题,大家都说是SSH要设置成22.但是我已经设置过了,为什么还是遇到这个问题呢? ...
 - AWS EC2实例Linux系统创建root用户并更改为root用户登录
		
对于刚创建AWS EC2实例,或者经常使用AWS 实例的小伙伴们来说,刚创建的EC2实例是没有ROOT权限的,因此不能直接使用ROOT用户去登陆实例,也无法获取到root权限.一般情况下,EC2实例默 ...
 - vue error:The template root requires exactly one element.
		
error:[vue/valid-template-root] The template root requires exactly one element. 原因: 因为vue的模版中只有能一个根节 ...
 - ERROR 1130: Host 'root@localhost' is not allowed to connect to MySQL server
		
连接mysql时遇到的错误. 原因:该用户没有权限连接访问mysql数据库 解决方法:网站上搜了好多,试了都没有用.最终在登陆的信息页面用root用户登陆时不输入root密码即可.
 - aws linux主机root帐号登录
		
默认情况下,aws主机必须使用pem密码文件并且以ec2-user用户登录系统,之后很多操作都必须用sudo来以root权限执行操作,显得比较麻烦. 以下来自知乎的一个问答,亲测ok ## AWS E ...
 - ERROR 1130: Host ’...′ is not allowed to connect to this MySQL server
		
/******************************************************************** * ERROR 1130: Host ’...′ is no ...
 
随机推荐
- bitbucket迁移
			
bitbucket 迁移 1.停止向旧仓库地址提交代码 [dev]2.导入代码至新仓库地址 [op]3.修改本地仓库地址 第一种方式:git remote set-url origin [url] ; ...
 - Odd Gnome【枚举】
			
问题 I: Odd Gnome 时间限制: 1 Sec 内存限制: 128 MB 提交: 234 解决: 144 [提交] [状态] [命题人:admin] 题目描述 According to t ...
 - UVA1388 Graveyard
			
思路 就是对于每个点,找出离他最近的目标点的距离 我使用了上取整和下取整实现,蓝书上的实现方法是坐标系缩放,每个点的目标位置就是它四舍五入的结果 具体证明见蓝书 代码 #include <cst ...
 - 编码原则 之 Persistence Ignorance
			
原文 The principle of Persistence Ignorance (PI) holds that classes modeling the business domain in a ...
 - Linux 软连接 (ln命令)
			
这是linux中一个非常重要命令.它的功能是为某一个文件在另外一个位置建立一个同不的链接,这个命令最常用的参数是-s,具体用法是:ln -s 源文件 目标文件. 当我们需要在不同的目录,用到相同的文件 ...
 - 使用python内置库pytesseract实现图片验证码的识别
			
环境准备: 1.安装Tesseract模块 git文档地址:https://digi.bib.uni-mannheim.de/tesseract/ 下载后就是一个exe安装包,直接右击安装即可,安装完 ...
 - while循环和递归
			
这个问题是在数据结构的二叉树添加结点的时候碰见 添加新结点的时候可以用while循环自身解决(这里这个方式更好) 也可以用递归解决 递归就像小明去楼顶取东西 ,从一楼开始爬,看,不是的,继续爬,每层 ...
 - Struts2 环境配置
			
1.下载struts2开发包 2.将apps中的 Struts-blank.war 解压,里面的 lib 中就是所需jar包 3.新建一个web project项目,将jar包导入复制粘贴到lib中 ...
 - Windows下使用CMD命令进入和退出MySQL数据库
			
一.进入 1.在CMD命令窗口敲入命令 mysql -hlocalhost -uroot -p 后按回车(注意这里的"-h"."-u"."-p&quo ...
 - ionic3使用cnpm可能会出现的问题
			
在跑一个ionic3项目的时候发现,新建的页面无法被识别,总是报错 cannot find modules '../pages/login/login.modules' 在排除多种可能性后,确定了是因 ...