AWS Redshift typical error and potential root cause:
Full join issue: When use full join, the below join condition should not occur:
1, OR statement
2, an obvious false or true condition, like 1 = 0 or 1=1
3, the datatype of the join column should not be timestamp, this case is very common in pcp join, when use the report date as the join key.
ERROR: 42846: could not convert type "unknown" to numeric because of modifier
When the datatype of a column is supposed to be numeric in a UNION ALL context(basically there are more than three sub query parts),
If there exists NULL value for some sub query, it is needed to cast the NULL value to the numeric datatype.
Error info: [Assert error]
This means the Redshift performance is bad, usually multiple large table join, more CTAS is needed.
Error info: query is automatically killed saying cancelled on user's request
Error info: could not devise a query plan for the given query
The issue is prabably due to there is an join condition in the full join clause. The join column is report_date, which is a date datatype, but there is function applied to report_date, which convert the datatype to timestamp. Unluckily redshift seems not supported this datatype in the join condition.
To solve this issue, we need to cast the join column to date again, like below:
DATEADD(MONTH,-12,CUR.REPORT_DATE) :: DATE = RESULT_PCP.REPORT_DATE ::DATE
Tips: Avoid to use the string concatenation when doing a join, create a temporary table for that particular column.
Redshift related documents
http://docs.aws.amazon.com/redshift/latest/dg/c_Byte_dictionary_encoding.html
http://docs.aws.amazon.com/redshift/latest/dg/welcome.html
https://aws.amazon.com/redshift/pricing/
Sort key testing
Currently test on a table about 10millon rows, no significant difference, Need a larger data set to do the testing.
Previously, the table with sort key is not applied column compression, so this could be an potential reason why there is no significant improvement.
AWS Redshift typical error and potential root cause:的更多相关文章
- mount: error mounting /dev/root on /sysroot as ext3: Invalid argument
/************************************************************************ * mount: error mounting /d ...
- [ERROR] error: error while loading <root>, error in opening zip file error: scala.reflect.internal.MissingRequirementError: object scala.runtime in compiler mirror not found.
在家编译一个Apache的开源项目,在编译时遇到错误如下: error: error while loading <root>, error in opening zip file [ER ...
- AWS Redshift summary
https://blogs.aws.amazon.com/bigdata/post/Tx31034QG0G3ED1/Top-10-Performance-Tuning-Techniques-for-A ...
- 亚马逊的PuTTY连接AWS出现network error connection refused,终极解决方案。
使用PuTTY连接AWS的时候,一直出现network error connection refused.百度了这个问题,大家都说是SSH要设置成22.但是我已经设置过了,为什么还是遇到这个问题呢? ...
- AWS EC2实例Linux系统创建root用户并更改为root用户登录
对于刚创建AWS EC2实例,或者经常使用AWS 实例的小伙伴们来说,刚创建的EC2实例是没有ROOT权限的,因此不能直接使用ROOT用户去登陆实例,也无法获取到root权限.一般情况下,EC2实例默 ...
- vue error:The template root requires exactly one element.
error:[vue/valid-template-root] The template root requires exactly one element. 原因: 因为vue的模版中只有能一个根节 ...
- ERROR 1130: Host 'root@localhost' is not allowed to connect to MySQL server
连接mysql时遇到的错误. 原因:该用户没有权限连接访问mysql数据库 解决方法:网站上搜了好多,试了都没有用.最终在登陆的信息页面用root用户登陆时不输入root密码即可.
- aws linux主机root帐号登录
默认情况下,aws主机必须使用pem密码文件并且以ec2-user用户登录系统,之后很多操作都必须用sudo来以root权限执行操作,显得比较麻烦. 以下来自知乎的一个问答,亲测ok ## AWS E ...
- ERROR 1130: Host ’...′ is not allowed to connect to this MySQL server
/******************************************************************** * ERROR 1130: Host ’...′ is no ...
随机推荐
- 2018年12月25日 圣诞节快乐 生成器plus
import time#导入时间模块 time.sleep(5)#输入间隔时间5秒 def p_bz():#使用函数 生产包子 for i in range(100): yield "包子% ...
- 从GitHub远程仓库中删除文件夹或文件
在上传项目到github时,忘记忽略了某个文件夹target,就直接push上去了, 最后意识到了此问题,决定删除掉远程仓库中的target文件夹 删除前: 删除后: 在github上只能删除仓库,却 ...
- jenkins安装部署全过程
基本配置: 1.Linux安装配置jdk环境 1.1.上传到 Linux 服务器:例如: 上传至: cd /usr/local 1.2.解压: rpm -ivh jdk-8u111-linux-x64 ...
- 2.nginx配置教程
1. nginx配置域名访问: 路径:vim /usr/local/nginx/conf/nginx.conf (这是我nginx的安装路径) 配置以后就不用通过ip(192.168.0.103)去访 ...
- linux存储管理之mount挂载
Mount 挂载详解 ====================================================================================本节内容: ...
- 浅谈中大型企业CMDB的建设
作者:嘉维蓝鲸产品总监,贺勇 针对CMDB这个主题,之前一直想写一篇文章来表达我的看法,但是之前一直不敢写,为什么?因为CMDB这个主题属于一提大家都懂,但是深入讨论大家都晕菜的一个话题:在2018年 ...
- The `android.dexOptions.incremental` property is deprecated and it has no effect on the build process.
编译报错:The android.dexOptions.incremental property is deprecated and it has no effect on the build pro ...
- jq初入行常用动画
--jq动画分类--(1)jQuery的动画其实就是将之前提到过的各种特效进行封装,并对其性能进行优化.(2)jQuery动画分为三个部分:非自定义动画,自定义动画,和全局动画设置. 一.非自定义动画 ...
- android SDK SDK Manager.exe 无法打开,一闪而过最终解决办法
这个问题是由于Android Studio 自动更新之后 ,针对android-sdk-windows\tools 这个文件进行了拆分删除 这里做一下说明: 经过Android studio处理之后, ...
- 简述采用四次握手机制释放TCP连接的四个步骤
(1)源结点A向目的结点B发送连接释放请求(FIN,seg=x),并且不再向B发送数据,但仍继续接收从B发来的数据. (2)目的结点B收到此连接释放请求后立即向A发出确认(ACK,ack=x+1),但 ...