AWS Redshift typical error and potential root cause:
Full join issue: When use full join, the below join condition should not occur:
1, OR statement
2, an obvious false or true condition, like 1 = 0 or 1=1
3, the datatype of the join column should not be timestamp, this case is very common in pcp join, when use the report date as the join key.
ERROR: 42846: could not convert type "unknown" to numeric because of modifier
When the datatype of a column is supposed to be numeric in a UNION ALL context(basically there are more than three sub query parts),
If there exists NULL value for some sub query, it is needed to cast the NULL value to the numeric datatype.
Error info: [Assert error]
This means the Redshift performance is bad, usually multiple large table join, more CTAS is needed.
Error info: query is automatically killed saying cancelled on user's request
Error info: could not devise a query plan for the given query
The issue is prabably due to there is an join condition in the full join clause. The join column is report_date, which is a date datatype, but there is function applied to report_date, which convert the datatype to timestamp. Unluckily redshift seems not supported this datatype in the join condition.
To solve this issue, we need to cast the join column to date again, like below:
DATEADD(MONTH,-12,CUR.REPORT_DATE) :: DATE = RESULT_PCP.REPORT_DATE ::DATE
Tips: Avoid to use the string concatenation when doing a join, create a temporary table for that particular column.
Redshift related documents
http://docs.aws.amazon.com/redshift/latest/dg/c_Byte_dictionary_encoding.html
http://docs.aws.amazon.com/redshift/latest/dg/welcome.html
https://aws.amazon.com/redshift/pricing/
Sort key testing
Currently test on a table about 10millon rows, no significant difference, Need a larger data set to do the testing.
Previously, the table with sort key is not applied column compression, so this could be an potential reason why there is no significant improvement.
AWS Redshift typical error and potential root cause:的更多相关文章
- mount: error mounting /dev/root on /sysroot as ext3: Invalid argument
/************************************************************************ * mount: error mounting /d ...
- [ERROR] error: error while loading <root>, error in opening zip file error: scala.reflect.internal.MissingRequirementError: object scala.runtime in compiler mirror not found.
在家编译一个Apache的开源项目,在编译时遇到错误如下: error: error while loading <root>, error in opening zip file [ER ...
- AWS Redshift summary
https://blogs.aws.amazon.com/bigdata/post/Tx31034QG0G3ED1/Top-10-Performance-Tuning-Techniques-for-A ...
- 亚马逊的PuTTY连接AWS出现network error connection refused,终极解决方案。
使用PuTTY连接AWS的时候,一直出现network error connection refused.百度了这个问题,大家都说是SSH要设置成22.但是我已经设置过了,为什么还是遇到这个问题呢? ...
- AWS EC2实例Linux系统创建root用户并更改为root用户登录
对于刚创建AWS EC2实例,或者经常使用AWS 实例的小伙伴们来说,刚创建的EC2实例是没有ROOT权限的,因此不能直接使用ROOT用户去登陆实例,也无法获取到root权限.一般情况下,EC2实例默 ...
- vue error:The template root requires exactly one element.
error:[vue/valid-template-root] The template root requires exactly one element. 原因: 因为vue的模版中只有能一个根节 ...
- ERROR 1130: Host 'root@localhost' is not allowed to connect to MySQL server
连接mysql时遇到的错误. 原因:该用户没有权限连接访问mysql数据库 解决方法:网站上搜了好多,试了都没有用.最终在登陆的信息页面用root用户登陆时不输入root密码即可.
- aws linux主机root帐号登录
默认情况下,aws主机必须使用pem密码文件并且以ec2-user用户登录系统,之后很多操作都必须用sudo来以root权限执行操作,显得比较麻烦. 以下来自知乎的一个问答,亲测ok ## AWS E ...
- ERROR 1130: Host ’...′ is not allowed to connect to this MySQL server
/******************************************************************** * ERROR 1130: Host ’...′ is no ...
随机推荐
- 加密方法与HTTPS 原理详解
一:加密方法: 1,对称加密 AES,3DES,DES等,适合做大量数据或数据文件的加解密. 2,非对称加密 如RSA,Rabin.公钥加密,私钥解密.对大数据量进行加解密时性能较低. 二:https ...
- NodeJS:(二)基础常用API
node.js中文网:http://nodejs.cn/api/ (path.Buffer.events.fs) ①path路径-----const {resolve} = require('path ...
- P1325 雷达安装
传送门 思路: 采取贪心的思想. 把每个岛屿看作圆心,以雷达的范围 d 为半径,求出与 x 轴的左右两个交点,两交点所夹的区间就需要放置一个雷达,这样就把这道题转换为了区间取点问题.在枚举岛屿时,记录 ...
- STL.vector.iterator的序号
ZC:网上查到,使用vector时,只要将 find到的iterator(itX)减去vector::begin() 就可以得到itX的序号. 1.需求:得到 某个 iterator在 vector中 ...
- IDEA spirng boot @Autowired注解 mapper出现红色下划线解决方法
如图所示,解决方法为: 把勾去掉即可.
- mac 安装工具列表
1,jdk idea datagrip 2,maven download 3.1idea config maven 3.2 down load myql connector & datagr ...
- Go语言开发Prometheus Exporter示例
一.Prometheus中的基本概念 Prometheus将所有数据存储为时间序列,这里先来了解一下prometheus中的一些基本概念 指标名和标签每个时间序列都由指标名和一组键值对(也称为标签)唯 ...
- 机器学习 之梯度提升树GBDT
目录 1.基本知识点简介 2.梯度提升树GBDT算法 2.1 思路和原理 2.2 梯度代替残差建立CART回归树 1.基本知识点简介 在集成学习的Boosting提升算法中,有两大家族:第一是AdaB ...
- Luffy之支付宝支付开发API
发起支付 接入支付宝 支付的大致流程如下图: 部分节点详解: 沙箱环境 是支付宝提供给开发者的 ...
- [hdu P1599] find the mincost route
[hdu P1599] find the mincost route 杭州有N个景区,景区之间有一些双向的路来连接,现在8600想找一条旅游路线,这个路线从A点出发并且最后回到A点,假设经过的路线为V ...