AWS Redshift typical error and potential root cause:
Full join issue: When use full join, the below join condition should not occur:
1, OR statement
2, an obvious false or true condition, like 1 = 0 or 1=1
3, the datatype of the join column should not be timestamp, this case is very common in pcp join, when use the report date as the join key.
ERROR: 42846: could not convert type "unknown" to numeric because of modifier
When the datatype of a column is supposed to be numeric in a UNION ALL context(basically there are more than three sub query parts),
If there exists NULL value for some sub query, it is needed to cast the NULL value to the numeric datatype.
Error info: [Assert error]
This means the Redshift performance is bad, usually multiple large table join, more CTAS is needed.
Error info: query is automatically killed saying cancelled on user's request
Error info: could not devise a query plan for the given query
The issue is prabably due to there is an join condition in the full join clause. The join column is report_date, which is a date datatype, but there is function applied to report_date, which convert the datatype to timestamp. Unluckily redshift seems not supported this datatype in the join condition.
To solve this issue, we need to cast the join column to date again, like below:
DATEADD(MONTH,-12,CUR.REPORT_DATE) :: DATE = RESULT_PCP.REPORT_DATE ::DATE
Tips: Avoid to use the string concatenation when doing a join, create a temporary table for that particular column.
Redshift related documents
http://docs.aws.amazon.com/redshift/latest/dg/c_Byte_dictionary_encoding.html
http://docs.aws.amazon.com/redshift/latest/dg/welcome.html
https://aws.amazon.com/redshift/pricing/
Sort key testing
Currently test on a table about 10millon rows, no significant difference, Need a larger data set to do the testing.
Previously, the table with sort key is not applied column compression, so this could be an potential reason why there is no significant improvement.
AWS Redshift typical error and potential root cause:的更多相关文章
- mount: error mounting /dev/root on /sysroot as ext3: Invalid argument
/************************************************************************ * mount: error mounting /d ...
- [ERROR] error: error while loading <root>, error in opening zip file error: scala.reflect.internal.MissingRequirementError: object scala.runtime in compiler mirror not found.
在家编译一个Apache的开源项目,在编译时遇到错误如下: error: error while loading <root>, error in opening zip file [ER ...
- AWS Redshift summary
https://blogs.aws.amazon.com/bigdata/post/Tx31034QG0G3ED1/Top-10-Performance-Tuning-Techniques-for-A ...
- 亚马逊的PuTTY连接AWS出现network error connection refused,终极解决方案。
使用PuTTY连接AWS的时候,一直出现network error connection refused.百度了这个问题,大家都说是SSH要设置成22.但是我已经设置过了,为什么还是遇到这个问题呢? ...
- AWS EC2实例Linux系统创建root用户并更改为root用户登录
对于刚创建AWS EC2实例,或者经常使用AWS 实例的小伙伴们来说,刚创建的EC2实例是没有ROOT权限的,因此不能直接使用ROOT用户去登陆实例,也无法获取到root权限.一般情况下,EC2实例默 ...
- vue error:The template root requires exactly one element.
error:[vue/valid-template-root] The template root requires exactly one element. 原因: 因为vue的模版中只有能一个根节 ...
- ERROR 1130: Host 'root@localhost' is not allowed to connect to MySQL server
连接mysql时遇到的错误. 原因:该用户没有权限连接访问mysql数据库 解决方法:网站上搜了好多,试了都没有用.最终在登陆的信息页面用root用户登陆时不输入root密码即可.
- aws linux主机root帐号登录
默认情况下,aws主机必须使用pem密码文件并且以ec2-user用户登录系统,之后很多操作都必须用sudo来以root权限执行操作,显得比较麻烦. 以下来自知乎的一个问答,亲测ok ## AWS E ...
- ERROR 1130: Host ’...′ is not allowed to connect to this MySQL server
/******************************************************************** * ERROR 1130: Host ’...′ is no ...
随机推荐
- 【题解】Luogu P4054 [JSOI2009]计数问题
原题传送门 我自闭了qaq 这道题非常简单,因为1<=c<=100,所以直接对每个c开二维树状数组,操作就跟模板一样 写码5分钟,调码半小时,这道题的输入顺序是x1,x2,y1,y2,我真 ...
- vue 文件目录结构详解
vue 文件目录结构详解 本篇文章主要介绍了vue 文件目录结构详解,小编觉得挺不错的,现在分享给大家,也给大家做个参考.一起跟随小编过来看看吧 项目简介 基于 vue.js 的前端开发环境,用于前后 ...
- LOJ #10132. 「一本通 4.4 例 3」异象石
题目地址 LOJ 题解 神仙思路.思路参考自<算法竞赛进阶指南>. 考虑维护dfs序中相邻两个石头的距离,那么每次?的答案就是sum/2(首尾算相邻) 然后维护一下拿个平衡树/set维护一 ...
- Learning-Python【33】:并发编程之多进程
一.multiprocessing 模块介绍 Python 中的多线程无法利用多核优势,如果想要充分地使用多核 CPU 的资源(os.cpu_count()查看),在 Python 中大部分情况需要使 ...
- Apache web服务器(LAMP架构)
Apache web服务器(LAMP架构) apache介绍 1).世界上使用率最高的网站服务器,最高时可达70%:官方网站:apache.org 2).http 超文本协议 HTML 超文本标记语言 ...
- Pandas 基础(4) - 读/写 Excel 和 CSV 文件
这一节将分别介绍读/写 Excel 和 CSV 文件的各种方式: - 读入 CSV 文件 首先是准备一个 csv 文件, 这里我用的是 stock_data.csv, 文件我已上传, 大家可以直接下载 ...
- ckeditor5 增加居中alignment
https://ckeditor.com/docs/ckeditor5/latest/builds/guides/integration/installing-plugins.html 克隆下来 gi ...
- spark streaming 整合kafka(二)
转载:https://www.iteblog.com/archives/1326.html 和基于Receiver接收数据不一样,这种方式定期地从Kafka的topic+partition中查询最新的 ...
- 注册一个gitHub
GitHub 是一个面向开源及私有软件项目的托管平台,因为只支持 Git 作为唯一的版本库格式进行托管,故名 GitHub. GitHub 于 2008 年 4 月 10 日正式上线,除了 Git 代 ...
- linux安装elasticsearch及遇到的各种问题
1.获取elasticsearch https://www.elastic.co/downloads/elasticsearch 终端输入赋值的下载链接进行下载 wget https://artifa ...