SSIS中出现数据流数据源假死状态的解决办法
相信开发过Sql Server SSIS的人都遇到过在数据流中数据源假死的问题,特别是Excel Source特别容易假死,当job执行到数据流中的Excel Source时,既不报错也不执行,也没有任何日志,Job就一直死在这里了。。。然后当你停掉假死的Job,重新执行Job,Excel Source又莫名其妙的顺利执行完成了。。。毫无疑问这是SSIS中的一个Bug,但是问题是如何避免这个Bug。
下面是我收集到的一些关于SSIS中数据流的数据源假死信息,供大家参考:
解决办法一,将数据流中数据源连接器的Delay Validation一定要设置为True:
I have a Data Flow Task that is hanging on excecution. The flow is simple, makes two queries to different tables (Both with a couple of joins), then sorts and merges the otuputs through a common id, adds a static column to all the records, saves the row count in a user variable for later use and finally inserts into a table on another DB. We are using OLE DB Sources and Destination. Source is MSSQL 2000 and Destination is MSSQL 2012
Symptoms:
- When excecuting, the Data Flow gets the usual yellow "running" icon. However when you double click to see the Data Flow, non of the elements have any yellow, red or green mark.
- This goes on for long periods of time, at first it lasted around 20 minutes, after that it started getting longer or simply not returning at all.
- Output shows: Information: 0x40043006 at Load Sandbox Table, SSIS.Pipeline: Prepare for Execute phase is beginning. Information: 0x40043007 at Load Sandbox Table, SSIS.Pipeline: Pre-Execute phase is beginning. And nothing more until the excecution is stopped.
- Yes, this has worked before. And yes, we have used a single query (in a stored procedure) to do this ETL but we wanted to migrate all the steps to SSIS.
Failed solutions:
- There are no lookups.
- Default buffer size for the task flow was increased to 40485760 then to 80971520.
- Default buffer max rows for task was set to 1000000.
- Delay Validation was set to True for the task.
- All elements inside the task were set Validate External Data to False.
- Both queries had: SET FMTONLY OFF; SET NOCOUNT ON; added at the beggining.
- Both queries had MAXDOP set to 1.
- Setting project's Run 64 bit Runtime to False.
- Changed destination load from Table or View to Table or View - Fast load with no locks or constraints.
- Set rows per batch to 1000 for fast load.
- Some work arounds propose to separate the task flow into two or more task flows. But this is not possible since what we need to do is a merge of the information found on both source queries.
Extra bits: I really hope someone can help me. I am fairly new to SSIS, this is the first time I use it. I usually work with Pentaho for my ETL but the client needs the solution to be implemented on SSIS. I've been battling with this issue for a couple of days now and I'm starting to run out of ideas to solve it.
When ran through the command line it gets stuck too and I get the following output:
Progress: 2013-03-19 14:36:26.21
Source: Load Sandbox Table
Validating: 0% complete
End Progress
Progress: 2013-03-19 14:36:26.21
Source: Load Sandbox Table
Validating: 12% complete
End Progress
Progress: 2013-03-19 14:36:26.22
Source: Load Sandbox Table
Validating: 25% complete
End Progress
Progress: 2013-03-19 14:36:26.22
Source: Load Sandbox Table
Validating: 37% complete
End Progress
Progress: 2013-03-19 14:36:26.23
Source: Load Sandbox Table
Validating: 50% complete
End Progress
Progress: 2013-03-19 14:36:26.25
Source: Load Sandbox Table
Validating: 62% complete
End Progress
Progress: 2013-03-19 14:36:26.25
Source: Load Sandbox Table
Validating: 75% complete
End Progress
Progress: 2013-03-19 14:36:26.25
Source: Load Sandbox Table
Validating: 87% complete
End Progress
Progress: 2013-03-19 14:36:26.25
Source: Load Sandbox Table
Validating: 100% complete
End Progress
Warning: 2013-03-19 14:36:26.26
Code: 0x80047076
Source: Load Sandbox Table SSIS.Pipeline
Description: The output column "ITEM_OID (1)" (47) on output "Merge Join Outp
ut" (28) and component "Merge Join" (11) is not subsequently used in the Data Fl
ow task. Removing this unused output column can increase Data Flow task performa
nce.
End Warning
Progress: 2013-03-19 14:36:26.27
Source: Load Sandbox Table
Prepare for Execute: 0% complete
End Progress
Progress: 2013-03-19 14:36:26.27
Source: Load Sandbox Table
Prepare for Execute: 12% complete
End Progress
Progress: 2013-03-19 14:36:26.27
Source: Load Sandbox Table
Prepare for Execute: 25% complete
End Progress
Progress: 2013-03-19 14:36:26.27
Source: Load Sandbox Table
Prepare for Execute: 37% complete
End Progress
Progress: 2013-03-19 14:36:26.27
Source: Load Sandbox Table
Prepare for Execute: 50% complete
End Progress
Progress: 2013-03-19 14:36:26.27
Source: Load Sandbox Table
Prepare for Execute: 62% complete
End Progress
Progress: 2013-03-19 14:36:26.27
Source: Load Sandbox Table
Prepare for Execute: 75% complete
End Progress
Progress: 2013-03-19 14:36:26.27
Source: Load Sandbox Table
Prepare for Execute: 87% complete
End Progress
Progress: 2013-03-19 14:36:26.27
Source: Load Sandbox Table
Prepare for Execute: 100% complete
End Progress
Progress: 2013-03-19 14:36:26.31
Source: Load Sandbox Table
Pre-Execute: 0% complete
End Progress
Progress: 2013-03-19 14:36:26.31
Source: Load Sandbox Table
Pre-Execute: 12% complete
End Progress
Progress: 2013-03-19 14:36:26.31
Source: Load Sandbox Table
Pre-Execute: 25% complete
End Progress
Progress: 2013-03-19 14:36:26.34
Source: Load Sandbox Table
Pre-Execute: 37% complete
End Progress
Progress: 2013-03-19 14:36:45.69
Source: Load Sandbox Table
Pre-Execute: 50% complete
End Progress
After that it freezes again.
SOLUTION (Posting this here because I can't answer my own question for another 5 hours, I'll do it when I'm allowed to.) I finally got it. It turns out there is an issue with the validation, but not only SSIS elements go through that validation, as stated in the fourth failed solution of the question. The CONNECTIONS also get validated and have their own Delay Validation property, which needs to be set to true. After that the excecution time went from 40+ mins or no running to less than a minute for the full process (This is just one step of a much bigger process) I hope people with this same problem can find this solution easily because there's a lot of people running into this problem and nearly no solutions posted online.
In a nutshell: Check that all your elements involved in the task, including the DB connections have Delay Validation Property set to True.
解决办法二,将Oledb Source中的表或视图改为Sql Command
Fixed my problem by changing the Data access mode to SQL Command and pasting my view into the SQL command text in the OLE DB Source.
解决办法三,将数据流中Source和Destination控件属性中的ValidateExternalMetadata属性设置为False
引用老外的原话
We already had our Delayed Validations set to True and couldnt/didnt want to change it to a SQL statement.
I came across ValidateExternalMetadata in the dataflows which I changed to False and that seem to work like a champ.

SSIS中出现数据流数据源假死状态的解决办法的更多相关文章
- IIS关于“ 配置错误 不能在此路径中使用此配置节”的解决办法
IIS关于“ 配置错误 不能在此路径中使用此配置节”的解决办法 原文链接:http://www.cnblogs.com/200325074/p/3679316.html 今天刚安装好IIS8.5, 我 ...
- Excel在任务栏中只显示一个窗口的解决办法
Excel在任务栏中只显示一个窗口的解决办法 以前朋友遇到过这个问题,这次自己又遇到了,习惯了以前的那种在任务栏中显示全部窗口,方便用Alt+Tab键进行切换. 如果同时打开许多Excel工作簿, ...
- 在ASP.net中的UpdatePanel,弹窗失败解决办法
原文:在ASP.net中的UpdatePanel,弹窗失败解决办法 最开始我用: Response.Write("<script>alert('和哈呵呵呵呵呵呵!')</s ...
- Ubuntu中Android SDK Manager无法更新解决办法
Ubuntu中Android SDK Manager无法更新解决办法http://hi.baidu.com/petercao2008/item/d7a64441f04668e81e19bc1a
- 工作总结 EntityFramework中出现DateTime2异常的完美解决办法
EntityFramework中出现DateTime2异常的完美解决办法 今天在使用entityframework往数据库插入数据的时候,突然出现了一个数据类型转换异常的问题: System.Da ...
- 把myeclipse中的web项目导入eclipse中不能编程web项目的解决办法
title: 把myeclipse中的web项目导入eclipse中不能编程web项目的解决办法 tags: grammar_cjkRuby: true --- 右键单击项目,properties-- ...
- WebAPI中无法获取Session对象的解决办法
在MVC的WebApi中默认是没有开启Session会话支持的.需要在Global中重写Init方法来指定会话需要支持的类型 public override void Init() { PostAut ...
- Windows中遇到的些问题及解决办法
1.win7 无法将程序锁定到任务栏 由于不正确的使用去掉桌面快捷键小箭头等,导致无法将程序固定到任务栏 如:http://jingyan.baidu.com/article/ac6a9a5e41bd ...
- asp.net开发中遇到的奇葩bug及解决办法(会持续更新。。。)
1,不知道你们遇没遇到过,在vs2010或更高版本上运行程序的时候,完全没问题,放在IIS中出现了问题,就比如左侧是菜单项,点击菜单右边显示,如果菜单链接是这样:content.aspx,而另一个链接 ...
随机推荐
- C#迭代器、装箱/拆箱、重载等
迭代器 迭代器是什么? 迭代器是作为一个容器,将要遍历的数据放入,通过统一的接口返回相同类型的值. 为什么要用迭代器? 为何了为集合提供统一的遍历方式,迭代器模式使得你能够获取到序列中的所有元素而不用 ...
- Angular中EventEmitter不是泛型类型
今天在做angular发射数据时,报错:EventEmitter不是泛型类型, 当时第一点就想到是不是没有引入,一看引入了 后面看api文档发现引入错了,其实是引入下面的 可能是之前快捷输入时,IDE ...
- 【转载】windows 下重置 mysql 的 root 密码
今天发现 WordPress 连接不上数据库,登录 window server 服务器查看,所有服务均运行正常. 使用 root 账号登录 mysql 数据库,结果提示密码不匹配.我突然意识到,服 ...
- bootstrap学习笔记细化(表单)
主要属性: class="form-inline" 水平排列 class="form-group" 组键 form-control 圆角方框发光 input-l ...
- QT5.9 新特性与版本回顾
原文链接: http://blog.qt.io/blog/2017/05/31/qt-5-9-released 翻译内容如下,采用的是第三方某在线翻译软件,所以有些地方不是太精确,纵然大吉做了一定的调 ...
- Python3 中日语料分句实现
0. 背景 因为最近在看平行语料句对齐.词对齐的缘故,想做对齐的话需要先做一个分句. 一开始利用正则和引号开关标志写了一种方法,中间想到一个小技巧,写出来比较简单通用,想把这一小段代码分享一下. 1. ...
- 一次线上bug引起的反思
今天线上又出现了一个bug,而且代码是我写的.之前这个问题也出现过,不过由于每次情况都不同,改来改去总是改不完.之后领导知道后也很恼火,让测试把每种情况都测试了下,而我也又一次重新检查了下代码.当时确 ...
- asp.net mvc4 小问题
最近在学习mvc4中间出现一些问题.留作记录.. 1.新建立的项目在vs2013中运行后会出现一个长轮询..这个叫browserLink 是vs2013中新加入的东西.至于更多解释.直接百度.. 关闭 ...
- Mybatis源码解析优秀博文
最近阅读了许久的mybatis源码,小有所悟.同时也发现网上有许多优秀的mybatis源码讲解博文.本人打算把自己阅读过的.觉得不错的一些博文列出来.以此进一步加深对mybatis框架的理解.其实还有 ...
- 第一个c程序和vs2017 在打开MFC rc文件时找不到rcdll.dl
第一个c程序 #include<stdio.h> int main() { ; ; printf_s("请输入两个数字:"); scanf_s("%d,%d& ...