Spark Streaming从flume 中使用Poll拉取数据时,报如下错误:

Error while processing transaction.
java.lang.IllegalStateException: begin() called when transaction is OPEN!
  at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
  at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor$$anonfun$populateEvents$1.apply(TransactionProcessor.scala:114)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor$$anonfun$populateEvents$1.apply(TransactionProcessor.scala:113)
  at scala.Option.foreach(Option.scala:236)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.populateEvents(TransactionProcessor.scala:113)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:243)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:43)
  at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  at java.lang.Thread.run(Thread.java:745)
2020-04-08 05:20:15,120 (Spark Sink Processor Thread - 10) [WARN - org.apache.spark.streaming.flume.sink.Logging$class.logWarning(Logging.scala:59)] Spark was unable to successfully process the events. Transaction is being rolled back.
2020-04-08 05:20:15,120 (New I/O worker #1) [WARN - org.apache.spark.streaming.flume.sink.Logging$class.logWarning(Logging.scala:59)] Received an error batch - no events were received from channel!

报错原因:  

  flume的lib目录下的scala-library.xx jar 版本有问题。

解决办法:

1、将flume的lib目录下的 scala-library-2.10.5.jar 删除或者重命名。

2、建议上传 scala-library-2.11.x.jar 包。

如下图:

3、附上 scala-library-2.11.8.jar ,需要请自提。

  [百度网盘:]
  链接:https://pan.baidu.com/s/1GsNwQyt89pobWHWkZO52jw

  提取码:41ik

Error while processing transaction.java.lang.IllegalStateException: begin() called when transaction is OPEN!的更多相关文章

  1. flume-sink报错 java.lang.IllegalStateException: close() called when transaction is OPEN - you must either commit or rollback first

    1. 确认代码无误(根据情况修改,表示若获得不了数据不会自动commit或者rollback): Event event = channel.take(); if (event == null) { ...

  2. java.lang.IllegalStateException: Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

    java.lang.IllegalStateException: Not allowed to create transaction on sharedEntityManager - use Spri ...

  3. [spring cloud] [error] java.lang.IllegalStateException: Only one connection receive subscriber allowed.

    前言 最近在开发api-gateway的时候遇到了一个问题,网上能够找到的解决方案也很少,之后由公司的大佬解决了这个问题.写下这篇文章记录一下解决方案.希望可以帮助到更多的人. 环境 java版本:8 ...

  4. java.lang.IllegalStateException: Active Spring transaction synchronization or active JTA transaction with specified [javax.transaction.TransactionManager] required

    错误信息: java.lang.IllegalStateException: Active Spring transaction synchronization or active JTA trans ...

  5. Error:Execution failed for task ':app:compileDebugAidl'. > java.lang.IllegalStateException: aidl is missing from '/Users/renguodong/Library/Android/sdk/build-tools/26.0.2/aidl'

    错误信息:Error:Execution failed for task ':app:compileDebugAidl'. > java.lang.IllegalStateException: ...

  6. cause: java.lang.IllegalStateException: Serialized class com.taotao.pojo.TbItem must implement java.io.Serializable

    HTTP Status 500 - Request processing failed; nested exception is com.alibaba.dubbo.rpc.RpcException: ...

  7. nested exception is java.lang.IllegalStateException: Cannot forward after response has been committed

    org.springframework.web.util.NestedServletException: Request processing failed; nested exception is ...

  8. myeclipse 无法启动 java.lang.IllegalStateException: Unable to acquire application service. Ensure that the org.eclipse.core.runtime bundle is resolved and started (see config.ini).

    把myeclipse10 按照目录完整拷贝到了另外一台电脑, 另外的目录 原安装目录 D\:\soft\i\myeclipse10 新安装目录 E\:\soft\myeclipse10 双击启动失败, ...

  9. java.lang.IllegalStateException: getOutputStream() has already been called for this response

    ERROR [Engine] StandardWrapperValve[jsp]: Servlet.service() for servlet jsp threw exceptionjava.lang ...

随机推荐

  1. 超详细,多图文介绍redis集群方式并搭建redis伪集群

    超详细,多图文介绍redis集群方式并搭建redis伪集群 超多图文,对新手友好度极好.敲命令的过程中,难免会敲错,但为了截好一张合适的图,一旦出现一点问题,为了好的演示效果,就要从头开始敲.且看且珍 ...

  2. NetAnalyzer笔记 之 十一 打造自己的协议分析语言(1)初衷与语法构想

    回头看看NetAnalyzer开发系文档上次一篇竟然是2016年,老脸一红.不过这几年墨云成功过的讨到一个温柔贤淑的老婆,有了一个幸福的家庭,去年9月又有了一个大胖儿子,想想也就释然了^_^ 其实这几 ...

  3. jenkins操作

    jenkins忘记用户名以及登录密码的解决方法 1.jenkins 根目录下找到config.xml,修改配置  <useSecurity>true</useSecurity> ...

  4. C语言程序设计(八) 数组

    第八章 数组 //L8-1 #include <stdio.h> int main() { int score1, score2, score3, score4, score5; int ...

  5. vscode使用cnpm报错

     1.在wind10搜索框里输入 Windows PowerShell 进入这个界面 2.打开Windows PowerShell 之后 输入命令:set-ExecutjionPolicy Remot ...

  6. Spark入门(六)--Spark的combineByKey、sortBykey

    spark的combineByKey combineByKey的特点 combineByKey的强大之处,在于提供了三个函数操作来操作一个函数.第一个函数,是对元数据处理,从而获得一个键值对.第二个函 ...

  7. python3.4.3 调用http接口 解析response xml后插入数据库

    工作中需要调用一个http的接口,等不及java组开发,就试着用python去调用.Python版本3.4.3 完整的流程包括:从sqlServer取待调用的合同列表 -> 循环调用http接口 ...

  8. 详解如何快速使用数据可视化BI软件创建医疗运营监控数据中心大屏

    灯果数据可视化BI软件是新一代人工智能数据可视化大屏软件,内置丰富的大屏模板,可视化编辑操作,无需任何经验就可以创建属于你自己的大屏.大家可以在他们的官网下载软件.   本文以医疗运营监控数据中心大屏 ...

  9. 4. selenium中鼠标和键盘操作

    一.鼠标操作 第一步:引入模块函数 from selenium.webdriver.common.action_chains import ActionChains 第二步:元素定位 element ...

  10. Java并发编程(03):多线程并发访问,同步控制

    本文源码:GitHub·点这里 || GitEE·点这里 一.并发问题 多线程学习的时候,要面对的第一个复杂问题就是,并发模式下变量的访问,如果不理清楚内在流程和原因,经常会出现这样一个问题:线程处理 ...