Spark Streaming从flume 中使用Poll拉取数据时,报如下错误:

Error while processing transaction.
java.lang.IllegalStateException: begin() called when transaction is OPEN!
  at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
  at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor$$anonfun$populateEvents$1.apply(TransactionProcessor.scala:114)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor$$anonfun$populateEvents$1.apply(TransactionProcessor.scala:113)
  at scala.Option.foreach(Option.scala:236)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.populateEvents(TransactionProcessor.scala:113)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:243)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:43)
  at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  at java.lang.Thread.run(Thread.java:745)
2020-04-08 05:20:15,120 (Spark Sink Processor Thread - 10) [WARN - org.apache.spark.streaming.flume.sink.Logging$class.logWarning(Logging.scala:59)] Spark was unable to successfully process the events. Transaction is being rolled back.
2020-04-08 05:20:15,120 (New I/O worker #1) [WARN - org.apache.spark.streaming.flume.sink.Logging$class.logWarning(Logging.scala:59)] Received an error batch - no events were received from channel!

报错原因:  

  flume的lib目录下的scala-library.xx jar 版本有问题。

解决办法:

1、将flume的lib目录下的 scala-library-2.10.5.jar 删除或者重命名。

2、建议上传 scala-library-2.11.x.jar 包。

如下图:

3、附上 scala-library-2.11.8.jar ,需要请自提。

  [百度网盘:]
  链接:https://pan.baidu.com/s/1GsNwQyt89pobWHWkZO52jw

  提取码:41ik

Error while processing transaction.java.lang.IllegalStateException: begin() called when transaction is OPEN!的更多相关文章

  1. flume-sink报错 java.lang.IllegalStateException: close() called when transaction is OPEN - you must either commit or rollback first

    1. 确认代码无误(根据情况修改,表示若获得不了数据不会自动commit或者rollback): Event event = channel.take(); if (event == null) { ...

  2. java.lang.IllegalStateException: Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

    java.lang.IllegalStateException: Not allowed to create transaction on sharedEntityManager - use Spri ...

  3. [spring cloud] [error] java.lang.IllegalStateException: Only one connection receive subscriber allowed.

    前言 最近在开发api-gateway的时候遇到了一个问题,网上能够找到的解决方案也很少,之后由公司的大佬解决了这个问题.写下这篇文章记录一下解决方案.希望可以帮助到更多的人. 环境 java版本:8 ...

  4. java.lang.IllegalStateException: Active Spring transaction synchronization or active JTA transaction with specified [javax.transaction.TransactionManager] required

    错误信息: java.lang.IllegalStateException: Active Spring transaction synchronization or active JTA trans ...

  5. Error:Execution failed for task ':app:compileDebugAidl'. > java.lang.IllegalStateException: aidl is missing from '/Users/renguodong/Library/Android/sdk/build-tools/26.0.2/aidl'

    错误信息:Error:Execution failed for task ':app:compileDebugAidl'. > java.lang.IllegalStateException: ...

  6. cause: java.lang.IllegalStateException: Serialized class com.taotao.pojo.TbItem must implement java.io.Serializable

    HTTP Status 500 - Request processing failed; nested exception is com.alibaba.dubbo.rpc.RpcException: ...

  7. nested exception is java.lang.IllegalStateException: Cannot forward after response has been committed

    org.springframework.web.util.NestedServletException: Request processing failed; nested exception is ...

  8. myeclipse 无法启动 java.lang.IllegalStateException: Unable to acquire application service. Ensure that the org.eclipse.core.runtime bundle is resolved and started (see config.ini).

    把myeclipse10 按照目录完整拷贝到了另外一台电脑, 另外的目录 原安装目录 D\:\soft\i\myeclipse10 新安装目录 E\:\soft\myeclipse10 双击启动失败, ...

  9. java.lang.IllegalStateException: getOutputStream() has already been called for this response

    ERROR [Engine] StandardWrapperValve[jsp]: Servlet.service() for servlet jsp threw exceptionjava.lang ...

随机推荐

  1. 集成google翻译的小tips

    文章首发于github.io 2018-08-04 12:43:20 google翻译的强大,就像我们公司的slogan : "让语言无国界,让世人心相通" 友情提示: googl ...

  2. 必备技能一、webpack

    https://cloud.tencent.com/developer/section/1477376----->配置很重要 一.基本安装 mkdir webpack-demo &&am ...

  3. Nginx之负载均衡配置(一)

    前文我们聊了下nginx作为反向代理服务器,代理后端动态应用服务器的配置,回顾请参考https://www.cnblogs.com/qiuhom-1874/p/12430543.html:今天我们来聊 ...

  4. mysql中实现更新数据+1,再次更新数据-1

    $sql="update article set comment=comment^1 where a_id=2"; 这条语句你会发现当你更新comment字段时为1,再次更新时为0 ...

  5. JavaScript中的内存释放

    C.C++语言需要手动管理内存的分配与释放(常用方法:malloc(), calloc(), realloc()和free()等).而JavaScript与Java.C#相似,内置了垃圾回收器,能自动 ...

  6. 【工具】Intel HLS工具

    目前新版本的Quartus 软件 都自带有  HLS 工具,比如 18.1版本安装以后,HLS相关文件夹如下: HLS工具 也可以单独下载: https://www.intel.com/content ...

  7. 何为内存模型(JMM)?

    前言 任何一门语言都有其语言规范,从逻辑上我们可划分为语法规范和语义规范,语法规范则是描述了如何通过相关语法编写可执行的程序,而语义规范则是指通过语法编写的程序所构造出的具体含义.语言只要具备存储(比 ...

  8. Flask-Sockets实时的监控画面

    Ajax轮询是通过特定的的时间间隔(如每1秒),由浏览器对服务器发出HTTP请求,然后由服务器返回最新的数据给客户端的浏览器.这种简单粗暴模式有一个明显的缺点,就是浏览器需要不断的向服务器发出请求,H ...

  9. python飞机大战简单实现

    小游戏飞机大战的简单代码实现: # 定义敌机类 class Enemy: def restart(self): # 重置敌机的位置和速度 self.x = random.randint(50, 400 ...

  10. VSCode 配置C++开发环境

    目录 安装VSCode应用程序 安装相关插件 汉化插件 C++编辑器插件 编写配置文件 tasks.json launch.json c_cpp_properties.json 第一步.安装VSCod ...