Spark Streaming从flume 中使用Poll拉取数据时,报如下错误:

Error while processing transaction.
java.lang.IllegalStateException: begin() called when transaction is OPEN!
  at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
  at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor$$anonfun$populateEvents$1.apply(TransactionProcessor.scala:114)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor$$anonfun$populateEvents$1.apply(TransactionProcessor.scala:113)
  at scala.Option.foreach(Option.scala:236)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.populateEvents(TransactionProcessor.scala:113)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:243)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:43)
  at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  at java.lang.Thread.run(Thread.java:745)
2020-04-08 05:20:15,120 (Spark Sink Processor Thread - 10) [WARN - org.apache.spark.streaming.flume.sink.Logging$class.logWarning(Logging.scala:59)] Spark was unable to successfully process the events. Transaction is being rolled back.
2020-04-08 05:20:15,120 (New I/O worker #1) [WARN - org.apache.spark.streaming.flume.sink.Logging$class.logWarning(Logging.scala:59)] Received an error batch - no events were received from channel!

报错原因:  

  flume的lib目录下的scala-library.xx jar 版本有问题。

解决办法:

1、将flume的lib目录下的 scala-library-2.10.5.jar 删除或者重命名。

2、建议上传 scala-library-2.11.x.jar 包。

如下图:

3、附上 scala-library-2.11.8.jar ,需要请自提。

  [百度网盘:]
  链接:https://pan.baidu.com/s/1GsNwQyt89pobWHWkZO52jw

  提取码:41ik

Error while processing transaction.java.lang.IllegalStateException: begin() called when transaction is OPEN!的更多相关文章

  1. flume-sink报错 java.lang.IllegalStateException: close() called when transaction is OPEN - you must either commit or rollback first

    1. 确认代码无误(根据情况修改,表示若获得不了数据不会自动commit或者rollback): Event event = channel.take(); if (event == null) { ...

  2. java.lang.IllegalStateException: Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

    java.lang.IllegalStateException: Not allowed to create transaction on sharedEntityManager - use Spri ...

  3. [spring cloud] [error] java.lang.IllegalStateException: Only one connection receive subscriber allowed.

    前言 最近在开发api-gateway的时候遇到了一个问题,网上能够找到的解决方案也很少,之后由公司的大佬解决了这个问题.写下这篇文章记录一下解决方案.希望可以帮助到更多的人. 环境 java版本:8 ...

  4. java.lang.IllegalStateException: Active Spring transaction synchronization or active JTA transaction with specified [javax.transaction.TransactionManager] required

    错误信息: java.lang.IllegalStateException: Active Spring transaction synchronization or active JTA trans ...

  5. Error:Execution failed for task ':app:compileDebugAidl'. > java.lang.IllegalStateException: aidl is missing from '/Users/renguodong/Library/Android/sdk/build-tools/26.0.2/aidl'

    错误信息:Error:Execution failed for task ':app:compileDebugAidl'. > java.lang.IllegalStateException: ...

  6. cause: java.lang.IllegalStateException: Serialized class com.taotao.pojo.TbItem must implement java.io.Serializable

    HTTP Status 500 - Request processing failed; nested exception is com.alibaba.dubbo.rpc.RpcException: ...

  7. nested exception is java.lang.IllegalStateException: Cannot forward after response has been committed

    org.springframework.web.util.NestedServletException: Request processing failed; nested exception is ...

  8. myeclipse 无法启动 java.lang.IllegalStateException: Unable to acquire application service. Ensure that the org.eclipse.core.runtime bundle is resolved and started (see config.ini).

    把myeclipse10 按照目录完整拷贝到了另外一台电脑, 另外的目录 原安装目录 D\:\soft\i\myeclipse10 新安装目录 E\:\soft\myeclipse10 双击启动失败, ...

  9. java.lang.IllegalStateException: getOutputStream() has already been called for this response

    ERROR [Engine] StandardWrapperValve[jsp]: Servlet.service() for servlet jsp threw exceptionjava.lang ...

随机推荐

  1. 峰哥说技术:10-Spring Boot静态资源处理

    Spring Boot深度课程系列 峰哥说技术—2020庚子年重磅推出.战胜病毒.我们在行动 10  峰哥说技术:Spring Boot静态资源处理 今天我们聊聊关于 Spring Boot 中关于静 ...

  2. JAVA 16bit CRC_CCITT

    JAVA 16bit CRC_CCITT public class CRC_CCITT { static int CRC16_ccitt_table[] = { 0x0000, 0x1189, 0x2 ...

  3. Node的require和module.exports

    node编程中最重要的思想之一就是模块,在 Node.js 模块系统中,每个文件都被视为独立的模块.这是这个思想,让javascript的大规模工程成为可能.模块化编程在前端大肆盛行,在node中导出 ...

  4. openwrt 为软件包添加服务

    手动修改 rc.local 加入也可以实现自启动,缺点手动修改太麻烦,停止只能用 kill . 配置成服务最方便了,可以启用或禁用,启动,停止,重启非常方便. 在openwrt 中使用服务 servi ...

  5. 深度学习与人类语言处理-语音识别(part1)

    语音识别 语音识别该何去何从? 1969年,J.R. PIERCE:"语音识别就像把水变成汽油.从大海中淘金.治疗癌症.人类登陆月球" 当然,这是50年前的想法,那么语音识别该如何 ...

  6. Java集合01——List 的几个实现类,了解一下?

    从本文起,我们将开始分享 Java 集合方面的知识,关注公众号「Java面典」了解更多 Java 知识点. List 是继承于 Collection 的接口,其实现类有 ArrayList,Linke ...

  7. go中处理各种请求方式以及处理接口请求参数

    话不多说直接上代码,解读内容全部在代码中 1.处理请求方式 package main import ( "fmt" "io/ioutil" "net/ ...

  8. .NET写入文件操作

    2018-01-16  22:44:35 许多程序需要记录运行日志,这就需要将程序运行记录写入本机,一般是.txt 文本或.csv 文件.具体操作如下: 一.C#   using System.IO; ...

  9. Scikit-learn之特征抽取

    一.安装包 pip install Scikit-learn 二.字典特征抽取 1.字典特征抽取 #!/usr/bin/env python # -*- coding: utf-8 -*- #auth ...

  10. 如何理解SiamRPN++?

    如何理解SiamRPN++? 目标跟踪: 使用视频序列第一帧的图像(包括bounding box的位置),来找出目标出现在后序帧位置的一种方法. 孪生网络结构: 在进入到正式理解SiamRPN++之前 ...