java.lang.IllegalStateException: File name has been re-used with different files. (flume报错)
报错日志:
java.lang.IllegalStateException: File name has been re-used with different files. Spooling assumptions violated for /usr/local/data/flumedata/words.txt.COMPLETED
at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.rollCurrentFile(ReliableSpoolingFileEventReader.java:378)
at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.retireCurrentFile(ReliableSpoolingFileEventReader.java:330)
at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:259)
at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:228)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
原因:
重复使用了同一个数据文件
解决方案1:
更改flume的配置文件conf的目录
a1.sources.r1.spoolDir = /usr/local/data/flumedata
#push mode
a1.sources = r1
a1.sinks = k1
a1.channels = c1
#source
a1.sources.r1.channels = c1
a1.sources.r1.type = spooldir
a1.sources.r1.spoolDir = /usr/local/data/flumedata
a1.sources.r1.fileHeader = true
#channel
a1.channels.c1.type =memory
a1.channels.c1.capacity = 20000
a1.channels.c1.transactionCapacity=5000
#sinks
a1.sinks.k1.channel = c1
a1.sinks.k1.type = avro
a1.sinks.k1.hostname=192.168.13.24
a1.sinks.k1.port = 9999
a1.sinks.k1.batchSize= 2000
解决方案2:
在flume配置文件的目录下重新定义数据文件
java.lang.IllegalStateException: File name has been re-used with different files. (flume报错)的更多相关文章
- 云笔记项目- 上传文件报错"java.lang.IllegalStateException: File has been moved - cannot be read again"
在做文件上传时,当写入上传的文件到文件时,会报错“java.lang.IllegalStateException: File has been moved - cannot be read again ...
- Springboot 上传报错: Failed to parse multipart servlet request; nested exception is java.lang.IllegalStateException: The multi-part request contained parameter data (excluding uploaded files) that exceede
Failed to parse multipart servlet request; nested exception is java.lang.IllegalStateException: The ...
- Eclipse------使用Debug As时报错java.lang.IllegalStateException: Failed to read Class-Path attribute from manifest of jar file:/XXX
报错信息: java.lang.IllegalStateException: Failed to read Class-Path attribute from manifest of jar file ...
- java.lang.IllegalStateException: Zip File is closed
最近在研究利用sax读取excel大文件时,出现了以下的错误: java.lang.IllegalStateException: Zip File is closed at org.apache.po ...
- myeclipse 无法启动 java.lang.IllegalStateException: Unable to acquire application service. Ensure that the org.eclipse.core.runtime bundle is resolved and started (see config.ini).
把myeclipse10 按照目录完整拷贝到了另外一台电脑, 另外的目录 原安装目录 D\:\soft\i\myeclipse10 新安装目录 E\:\soft\myeclipse10 双击启动失败, ...
- java.lang.IllegalStateException:Web app root system property already set to different value 错误原因及解决 Log4j
Log4j是Apache的一个开放源代码项目,通过使用Log4j,我们可以控制日志信息输送的目的地是控制台.文件.GUI组件.甚至是套接口 服务器.NT的事件记录器.UNIX Syslog守护进程等: ...
- struts2 下载时报java.lang.IllegalStateException
-----------------------------------------struts2 下载时报java.lang.IllegalStateException---------------- ...
- 处理eclipse启动时报java.lang.IllegalStateException
这是我写的第一篇博客,博客我来了: 我是好学的人,希望在这上面遇到志同道合的人,对技术有更高追求的人: 重启eclipse的时候报出来 An error has occurred, See the l ...
- 解决java.lang.IllegalStateException: BeanFactory not initialized or already closed - call 'refresh' before accessing beans via the ApplicationContext这个问题
今天在运行别人的SSH项目时,遇到了这个问题 严重: Exception sending context initialized event to listener instance of class ...
- Spring Scheduled定时任务报错 java.lang.IllegalStateException: Encountered invalid @Scheduled method 'xxx': For input string: "2S"
报错信息如下: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ding ...
随机推荐
- 【Python】【Pandas】使用concat添加行
添加行 t = pd.DataFrame(columns=["姓名","平均分"]) t = t.append({"姓名":"小红 ...
- 使用JSch进行sftp的连接运行状况检查
public boolean checkConnection() throws JSchException { try { JSch jsch = new JSch(); Session sessio ...
- Qt音视频开发25-ffmpeg音量设置
一.前言 音视频的播放.关闭.暂停.继续这几个基本功能,绝大部分人都是信手拈来的搞定,关于音量调节还是稍微饶了下弯弯,最开始打算采用各个系统的api来处理,坐下来发现不大好,系统的支持不完美,比如有些 ...
- 混合云网络过于复杂?ENS给你全局一张网的极致体验
本文分享自华为云社区<[华为云Stack][大架光临]第19期:混合云网络过于复杂?ENS给你全局一张网的极致体验>,作者:华为云Stack ENS研发团队. 政企IT的混合形态 经过几十 ...
- [转]OpenCV学习笔记(十五)——摄像机的标定和3D重建calib3D
OpenCV学习笔记(十五)--摄像机的标定和3D重建calib3D OpenCV学习笔记(16)双目测距与三维重建的OpenCV实现问题集锦(一)图像获取与单目定标 翻译 搜索 复制
- 提供Chrome浏览器插件网站
提供Chrome浏览器插件网站: 1.收藏猫插件 2.极简插件
- openEuler RISC-V上磁盘I/O性能测试的IOZone实例
IOZone 是一个广泛使用的文件系统性能基准测试工具,旨在评估磁盘 I/O 性能.它能够测试各种类型的文件操作,包括顺序读写.随机读写.重载(re-write).读取已写入的数据等.通过这些测试 ...
- DVWA靶场JavaScript Attacks漏洞low(低),medium(中等),high(高),impossible(不可能的)所有级别通关教程
JavaScript Attacks (前端攻击) JavaScript Attacks(前端攻击)漏洞通常涉及利用Web应用程序中的安全漏洞,特别是在JavaScript代码的使用和实现上,从而对用 ...
- Linux 压缩命令集合
01. tar格式 解包:tar xvf FileName.tar 打包:tar cvf FileName.tar DirName(注:tar是打包,不是压缩!) 02. gz格式 解压1:gunzi ...
- win10 bat文件命令大全-快捷键
具体介绍: 1. echo 和 @回显命令 @ #关闭单行回显 echo off #从下一行开始关闭回显 @echo off ...