java.lang.IllegalStateException: File name has been re-used with different files. (flume报错)
报错日志:
java.lang.IllegalStateException: File name has been re-used with different files. Spooling assumptions violated for /usr/local/data/flumedata/words.txt.COMPLETED
at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.rollCurrentFile(ReliableSpoolingFileEventReader.java:378)
at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.retireCurrentFile(ReliableSpoolingFileEventReader.java:330)
at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:259)
at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:228)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
原因:
重复使用了同一个数据文件
解决方案1:
更改flume的配置文件conf的目录
a1.sources.r1.spoolDir = /usr/local/data/flumedata
#push mode
a1.sources = r1
a1.sinks = k1
a1.channels = c1
#source
a1.sources.r1.channels = c1
a1.sources.r1.type = spooldir
a1.sources.r1.spoolDir = /usr/local/data/flumedata
a1.sources.r1.fileHeader = true
#channel
a1.channels.c1.type =memory
a1.channels.c1.capacity = 20000
a1.channels.c1.transactionCapacity=5000
#sinks
a1.sinks.k1.channel = c1
a1.sinks.k1.type = avro
a1.sinks.k1.hostname=192.168.13.24
a1.sinks.k1.port = 9999
a1.sinks.k1.batchSize= 2000
解决方案2:
在flume配置文件的目录下重新定义数据文件
java.lang.IllegalStateException: File name has been re-used with different files. (flume报错)的更多相关文章
- 云笔记项目- 上传文件报错"java.lang.IllegalStateException: File has been moved - cannot be read again"
在做文件上传时,当写入上传的文件到文件时,会报错“java.lang.IllegalStateException: File has been moved - cannot be read again ...
- Springboot 上传报错: Failed to parse multipart servlet request; nested exception is java.lang.IllegalStateException: The multi-part request contained parameter data (excluding uploaded files) that exceede
Failed to parse multipart servlet request; nested exception is java.lang.IllegalStateException: The ...
- Eclipse------使用Debug As时报错java.lang.IllegalStateException: Failed to read Class-Path attribute from manifest of jar file:/XXX
报错信息: java.lang.IllegalStateException: Failed to read Class-Path attribute from manifest of jar file ...
- java.lang.IllegalStateException: Zip File is closed
最近在研究利用sax读取excel大文件时,出现了以下的错误: java.lang.IllegalStateException: Zip File is closed at org.apache.po ...
- myeclipse 无法启动 java.lang.IllegalStateException: Unable to acquire application service. Ensure that the org.eclipse.core.runtime bundle is resolved and started (see config.ini).
把myeclipse10 按照目录完整拷贝到了另外一台电脑, 另外的目录 原安装目录 D\:\soft\i\myeclipse10 新安装目录 E\:\soft\myeclipse10 双击启动失败, ...
- java.lang.IllegalStateException:Web app root system property already set to different value 错误原因及解决 Log4j
Log4j是Apache的一个开放源代码项目,通过使用Log4j,我们可以控制日志信息输送的目的地是控制台.文件.GUI组件.甚至是套接口 服务器.NT的事件记录器.UNIX Syslog守护进程等: ...
- struts2 下载时报java.lang.IllegalStateException
-----------------------------------------struts2 下载时报java.lang.IllegalStateException---------------- ...
- 处理eclipse启动时报java.lang.IllegalStateException
这是我写的第一篇博客,博客我来了: 我是好学的人,希望在这上面遇到志同道合的人,对技术有更高追求的人: 重启eclipse的时候报出来 An error has occurred, See the l ...
- 解决java.lang.IllegalStateException: BeanFactory not initialized or already closed - call 'refresh' before accessing beans via the ApplicationContext这个问题
今天在运行别人的SSH项目时,遇到了这个问题 严重: Exception sending context initialized event to listener instance of class ...
- Spring Scheduled定时任务报错 java.lang.IllegalStateException: Encountered invalid @Scheduled method 'xxx': For input string: "2S"
报错信息如下: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ding ...
随机推荐
- ng-alain: Title Service
文档地址:https://ng-alain.com/theme/title/zh 源码地址: https://github.com/ng-alain/delon/blob/master/package ...
- 中电金信:GienTech动态|一波好消息→中标!多领域“开花”
- Java 和 native 的一些进展
GraalVM Java 生成 DLL/SO https://medium.com/graalvm/3-ways-to-polyglot-with-graalvm-fb28c1542b45 Proje ...
- UML之包与包图
了解UML的人都知道UML中也有包的概念,包在UML中作用与面向对象编程语言中类似,它是管理对象的工具,也是解决对象同名冲突的手段. 在UML中,包的表示图形是一个左上角带标签的矩形,而包名可以标注于 ...
- 即时通讯技术文集(第28期):IM开发技术合集(Part1) [共18篇]
为了更好地分类阅读 52im.net 总计1000多篇精编文章,我将在每周三推送新的一期技术文集,本次是第28 期. [- 1 -] 新手入门一篇就够:从零开发移动端IM [链接] http://ww ...
- IM通讯协议专题学习(七):手把手教你如何在NodeJS中从零使用Protobuf
1.前言 Protobuf是Google开源的一种混合语言数据标准,已被各种互联网项目大量使用. Protobuf最大的特点是数据格式拥有极高的压缩比,这在移动互联时代是极具价值的(因为移动网络流量到 ...
- KMS for Office 2021
I. 镜像下载 官方镜像下载地址: Office 2021 专业增强版: https://officecdn.microsoft.com/pr/492350f6-3a01-4f97-b9c0-c7c6 ...
- Docker 迁移数据目录
Centos7 环境,采用yum安装的,默认数据目录在/var/lib/docker中 1. 关闭docker服务 systemctl stop docker 2. 备份和迁移 # 迁移 cp -r ...
- Solution Set - “一二行诗句相遇,十万颗恒星解体”
目录 0.「集训队互测 2018」Fim4 1.「ABC 294Ex」K-Coloring 2.「NOI Simu.」解码 3.「NOI Simu.」图 4.「NOI Simu.」表达式 5.「ULR ...
- 基于C#实现多线程启动停止暂停继续
大家好!我是付工. 大部分初学者在学习C#上位机编程时,多线程是一个很难逾越的鸿沟,不合理地使用多线程,会导致经常出现各种奇怪的问题,这也是很多初学者不敢使用多线程的原因.但是在实际开发中,多线程是一 ...