org.apache.flume.ChannelException: Take list for MemoryTransaction, capacity 100 full, consider committing more frequently, increasing capacity, or increasing thread count
flume在抽取MySQL数据到kafka时报错，如下
[SinkRunner-PollingRunner-DefaultSinkProcessor] ERROR org.apache.flume.sink.kafka.KafkaSink - Failed to publish events
org.apache.flume.ChannelException: Take list for MemoryTransaction, capacity full, consider committing more frequently, increasing capacity, or increasing thread count
at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doTake(MemoryChannel.java:)
at org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:)
at org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:)
at org.apache.flume.sink.kafka.KafkaSink.process(KafkaSink.java:)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:)
at java.lang.Thread.run(Thread.java:)
[SinkRunner-PollingRunner-DefaultSinkProcessor] ERROR org.apache.flume.SinkRunner - Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: Failed to publish events
at org.apache.flume.sink.kafka.KafkaSink.process(KafkaSink.java:)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:)
at java.lang.Thread.run(Thread.java:)
Caused by: org.apache.flume.ChannelException: Take list for MemoryTransaction, capacity full, consider committing more frequently, increasing capacity, or increasing thread count
at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doTake(MemoryChannel.java:)
at org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:)
at org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:)
at org.apache.flume.sink.kafka.KafkaSink.process(KafkaSink.java:)
... more
MemoryChannel的事务的Put队列满
解决:将channel的capacity的值调大即可
a1.channels.ch-.capacity =
注意：Flume配置每行后边不能加注释，否则会导致配置不生效
org.apache.flume.ChannelException: Take list for MemoryTransaction, capacity 100 full, consider committing more frequently, increasing capacity, or increasing thread count的更多相关文章
- Put queue for MemoryTransaction of capacity 10000 full, consider committing more frequently, increasing capacity or increasing thread count flume capacity 时间数
package com.test; import org.apache.http.*;import org.apache.http.entity.ContentType;import org.apac ...
- 一个Flume 异常（Put queue for MemoryTransaction of capacity 100 full）的排查和解决思路
最近在做一个分布式调用链跟踪系统, 在两个地方采用了flume (我使用的flume版本是1.5.0-cdh5.4.4),一个是宿主系统 ,用flume agent进行日志搜集. 一个是从kafka拉 ...
- Flume启动运行时报错org.apache.flume.ChannelFullException: Space for commit to queue couldn't be acquired. Sinks are likely not keeping up with sources, or the buffer size is too tight解决办法（图文详解）
前期博客 Flume自定义拦截器(Interceptors)或自带拦截器时的一些经验技巧总结(图文详解) 问题详情 启动agent服务 [hadoop@master flume-1.7.0]$ ...
- Apache Flume 学习笔记
# 从http://flume.apache.org/download.html 下载flume ############################################# # 概述: ...
- Apache Flume 1.7.0 发布，日志服务器
Apache Flume 1.7.0 发布了,Flume 是一个分布式.可靠和高可用的服务,用于收集.聚合以及移动大量日志数据,使用一个简单灵活的架构,就流数据模型.这是一个可靠.容错的服务. 本次更 ...
- org.apache.flume.FlumeException: NettyAvroRpcClient { host: xxx.xxx.xxx.xxx, port: 41100 }: RPC
2014-12-19 01:05:42,141 (lifecycleSupervisor-1-1) [WARN - org.apache.flume.sink.AbstractRpcSink.star ...
- Apache Flume日志收集系统简介
Apache Flume是一个分布式.可靠.可用的系统,用于从大量不同的源有效地收集.聚合.移动大量日志数据进行集中式数据存储. Flume简介 Flume的核心是Agent,Agent中包含Sour ...
- Apache Flume 1.7.0 各个模块简介
Flume简介 Apache Flume是一个分布式.可靠.高可用的日志收集系统,支持各种各样的数据来源,如http,log文件,jms,监听端口数据等等,能将这些数据源的海量日志数据进行高效收集.聚 ...
- Flafka: Apache Flume Meets Apache Kafka for Event Processing
The new integration between Flume and Kafka offers sub-second-latency event processing without the n ...
随机推荐
- 事务处理笔记《一》ADO.NET级别的事务
现在我们对事务的概念和原理都有所了解了,并且作为已经有一些基础的C#开发者,我们已经熟知编写数据库交互程序的一些要点,即: (1)使用SqlConnection类的对象的Open()方法建立与数据库服 ...
- 【BZOJ3277/3473】串/字符串 后缀数组+二分+RMQ+双指针
[BZOJ3277]串 Description 字符串是oi界常考的问题.现在给定你n个字符串,询问每个字符串有多少子串(不包括空串)是所有n个字符串中至少k个字符串的子串(注意包括本身). Inpu ...
- Error: member names cannot be the same as their enclosing type
在编译的时候会遇到如下问题:member names cannot be the same as their enclosing type 原因:方法名和类名不能一样,如果一样就是一个构造函数.而构造 ...
- Magic Cast Method in Java Magic Trick In Java
https://www.atlassian.com/blog/archives/magic_trick_in_java https://www.gamlor.info/wordpress/2010/1 ...
- python函数回顾：abs()
函数:abs() 官方英文文档解释 abs(x) Return the absolute value of a number. The argument may be a plain or long ...
- 理解是最好的记忆方法 之 CSS中a链接的4个伪类为何有顺序
在CSS中,a标签有4种伪类,分别为: a:link, a:visited, a:hover, a:active 对其稍有了解的前端er都知道,4个伪类是有固定顺序的(LVHA),否则很容易出现预期之 ...
- xlwt 模块 操作excel
1.xlwt 基本用法 import xlwt #1 新建文件 new_file = open('test.xls', 'w') new_file.close() #2 创建工作簿 wookbook ...
- 2、Python request、BeautifulSoup（download mm_pic）
import requests from bs4 import BeautifulSoup import os class DownLoadImg(object): def __init__(self ...
- Android 4.4 Kitkat 音频实现及简要分析
版权声明:本文为博主原创文章,未经博主同意不得转载. https://blog.csdn.net/jingxia2008/article/details/26701899 在 Android 4.4 ...
- spring项目报org.apache.tiles.definition.DefinitionsFactoryException: I/O错误原因及解决办法。
今天升级一个spring项目遇到如下错: HTTP Status 500 - Request processing failed; nested exception is org.apache.til ...