1 错误详细信息

WARN conf.FlumeConfiguration: Could not configure sink  k1 due to: Channel c1 not in active set.
org.apache.flume.conf.ConfigurationException: Channel c1 not in active set.
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:685)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:347)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.access$000(FlumeConfiguration.java:213)
at org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:127)
at org.apache.flume.conf.FlumeConfiguration.<init>(FlumeConfiguration.java:109)
at org.apache.flume.node.PropertiesFileConfigurationProvider.getFlumeConfiguration(PropertiesFileConfigurationProvider.java:189)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:89)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
19/02/21 03:50:54 WARN conf.FlumeConfiguration: Could not configure sink k2 due to: Channel k2 not in active set.
org.apache.flume.conf.ConfigurationException: Channel k2 not in active set.
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:685)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:347)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.access$000(FlumeConfiguration.java:213)
at org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:127)
at org.apache.flume.conf.FlumeConfiguration.<init>(FlumeConfiguration.java:109)
at org.apache.flume.node.PropertiesFileConfigurationProvider.getFlumeConfiguration(PropertiesFileConfigurationProvider.java:189)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:89)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

2 错误原因

  配置文件写错了

# set channel for sinks
a1.sinks.k1.channel = c1
a1.sinks.k2.channel = k2

  

  

3 正确应该是配置c2 也就是channel

# set channel for sinks
a1.sinks.k1.channel = c1
a1.sinks.k2.channel = c2

4 完整的配置文件

# List the sources, sinks and channels for the agent
# <Agent>.sources = <Source1>
# <Agent>.sinks = <Sink1> <Sink2>
# <Agent>.channels = <Channel1> <Channel2>
#
# # set list of channels for source (separated by space)
# <Agent>.sources.<Source1>.channels = <Channel1> <Channel2>
#
# # set channel for sinks
# <Agent>.sinks.<Sink1>.channel = <Channel1>
# <Agent>.sinks.<Sink2>.channel = <Channel2>
#
# <Agent>.sources.<Source1>.selector.type = replicating # Name the components on this agent
a1.sources = r1
a1.sinks = k1 k2
a1.channels = c1 c2 # Describe/configure the source
a1.sources.r1.type = exec
a1.sources.r1.command = tail -F /tmp/logs/cmcc.log # Describe the sink
a1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink
a1.sinks.k1.topic = cmcc1
a1.sinks.k1.brokerList = hdp1:9092,hdp2:9092,hdp3:9092
a1.sinks.k1.requiredAcks = 1
a1.sinks.k1.batchSize = 20 a1.sinks.k2.type = org.apache.flume.sink.kafka.KafkaSink
a1.sinks.k2.topic = cmcc2
a1.sinks.k2.brokerList = hdp1:9092,hdp2:9092,hdp3:9092
a1.sinks.k2.requiredAcks = 1
a1.sinks.k1.batchSize = 20 a1.sinks.k2.type = org.apache.flume.sink.kafka.KafkaSink
a1.sinks.k2.topic = cmcc2
a1.sinks.k2.brokerList = hdp1:9092,hdp2:9092,hdp3:9092
a1.sinks.k2.requiredAcks = 1
a1.sinks.k2.batchSize = 20 # Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100 a1.channels.c2.type = file
a1.channels.c2.checkpointDir = /tmp/flume/checkpoint
a1.channels.c2.dataDirs = /tmp/flume/data # Bind the source and sink to the channel
a1.sources.r1.channels = c1 c2 # set channel for sinks
a1.sinks.k1.channel = c1
a1.sinks.k2.channel = c2
# #
a1.sources.r1.selector.type = replicating
#

  

org.apache.flume.conf.ConfigurationException: Channel c1 not in active set.的更多相关文章

  1. WARN conf.FlumeConfiguration: Could not configure sink sink1 due to: No channel configured for sink: sink1 org.apache.flume.conf.ConfigurationException: No channel configured for sink: sink1

    1.错误如下所示,启动flume采集文件到hdfs案例的时候,出现如下所示的错误: 大概是说No channel configured for sink,所以应该是sink哪里配置出现了错误,百度了一 ...

  2. Apache Flume日志收集系统简介

    Apache Flume是一个分布式.可靠.可用的系统,用于从大量不同的源有效地收集.聚合.移动大量日志数据进行集中式数据存储. Flume简介 Flume的核心是Agent,Agent中包含Sour ...

  3. Flume配置Multiplexing Channel Selector

    1 官网内容 上面配置的是根据不同的heder当中state值走不同的channels,如果是CZ就走c1 如果是US就走c2 c3 其他默认走c4 2 我的详细配置信息 一个监听http端口 然后 ...

  4. Apache Flume 学习笔记

    # 从http://flume.apache.org/download.html 下载flume ############################################# # 概述: ...

  5. GoldenGate实时投递数据到大数据平台(3)- Apache Flume

    Apache Flume Flume NG是一个分布式.可靠.可用的系统,它能够将不同数据源的海量日志数据进行高效收集.聚合,最后存储到一个中心化数据存储系统中,方便进行数据分析.事实上flume也可 ...

  6. Apache Flume入门指南[翻译自官方文档]

    声明: 根据官方文档选择性的翻译了下,不对请指正 https://flume.apache.org/FlumeUserGuide.html

  7. 实时事件统计项目:优化flume:用file channel代替mem channel

    背景:利用kafka+flume+morphline+solr做实时统计. solr从12月23号开始一直没有数据.查看日志发现,因为有一个同事加了一条格式错误的埋点数据,导致大量error. 据推断 ...

  8. Apache Flume 1.7.0 发布,日志服务器

    Apache Flume 1.7.0 发布了,Flume 是一个分布式.可靠和高可用的服务,用于收集.聚合以及移动大量日志数据,使用一个简单灵活的架构,就流数据模型.这是一个可靠.容错的服务. 本次更 ...

  9. Apache Flume 简介

    转自:http://blog.163.com/guaiguai_family/blog/static/20078414520138100562883/ Flume 是 Cloudera 公司开源出来的 ...

随机推荐

  1. BZOJ2940 条纹

    条纹游戏是一个双人的游戏.所需要的物品有一个棋盘以及三种颜色的长方形条纹,这三种颜色分别是红色.绿色和蓝色.所有的红色条纹的尺寸是c*1,所有的绿色条纹的尺寸是z*1,所有的蓝色条纹的尺寸是n*1,这 ...

  2. IO 流读取文件时候出现乱码 文件编码格式问题 怎么转换解决方法

    在使用下面这个写法时候UTF-8文件编码 在读取时候出现乱码问题. File myFile=new File("文件路径"); BufferedReader in = new Bu ...

  3. ImageMagick: DrawImage(Image*,DrawInfo*) 绘制填充图片时卡住的原因分析

    今天傍晚在测试的时候无意发现有两个动画会卡住,正常情况下,20秒就完成的操作,突然卡住. CPU:95%+,经过30 - 50秒左右后,程序又能正常的向下执行,结果是对的. 这种情况不是每次都发生,有 ...

  4. Springboot+WebSocket+Kafka(写着玩的)

    闹着玩的来源:前台发送消息,后台接受处理发给kafka,kafka消费者接到消息传给前台显示.联想到websocket. 最终效果如图: 页面解释: 不填写内容的话,表单值默认为Topic.Greet ...

  5. 关于JAVA-JS-JSP之间传值的各种方法

    https://blog.csdn.net/murex_dustyone/article/details/52945552 这篇博文写的非常好,保存下来,一起学习

  6. jsp+servlet+poi导出数据库中的数据

    index.jsp: <a href="POIout">简单导出数据</a> <a href="POIoutTemplate"&g ...

  7. 【C#】使用bat文件安装卸载Window服务

    1.安装服务 @echo off @title 安装windows服务path %SystemRoot%\Microsoft.NET\Framework\v4.0.30319echo========= ...

  8. CodeFirst+MySql开发

    CodeFirst+MySql开发简单入门 记录一下使用Mysql进行EF Codefirst方式开发的简单过程. 0.准备工作 安装MySql,mysql-connector-net,mysql-f ...

  9. 如何比较一个类型【模板使用】【sizeof用法】

    #include <iostream> using namespace std; void testEmptyClass(); struct Empty { }; struct Dummy ...

  10. tomcat优化之安装并配置apr库

    在谈到tomcat优化时,必然要说到apr库,这个库是C语言实现的,tomcat通过JNI方式使用该库可以大大提高性能. tomcat在使用apr时需要安装apr,apr-util和tomcat-na ...