1 错误详细信息

WARN conf.FlumeConfiguration: Could not configure sink  k1 due to: Channel c1 not in active set.
org.apache.flume.conf.ConfigurationException: Channel c1 not in active set.
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:685)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:347)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.access$000(FlumeConfiguration.java:213)
at org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:127)
at org.apache.flume.conf.FlumeConfiguration.<init>(FlumeConfiguration.java:109)
at org.apache.flume.node.PropertiesFileConfigurationProvider.getFlumeConfiguration(PropertiesFileConfigurationProvider.java:189)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:89)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
19/02/21 03:50:54 WARN conf.FlumeConfiguration: Could not configure sink k2 due to: Channel k2 not in active set.
org.apache.flume.conf.ConfigurationException: Channel k2 not in active set.
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:685)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:347)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.access$000(FlumeConfiguration.java:213)
at org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:127)
at org.apache.flume.conf.FlumeConfiguration.<init>(FlumeConfiguration.java:109)
at org.apache.flume.node.PropertiesFileConfigurationProvider.getFlumeConfiguration(PropertiesFileConfigurationProvider.java:189)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:89)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

2 错误原因

  配置文件写错了

# set channel for sinks
a1.sinks.k1.channel = c1
a1.sinks.k2.channel = k2

  

  

3 正确应该是配置c2 也就是channel

# set channel for sinks
a1.sinks.k1.channel = c1
a1.sinks.k2.channel = c2

4 完整的配置文件

# List the sources, sinks and channels for the agent
# <Agent>.sources = <Source1>
# <Agent>.sinks = <Sink1> <Sink2>
# <Agent>.channels = <Channel1> <Channel2>
#
# # set list of channels for source (separated by space)
# <Agent>.sources.<Source1>.channels = <Channel1> <Channel2>
#
# # set channel for sinks
# <Agent>.sinks.<Sink1>.channel = <Channel1>
# <Agent>.sinks.<Sink2>.channel = <Channel2>
#
# <Agent>.sources.<Source1>.selector.type = replicating # Name the components on this agent
a1.sources = r1
a1.sinks = k1 k2
a1.channels = c1 c2 # Describe/configure the source
a1.sources.r1.type = exec
a1.sources.r1.command = tail -F /tmp/logs/cmcc.log # Describe the sink
a1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink
a1.sinks.k1.topic = cmcc1
a1.sinks.k1.brokerList = hdp1:9092,hdp2:9092,hdp3:9092
a1.sinks.k1.requiredAcks = 1
a1.sinks.k1.batchSize = 20 a1.sinks.k2.type = org.apache.flume.sink.kafka.KafkaSink
a1.sinks.k2.topic = cmcc2
a1.sinks.k2.brokerList = hdp1:9092,hdp2:9092,hdp3:9092
a1.sinks.k2.requiredAcks = 1
a1.sinks.k1.batchSize = 20 a1.sinks.k2.type = org.apache.flume.sink.kafka.KafkaSink
a1.sinks.k2.topic = cmcc2
a1.sinks.k2.brokerList = hdp1:9092,hdp2:9092,hdp3:9092
a1.sinks.k2.requiredAcks = 1
a1.sinks.k2.batchSize = 20 # Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100 a1.channels.c2.type = file
a1.channels.c2.checkpointDir = /tmp/flume/checkpoint
a1.channels.c2.dataDirs = /tmp/flume/data # Bind the source and sink to the channel
a1.sources.r1.channels = c1 c2 # set channel for sinks
a1.sinks.k1.channel = c1
a1.sinks.k2.channel = c2
# #
a1.sources.r1.selector.type = replicating
#

  

org.apache.flume.conf.ConfigurationException: Channel c1 not in active set.的更多相关文章

  1. WARN conf.FlumeConfiguration: Could not configure sink sink1 due to: No channel configured for sink: sink1 org.apache.flume.conf.ConfigurationException: No channel configured for sink: sink1

    1.错误如下所示,启动flume采集文件到hdfs案例的时候,出现如下所示的错误: 大概是说No channel configured for sink,所以应该是sink哪里配置出现了错误,百度了一 ...

  2. Apache Flume日志收集系统简介

    Apache Flume是一个分布式.可靠.可用的系统,用于从大量不同的源有效地收集.聚合.移动大量日志数据进行集中式数据存储. Flume简介 Flume的核心是Agent,Agent中包含Sour ...

  3. Flume配置Multiplexing Channel Selector

    1 官网内容 上面配置的是根据不同的heder当中state值走不同的channels,如果是CZ就走c1 如果是US就走c2 c3 其他默认走c4 2 我的详细配置信息 一个监听http端口 然后 ...

  4. Apache Flume 学习笔记

    # 从http://flume.apache.org/download.html 下载flume ############################################# # 概述: ...

  5. GoldenGate实时投递数据到大数据平台(3)- Apache Flume

    Apache Flume Flume NG是一个分布式.可靠.可用的系统,它能够将不同数据源的海量日志数据进行高效收集.聚合,最后存储到一个中心化数据存储系统中,方便进行数据分析.事实上flume也可 ...

  6. Apache Flume入门指南[翻译自官方文档]

    声明: 根据官方文档选择性的翻译了下,不对请指正 https://flume.apache.org/FlumeUserGuide.html

  7. 实时事件统计项目:优化flume:用file channel代替mem channel

    背景:利用kafka+flume+morphline+solr做实时统计. solr从12月23号开始一直没有数据.查看日志发现,因为有一个同事加了一条格式错误的埋点数据,导致大量error. 据推断 ...

  8. Apache Flume 1.7.0 发布,日志服务器

    Apache Flume 1.7.0 发布了,Flume 是一个分布式.可靠和高可用的服务,用于收集.聚合以及移动大量日志数据,使用一个简单灵活的架构,就流数据模型.这是一个可靠.容错的服务. 本次更 ...

  9. Apache Flume 简介

    转自:http://blog.163.com/guaiguai_family/blog/static/20078414520138100562883/ Flume 是 Cloudera 公司开源出来的 ...

随机推荐

  1. PKUWC2019滚粗记

    真的不想说些什么了,还是自己太菜了,水平低啊.深深感受到NOIP2017以后的那种绝望,这最好的签约清北的机会,就这样白白错过了.本以为自己的应该有NOIAg的实力,运气好是前120吧,没想到自己还是 ...

  2. [NOIP2018]旅行(数据加强版)(图论+基环树)

    数据范围多了2个0就是不一样,O(n^2)只能68分了.(其中60分是n=m+1和原题一样的做法送的),这题直接从NOIP难度变为NOI Plus难度了不说废话直接写题解:首先dfs一遍找到环,然后和 ...

  3. 第五篇 - Selenium突破反爬获取qq邮件标题

    from selenium import webdriver from selenium.webdriver import ActionChains #1.打开登陆页面 wd = webdriver. ...

  4. 文献笔记:Genome-wide associations for birth weight and correlations with adult disease

    该文献纳入了EGG(Early Growth Genetics Consortium)和UK biobank两大数据库,分为欧洲祖先和非欧洲祖先群体.这两个数据用到的样本量分别如下: Early Gr ...

  5. postman 介绍

  6. POJ 2728 Desert King (01分数规划)

    Desert King Time Limit: 3000MS   Memory Limit: 65536K Total Submissions:29775   Accepted: 8192 Descr ...

  7. Luogu P3254 圆桌问题

    题目链接 \(Click\) \(Here\) 水题.记得记一下边的流量有没有跑完. #include <bits/stdc++.h> using namespace std; const ...

  8. 基本类型(2):oracle中有4个大对象(lobs)类型可用,分别是blob,clob,bfile,nclob。

    1)blob:二进制lob,为二进制数据,最长可达4GB,存贮在数据库中. 2)clob:字符lob,字符数据,最长可以达到4GB,存贮在数据库中. 3)bfile:二进制文件;存贮在数据库之外的只读 ...

  9. Windows Dll Injection、Process Injection、API Hook、DLL后门/恶意程序入侵技术

    catalogue 1. 引言2. 使用注册表注入DLL3. 使用Windows挂钩来注入DLL4. 使用远程线程来注入DLL5. 使用木马DLL来注入DLL6. 把DLL作为调试器来注入7. 使用c ...

  10. 面向对象【day08】:类的起源与metaclass(二)

    本节内容 1.概述 2.类的起源 3.__new__方法 4.__metaclass__方法 一.概述 前面我们学习了大篇幅的关于类,通过类创建对象,那我们想知道这个类到底是怎么产生的呢?它的一切来源 ...