When I try to compile I get an error about a missing SNAPSHOT dependency.

The error looks something like this:

Failed to execute goal on project kafka-connect-hdfs: Could not resolve dependencies for project
io.confluent:kafka-connect-hdfs:jar:3.1.-SNAPSHOT: Failure to find
io.confluent:kafka-connect-avro-converter:jar:3.1.-SNAPSHOT in http://packages.confluent.io/maven/ was cached in the local
repository, resolution will not be reattempted until the update interval of confluent has elapsed or updates are forced

The HDFS connector uses a few dependencies which we sometimes use SNAPSHOT versions of during development of a new release in order to build and test against new features. If you want to build a development version, you may need to build and install these dependencies to your local Maven repository in order to build the connector:

Project Clone Build with
Kafka https://github.com/apache/kafka/ ./gradlew installAll
Confluent Common library https://github.com/confluentinc/common mvn clean install
Confluent Rest-Utils https://github.com/confluentinc/rest-utils mvn clean install
Avro Converter https://github.com/confluentinc/schema-registry mvn clean install
Common library for Storage Connectors https://github.com/confluentinc/kafka-connect-storage-common mvn clean install
 

building confluentinc kafka-connect-hdfs的更多相关文章

  1. Kafka Connect HDFS

    概述 Kafka 的数据如何传输到HDFS?如果仔细思考,会发现这个问题并不简单. 不妨先想一下这两个问题? 1)为什么要将Kafka的数据传输到HDFS上? 2)为什么不直接写HDFS而要通过Kaf ...

  2. 使用kafka connect,将数据批量写到hdfs完整过程

    版权声明:本文为博主原创文章,未经博主允许不得转载 本文是基于hadoop 2.7.1,以及kafka 0.11.0.0.kafka-connect是以单节点模式运行,即standalone. 首先, ...

  3. Kafka到Hdfs的数据Pipeline整理

    作者:Syn良子 出处:http://www.cnblogs.com/cssdongl 转载请注明出处 找时间总结整理了下数据从Kafka到Hdfs的一些pipeline,如下 1> Kafka ...

  4. Streaming data from Oracle using Oracle GoldenGate and Kafka Connect

    This is a guest blog from Robin Moffatt. Robin Moffatt is Head of R&D (Europe) at Rittman Mead, ...

  5. Kafka connect in practice(3): distributed mode mysql binlog ->kafka->hive

    In the previous post Kafka connect in practice(1): standalone, I have introduced about the basics of ...

  6. Kafka Connect Architecture

    Kafka Connect's goal of copying data between systems has been tackled by a variety of frameworks, ma ...

  7. Build an ETL Pipeline With Kafka Connect via JDBC Connectors

    This article is an in-depth tutorial for using Kafka to move data from PostgreSQL to Hadoop HDFS via ...

  8. Kafka+Storm+HDFS整合实践

    在基于Hadoop平台的很多应用场景中,我们需要对数据进行离线和实时分析,离线分析可以很容易地借助于Hive来实现统计分析,但是对于实时的需求Hive就不合适了.实时应用场景可以使用Storm,它是一 ...

  9. Kafka connect快速构建数据ETL通道

    摘要: 作者:Syn良子 出处:http://www.cnblogs.com/cssdongl 转载请注明出处 业余时间调研了一下Kafka connect的配置和使用,记录一些自己的理解和心得,欢迎 ...

  10. [转载] Kafka+Storm+HDFS整合实践

    转载自http://www.tuicool.com/articles/NzyqAn 在基于Hadoop平台的很多应用场景中,我们需要对数据进行离线和实时分析,离线分析可以很容易地借助于Hive来实现统 ...

随机推荐

  1. cookie、session与用户认证组件

    1.cookie def login(request): if request.method == "GET": return render(request,"login ...

  2. webpack 配置react脚手架(五):mobx

    1.  配置项.使用mobx,因为语法时es6-next,所以先配置 .babelrc 文件 { "presets": [ ["es2015", { " ...

  3. DTcmsV4.0分析学习——(2)系统框架

    2.1物理结构 共8个项目,Web为启动项 项目间相互依赖关系图: 2.2逻辑结构 DTcms V4.0轻量级CMS系统框架采用的是典型的三层架构(项目与三层架构并非直接对应关系,至于什么是三层架构这 ...

  4. Mybatis接口中传递多个参数

    1.接口 public interface MemberMapper { public boolean insertMember(Members member); public Members sel ...

  5. Makefile:248: /usr/local/otp_src_18.1/make/x86_64-unknown-linux-gnu/otp_ded.mk: No such file

    安装erlang的时候,使用make命令一直报这个错 Makefile:248: /usr/local/otp_src_18.1/make/x86_64-unknown-linux-gnu/otp_d ...

  6. window.external 是调用外部方法

    ie中,window.external 是调用外部方法,比如,是在 winform 中的 webbrower 中使用 window.external.SendData(),那么,SendData()  ...

  7. maven-setting.xml文件详解

    <?xml version="1.0" encoding="UTF-8"?> <settings xmlns="http://mav ...

  8. Vagrant Docker Composer Yarn 国外资源下载慢或失败的问题

    1 问题 有时,我们请求国外资源时,下载巨慢,甚至失败.如: cd vue-devtools/ $ yarn install 进行到 cypress.... 时,可能失败. 2 解决 次日凌晨(7-8 ...

  9. java.lang.OutOfMemoryError:PermGen space tomcat7 内存溢出

    Tomcat 解决 在启动项目的时候,会报java.lang.OutOfMemoryError:PermGen space错误. 在tomcat/bin/catalina.sh  windows li ...

  10. TCP数据报结构以及三次握手

    TCP(Transmission Control Protocol,传输控制协议)是一种面向连接的.可靠的.基于字节流的通信协议,数据在传输前要建立连接,传输完毕后还要断开连接. 客户端在收发数据前要 ...