When I try to compile I get an error about a missing SNAPSHOT dependency.

The error looks something like this:

Failed to execute goal on project kafka-connect-hdfs: Could not resolve dependencies for project
io.confluent:kafka-connect-hdfs:jar:3.1.-SNAPSHOT: Failure to find
io.confluent:kafka-connect-avro-converter:jar:3.1.-SNAPSHOT in http://packages.confluent.io/maven/ was cached in the local
repository, resolution will not be reattempted until the update interval of confluent has elapsed or updates are forced

The HDFS connector uses a few dependencies which we sometimes use SNAPSHOT versions of during development of a new release in order to build and test against new features. If you want to build a development version, you may need to build and install these dependencies to your local Maven repository in order to build the connector:

Project Clone Build with
Kafka https://github.com/apache/kafka/ ./gradlew installAll
Confluent Common library https://github.com/confluentinc/common mvn clean install
Confluent Rest-Utils https://github.com/confluentinc/rest-utils mvn clean install
Avro Converter https://github.com/confluentinc/schema-registry mvn clean install
Common library for Storage Connectors https://github.com/confluentinc/kafka-connect-storage-common mvn clean install
 

building confluentinc kafka-connect-hdfs的更多相关文章

  1. Kafka Connect HDFS

    概述 Kafka 的数据如何传输到HDFS?如果仔细思考,会发现这个问题并不简单. 不妨先想一下这两个问题? 1)为什么要将Kafka的数据传输到HDFS上? 2)为什么不直接写HDFS而要通过Kaf ...

  2. 使用kafka connect,将数据批量写到hdfs完整过程

    版权声明:本文为博主原创文章,未经博主允许不得转载 本文是基于hadoop 2.7.1,以及kafka 0.11.0.0.kafka-connect是以单节点模式运行,即standalone. 首先, ...

  3. Kafka到Hdfs的数据Pipeline整理

    作者:Syn良子 出处:http://www.cnblogs.com/cssdongl 转载请注明出处 找时间总结整理了下数据从Kafka到Hdfs的一些pipeline,如下 1> Kafka ...

  4. Streaming data from Oracle using Oracle GoldenGate and Kafka Connect

    This is a guest blog from Robin Moffatt. Robin Moffatt is Head of R&D (Europe) at Rittman Mead, ...

  5. Kafka connect in practice(3): distributed mode mysql binlog ->kafka->hive

    In the previous post Kafka connect in practice(1): standalone, I have introduced about the basics of ...

  6. Kafka Connect Architecture

    Kafka Connect's goal of copying data between systems has been tackled by a variety of frameworks, ma ...

  7. Build an ETL Pipeline With Kafka Connect via JDBC Connectors

    This article is an in-depth tutorial for using Kafka to move data from PostgreSQL to Hadoop HDFS via ...

  8. Kafka+Storm+HDFS整合实践

    在基于Hadoop平台的很多应用场景中,我们需要对数据进行离线和实时分析,离线分析可以很容易地借助于Hive来实现统计分析,但是对于实时的需求Hive就不合适了.实时应用场景可以使用Storm,它是一 ...

  9. Kafka connect快速构建数据ETL通道

    摘要: 作者:Syn良子 出处:http://www.cnblogs.com/cssdongl 转载请注明出处 业余时间调研了一下Kafka connect的配置和使用,记录一些自己的理解和心得,欢迎 ...

  10. [转载] Kafka+Storm+HDFS整合实践

    转载自http://www.tuicool.com/articles/NzyqAn 在基于Hadoop平台的很多应用场景中,我们需要对数据进行离线和实时分析,离线分析可以很容易地借助于Hive来实现统 ...

随机推荐

  1. Mybatis3.1-[tp_34-35]-_映射文件_select_resultMap关联查询_collection定义关联集合封装规则_collection分步查询_延迟加载

    笔记要点出错分析与总结工程组织 1.定义接口 interface DepartmentMapper package com.dao; import com.bean.Department; publi ...

  2. Mybatis的一级缓存机制简介

    1.接口 public interface MemberMapperCache { public Members selectMembersById(Integer id); } 2.配置文件xml ...

  3. Spring第二次案例和AOP

    Spring加上AOP com.mapper.entity.UserInfo package com.Spring.entity; public class UserInfo { private In ...

  4. sql server newid() 的用法

    sql newid()随机函数   从A表随机取2条记录,用SELECT TOP 10 * FROM ywle order by newid()order by 一般是根据某一字段排序,newid() ...

  5. tmux-cheatsheet

    http://tmuxcheatsheet.com/ http://louiszhai.github.io/2017/09/30/tmux/ 滚屏 开启 Ctrl-b pageup/pagedown ...

  6. selenium+chromeDriver配合使用(运行js脚本)

    在python中调用selenium,访问百度,并运行js脚本爬取内容 python入口程序 from selenium import webdriver import time with open( ...

  7. Fidller抓包分析post请求

    目的:抓包是为了最近做接口测试做准备,以前没有用过这个工具,最近来学下,但是网上很多文章了,所以不一一记录,有一部分参考即可 1.如何抓取想要的web端或者手机端包,已经有很多文章谢了,推荐的参考文章 ...

  8. 安装cartographer遇到Unrecognized syntax identifier "proto3". This parser only recognizes "proto2"问题

    https://stackoverflow.com/questions/38605734/mac-cannot-find-eigen3 https://blog.csdn.net/qq_4214518 ...

  9. windows下java环境变量标准配置

    配置步骤 1.“此电脑”右键,选择“属性”,点击“高级系统设置”,点击“环境变量”. 2.在“系统变量”这一栏,点击“新建”,变量名:JAVA_HOME,变量值:C:\Program Files\Ja ...

  10. CF786E ALT

    题意 有一棵 \(n\) 个点的树和 \(m\) 个人,第 \(i\) 个人从 \(u_i\) 走到 \(v_i\) 现在要发宠物,要求一个人要么他自己发到宠物,要么他走的路径上的都有宠物. 求最小代 ...