KafkaManager编译安装使用(支持kerberos认证)
为了能够方便的查看及管理Kafka集群,yahoo提供了一个基于Web的管理工具(Kafka-Manager)。
这个工具可以方便的查看集群中Kafka的Topic的状态(分区、副本及消息量等),支持管理多个集群、重新分配Partition及创建Topic等功能。
jdk、sbt自行安装吧
sbt源可参考:https://www.cnblogs.com/felixzh/p/10255145.html
Deployment
The command below will create a zip file which can be used to deploy the application.
./sbt clean dist
演示:
root@root:~/Downloads/kafka-manager-1.3.3.22# ./sbt clean dist
[info] Loading project definition from /root/Downloads/kafka-manager-1.3.3.22/project
[info] Updating {file:/root/Downloads/kafka-manager-1.3.3.22/project/}kafka-manager-1-3-3-22-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
Missing bintray credentials /root/.bintray/.credentials. Some bintray features depend on this.
[info] Set current project to kafka-manager (in build file:/root/Downloads/kafka-manager-1.3.3.22/)
Missing bintray credentials /root/.bintray/.credentials. Some bintray features depend on this.
[warn] Credentials file /root/.bintray/.credentials does not exist
[success] Total time: 0 s, completed Jan 11, 2019 2:26:32 PM
[warn] Credentials file /root/.bintray/.credentials does not exist
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/.ivy2/cache/org.slf4j/slf4j-nop/jars/slf4j-nop-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/.ivy2/cache/org.slf4j/slf4j-simple/jars/slf4j-simple-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
[info] Packaging /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/kafka-manager_2.11-1.3.3.22-sources.jar ...
[info] Done packaging.
[info] Updating {file:/root/Downloads/kafka-manager-1.3.3.22/}root...
[info] Resolving org.apache.curator#curator-framework;2.10.0 ...
Warning: node.js detection failed, sbt will use the Rhino based Trireme JavaScript engine instead to run JavaScript assets[info] Resolving jline#jline;2.12.1 ...
[info] Done updating.
[warn] Scala version was updated by one of library dependencies:
[warn] * org.scala-lang:scala-library:(2.11.6, 2.11.8, 2.11.11, 2.11.7, 2.11.5, 2.11.0) -> 2.11.12
[warn] To force scalaVersion, add the following:
[warn] ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }
[warn] There may be incompatibilities among your library dependencies.
[warn] Here are some of the libraries that were evicted:
[warn] * org.webjars:jquery:1.11.1 -> 2.1.4
[warn] Run 'evicted' to see detailed eviction warnings
[info] Wrote /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/kafka-manager_2.11-1.3.3.22.pom
[info] Main Scala API documentation to /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/api...
[info] Compiling 127 Scala sources and 2 Java sources to /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/classes...
[info] LESS compiling on 1 source(s)
[info] Packaging /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/kafka-manager_2.11-1.3.3.22-web-assets.jar ...
[info] Done packaging.
model contains 709 documentable templates
[info] Main Scala API documentation successful.
[info] Packaging /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/kafka-manager_2.11-1.3.3.22-javadoc.jar ...
[info] Done packaging.
[info] Packaging /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/kafka-manager_2.11-1.3.3.22.jar ...
[info] Done packaging.
[info] Packaging /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/kafka-manager_2.11-1.3.3.22-sans-externalized.jar ...
[info] Done packaging.
[info]
[info] Your package is ready in /root/Downloads/kafka-manager-1.3.3.22/target/universal/kafka-manager-1.3.3.22.zip
[info]
[success] Total time: 80 s, completed Jan 11, 2019 2:27:52 PM
root@root:~/Downloads/kafka-manager-1.3.3.22# cd /root/Downloads/kafka-manager-1.3.3.22/target/universal/
root@root:~/Downloads/kafka-manager-1.3.3.22/target/universal# ls
kafka-manager-1.3.3.22.zip scripts
Please refer to play framework documentation on production deployment/configuration.
配置
修改conf/application.conf文件中zk的地址以及启用Kafka-Manager使用账号登录和消费者的配置
修改conf/consumer.properties内容如下:
security.protocol=SASL_PLAINTEXT
key.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer
value.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka
因为做了kerberos认证,所以需要jaas.conf文件指定到启动命令后面。
配置前要保证票据可用!
klist -l
kinit -kt /etc/security/keytabs/kafka.service.keytab kafka/你的主机
Starting the service
After extracting the produced zipfile, and changing the working directory to it, you can run the service like this:
$ bin/kafka-manager
By default, it will choose port 9000. This is overridable, as is the location of the configuration file. For example:
$ bin/kafka-manager -Dconfig.file=/path/to/application.conf -Dhttp.port=8080
Again, if java is not in your path, or you need to run against a different version of java, add the -java-home option as follows:
$ bin/kafka-manager -java-home /usr/local/oracle-java-8
Starting the service with Security
To add JAAS configuration for SASL, add the config file location at start:
$ bin/kafka-manager -Djava.security.auth.login.config=/path/to/my-jaas.conf
如:bin/kafka-manager -Djava.security.auth.login.config=/etc/kafka/conf/kafka_jaas.conf
注意:
kafka-manager-1.3.3.22之前的版本启动会有问题,异常log如下:
java.util.NoSuchElementException: key not found: PLAINTEXT
详见github issue:https://github.com/yahoo/kafka-manager/issues/502
已经在https://github.com/yahoo/kafka-manager/pull/532修复
NOTE: Make sure the user running kafka manager has read permissions on the jaas config file
Packaging(可以打RPM包)
If you'd like to create a Debian or RPM package instead, you can run one of:
sbt debian:packageBin
sbt rpm:packageBin
Credits
Logo/favicon used is from Apache Kafka.
Most of the utils code has been adapted to work with Apache Curator from Apache Kafka.
License
Apache Licensed. See accompanying LICENSE file.
https://github.com/yahoo/kafka-manager
https://blog.csdn.net/weixin_35852328/article/details/83656002
KafkaManager编译安装使用(支持kerberos认证)的更多相关文章
- 1-18 编译安装内核支持ntfs文件系统
大纲: 源码编译Linux内核 使用Linux内核模块 实战:编译一个NTFS内核模块,实现Linux挂载NTFS文件系统并实现读写功能 =============================== ...
- 重新编译安装swoole支持OpenSSL
1.下载:wget http://pecl.php.net/get/swoole-1.9.22.tgz 2.解压:tar zxvf swoole-1.9.22.tgz 3.扩展模块:cd swoole ...
- Kafka监控系统Kafka Eagle:支持kerberos认证
在线文档:https://ke.smartloli.org/ 作者博客:https://www.cnblogs.com/smartloli/p/9371904.html 源码地址:https://gi ...
- llvm+clang编译安装
最近一段时间在llvm+clang上做一些东西,所以顺便将自己如何编译安装llvm+clang写了篇文章发在这里,希望能帮助刚接触llvm+clang的童鞋少走一些弯路(我刚接触的时候为了编译安装这个 ...
- Nginx服务编译安装、日志功能、状态模块及访问认证模式实操
系统环境 [root@web ~]# cat /etc/redhat-release CentOS release 6.9 (Final) [root@web ~]# uname -a Linux d ...
- CentOS下PHP7的编译安装,MySQL的支持和一些问题的解决
最近试了一下PHP7,在编译和支持MySQL上都遇到一些问题,相信不少同学也同样遇到,所以在这里聊一下这个过程.简单来讲编译PHP7只需要3步: 1../buildconf --force 2../c ...
- nginx在centos 7中源码编译安装【添加grpc的支持】
安装依赖软件 1.安装编译工具gcc gcc是一个开源编译器集合,用于处理各种各样的语言:C.C++.Java.Ada等,在linux世界中是最通用的编译器,支持大量处理器:x86.AMD64.Pow ...
- 编译安装haproxy开启支持SSL
1.下载程序包 # wget http://www.haproxy.org/download/1.7/src/haproxy-1.7.5.tar.gz # tar xvf haproxy-1.7.5. ...
- centos6编译安装zabbix3.0和中文支持整理文档
编者按: 最近公司部分业务迁移机房,为了更方便的监控管理主机资源,决定上线zabbix监控平台.运维人员使用2.4版本的进行部署,个人在业余时间尝鲜,使用zabbix3.0进行部署,整理文档如下,仅供 ...
随机推荐
- .net core 2.0 Code First Fluent API配置
A.net core 2.0新特性支持通过IEntityTypeConfiguration<>添加Code First配置到一个封装类. 新建目标框架为.NET Core类库 新建完了以后 ...
- asp.net core系列 39 Web 应用Razor 介绍与详细示例
一. Razor介绍 在使用ASP.NET Core Web开发时, ASP.NET Core MVC 提供了一个新特性Razor. 这样开发Web包括了MVC框架和Razor框架.对于Razor来说 ...
- sql server 性能调优之 死锁排查
一.概述 记得以前客户在使用软件时,有偶发出现死锁问题,因为发生的时间不确定,不好做问题的重现,当时解决问题有点棘手了.现总结下查看死锁的常用二种方式. 1.1 第一种是图形化监听: sqlserve ...
- Lucene 06 - 使用Lucene的Query API查询数据
目录 1 Query对象的创建(方式一): 使用子类对象 1.1 常用的Query子类对象 1.2 常用的Query子类对象使用 1.2.1 使用TermQuery 1.2.2 使用NumericRa ...
- IntelliJ IDEA部署tomcat时Edit Configuration Deployment无artifact选项
IntelliJ IDEA,IntelliJ Idea创建web项目之后在配置web项目时,选择Edit Configration部署Tomcat,Deployment里点击添加无artifact选项 ...
- Kafka~Linux环境下的部署
概念 Kafka是一种高吞吐量的分布式发布订阅消息系统,它可以处理消费者规模的网站中的所有动作流数据. 这种动作(网页浏览,搜索和其他用户的行动)是在现代网络上的许多社会功能的一个关键因素. 这些数据 ...
- LeetCode专题-Python实现之第21题:Merge Two Sorted Lists
导航页-LeetCode专题-Python实现 相关代码已经上传到github:https://github.com/exploitht/leetcode-python 文中代码为了不动官网提供的初始 ...
- c#委托中的同步和异步方法即BeginInvoke和EndInvoke
学习多线程之前我们先了解一下电脑的一些概念,比如进程,线程,这个参考https://www.cnblogs.com/loverwangshan/p/10409755.html 这篇文章.今天我们接着来 ...
- Dapper批量操作实体
首先要安装 Dapper.Contrib,直接从nuget安装即可. dapper官方文档:http://dapper-tutorial.net/dapper T Get<T>(id); ...
- Vs2017 xaramin mac build agent部署后记
换了四种黑苹果,最终成功了 步骤: 1.升级vs2017, 2.安装XCODE 8.3 3.安装vs2017 for mac 企业版 4 ...