KafkaManager编译安装使用（支持kerberos认证）

为了能够方便的查看及管理Kafka集群，yahoo提供了一个基于Web的管理工具（Kafka-Manager）。

这个工具可以方便的查看集群中Kafka的Topic的状态（分区、副本及消息量等），支持管理多个集群、重新分配Partition及创建Topic等功能。

jdk、sbt自行安装吧

sbt源可参考：https://www.cnblogs.com/felixzh/p/10255145.html

Deployment

The command below will create a zip file which can be used to deploy the application.

./sbt clean dist

演示：

root@root:~/Downloads/kafka-manager-1.3.3.22# ./sbt clean dist
[info] Loading project definition from /root/Downloads/kafka-manager-1.3.3.22/project
[info] Updating {file:/root/Downloads/kafka-manager-1.3.3.22/project/}kafka-manager-1-3-3-22-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
Missing bintray credentials /root/.bintray/.credentials. Some bintray features depend on this.
[info] Set current project to kafka-manager (in build file:/root/Downloads/kafka-manager-1.3.3.22/)
Missing bintray credentials /root/.bintray/.credentials. Some bintray features depend on this.
[warn] Credentials file /root/.bintray/.credentials does not exist
[success] Total time: 0 s, completed Jan 11, 2019 2:26:32 PM
[warn] Credentials file /root/.bintray/.credentials does not exist
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/.ivy2/cache/org.slf4j/slf4j-nop/jars/slf4j-nop-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/.ivy2/cache/org.slf4j/slf4j-simple/jars/slf4j-simple-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
[info] Packaging /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/kafka-manager_2.11-1.3.3.22-sources.jar ...
[info] Done packaging.
[info] Updating {file:/root/Downloads/kafka-manager-1.3.3.22/}root...
[info] Resolving org.apache.curator#curator-framework;2.10.0 ...
Warning: node.js detection failed, sbt will use the Rhino based Trireme JavaScript engine instead to run JavaScript assets[info] Resolving jline#jline;2.12.1 ...
[info] Done updating.
[warn] Scala version was updated by one of library dependencies:
[warn]     * org.scala-lang:scala-library:(2.11.6, 2.11.8, 2.11.11, 2.11.7, 2.11.5, 2.11.0) -> 2.11.12
[warn] To force scalaVersion, add the following:
[warn]     ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }
[warn] There may be incompatibilities among your library dependencies.
[warn] Here are some of the libraries that were evicted:
[warn]     * org.webjars:jquery:1.11.1 -> 2.1.4
[warn] Run 'evicted' to see detailed eviction warnings
[info] Wrote /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/kafka-manager_2.11-1.3.3.22.pom
[info] Main Scala API documentation to /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/api...
[info] Compiling 127 Scala sources and 2 Java sources to /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/classes...
[info] LESS compiling on 1 source(s)
[info] Packaging /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/kafka-manager_2.11-1.3.3.22-web-assets.jar ...
[info] Done packaging.
model contains 709 documentable templates
[info] Main Scala API documentation successful.
[info] Packaging /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/kafka-manager_2.11-1.3.3.22-javadoc.jar ...
[info] Done packaging.
[info] Packaging /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/kafka-manager_2.11-1.3.3.22.jar ...
[info] Done packaging.
[info] Packaging /root/Downloads/kafka-manager-1.3.3.22/target/scala-2.11/kafka-manager_2.11-1.3.3.22-sans-externalized.jar ...
[info] Done packaging.
[info]
[info] Your package is ready in /root/Downloads/kafka-manager-1.3.3.22/target/universal/kafka-manager-1.3.3.22.zip
[info]
[success] Total time: 80 s, completed Jan 11, 2019 2:27:52 PM
root@root:~/Downloads/kafka-manager-1.3.3.22# cd /root/Downloads/kafka-manager-1.3.3.22/target/universal/
root@root:~/Downloads/kafka-manager-1.3.3.22/target/universal# ls
kafka-manager-1.3.3.22.zip  scripts

Please refer to play framework documentation on production deployment/configuration.

配置

修改conf/application.conf文件中zk的地址以及启用Kafka-Manager使用账号登录和消费者的配置

修改conf/consumer.properties内容如下：

security.protocol=SASL_PLAINTEXT
key.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer
value.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka

因为做了kerberos认证，所以需要jaas.conf文件指定到启动命令后面。

配置前要保证票据可用！

klist -l

kinit -kt /etc/security/keytabs/kafka.service.keytab kafka/你的主机

Starting the service

After extracting the produced zipfile, and changing the working directory to it, you can run the service like this:

$ bin/kafka-manager

By default, it will choose port 9000. This is overridable, as is the location of the configuration file. For example:

$ bin/kafka-manager -Dconfig.file=/path/to/application.conf -Dhttp.port=8080

Again, if java is not in your path, or you need to run against a different version of java, add the -java-home option as follows:

$ bin/kafka-manager -java-home /usr/local/oracle-java-8

Starting the service with Security

To add JAAS configuration for SASL, add the config file location at start:

$ bin/kafka-manager -Djava.security.auth.login.config=/path/to/my-jaas.conf
如：bin/kafka-manager -Djava.security.auth.login.config=/etc/kafka/conf/kafka_jaas.conf

注意：
kafka-manager-1.3.3.22之前的版本启动会有问题，异常log如下：

java.util.NoSuchElementException: key not found: PLAINTEXT

详见github issue：https://github.com/yahoo/kafka-manager/issues/502
已经在https://github.com/yahoo/kafka-manager/pull/532修复

NOTE: Make sure the user running kafka manager has read permissions on the jaas config file

Packaging（可以打RPM包）

If you'd like to create a Debian or RPM package instead, you can run one of:

sbt debian:packageBin

sbt rpm:packageBin

Credits

Logo/favicon used is from Apache Kafka.

Most of the utils code has been adapted to work with Apache Curator from Apache Kafka.

License

Apache Licensed. See accompanying LICENSE file.

https://github.com/yahoo/kafka-manager

https://blog.csdn.net/weixin_35852328/article/details/83656002