Cloudera Hadoop启用Kerberos认证
一、Kerberos
二、安装
node01服务器安装Kerberos的核心服务master KDC，node02和node03安装Kerberos client
cm也安装在node01上了
1.master节点配置
在node01上
yum install krb5-server krb5-libs krb5-auth-dialog krb5-workstation
修改配置文件，/etc/krb5.conf
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log [libdefaults]
default_realm = LOCAL.DOMAIN
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true [realms]
LOCAL.DOMAIN = {
kdc = node01
admin_server = node01
} [domain_realm]
.local.domain = LOCAL.DOMAIN
local.domain = LOCAL.DOMAIN
修改配置文件，/var/kerberos/krb5kdc/kadm5.acl
*/admin@LOCAL.DOMAIN *
修改配置文件， /var/kerberos/krb5kdc/kdc.conf
把aes256-cts去掉,不去掉则要增加jar包
[kdcdefaults]
kdc_ports =
kdc_tcp_ports = [realms]
EXAMPLE.COM = {
#master_key_type = aes256-cts
acl_file = /var/kerberos/krb5kdc/kadm5.acl
dict_file = /usr/share/dict/words
admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
supported_enctypes = aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
}
2.创建/初始化Kerberos
1)创建/初始化Kerberos数据库，kdb5_util create -s –r LOCAL.DOMAIN ，并设置密码
[-s]表示生成stash file，并在其中存储master server key（krb5kdc）；
[-r]来指定一个realm name，当krb5.conf中定义了多个realm时才是必要的。
保存路径为/var/Kerberos/krb5kdc 如果需要重建数据库，将该目录下的含有principal的文件全都删除即可
[root@node01 ~]# kdb5_util create –r LOCAL.DOMAIN -s
Loading random data
Initializing database '/var/Kerberos/krb5kdc/principal' for realm 'LOCAL.DOMAIN',
master key name 'K/M@LOCAL.DOMAIN'
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key:
Re-enter KDC database master key to verify:
两次输入密码
2)创建Kerberos的管理账号，两次输入密码
[root@node01 ~]# kadmin.local
Authenticating as principal root/admin@LOCAL.DOMAIN with password.
kadmin.local: addprinc admin/admin@LOCAL.DOMAIN
WARNING: no policy specified for admin/admin@LOCAL.DOMAIN; defaulting to no policy
Enter password for principal "admin/admin@LOCAL.DOMAIN":
Re-enter password for principal "admin/admin@LOCAL.DOMAIN":
Principal "admin/admin@LOCAL.DOMAIN" created.
kadmin.local:
kadmin.local: exit
3.安装Kerberos客户端
1)给集群所有节点安装Kerberos客户端
node02和node03
[root@node02 ~]# yum -y install krb5-workstation krb5-libs krb5-auth-dialog Installed:
krb5-workstation.x86_64 :1.10.-.el6 Dependency Installed:
libkadm5.x86_64 :1.10.-.el6 Updated:
krb5-libs.x86_64 :1.10.-.el6 Dependency Updated:
krb5-devel.x86_64 :1.10.-.el6 Complete!
2)CM节点安装额外组件
root@node01 ~]# yum -y install openldap-clients Running Transaction Test
Transaction Test Succeeded
Running Transaction
Updating : openldap-2.4.-.el6.x86_64 /
Installing : openldap-clients-2.4.-.el6.x86_64 /
Cleanup : openldap-2.4.-.el6.x86_64 /
Verifying : openldap-clients-2.4.-.el6.x86_64 /
Verifying : openldap-2.4.-.el6.x86_64 /
Verifying : openldap-2.4.-.el6.x86_64 / Installed:
openldap-clients.x86_64 :2.4.-.el6 Dependency Updated:
openldap.x86_64 :2.4.-.el6 Complete!
3)拷贝配置文件，将KDC Server上的krb5.conf文件拷贝到所有Kerberos客户端(集群所有节点)
将node01上的/etc/krb5.conf，利用scp等命令分发到node02和node03
4.CDH集群启用Kerberos
1)在KDC中给Cloudera Manager添加管理员账号,并设置密码
root@node01 ~]# kadmin.local
Authenticating as principal admin/admin@LOCAL.DOMAIN with password.
kadmin.local: addprinc cloudera-scm/admin@LOCAL.DOMAIN
WARNING: no policy specified for cloudera-scm/admin@LOCAL.DOMAIN; defaulting to no policy
Enter password for principal "cloudera-scm/admin@LOCAL.DOMAIN":
Re-enter password for principal "cloudera-scm/admin@LOCAL.DOMAIN":
Principal "cloudera-scm/admin@LOCAL.DOMAIN" created.
kadmin.local: exit
CDH启用Kerberos
2)进入Cloudera Manager，集群，操作，启用kerberos
3)检查信息，勾选
4)KDC信息
5)不建议让Cloudera Manager来管理krb5.conf，点击“继续”
6) 输入CM的Kerbers管理员账号
7)Kerberos主体
8) 重启集群
使用HDFS时，由于票据过期出错，使用kinit重新登录Cloudera Manager管理员账号即可
[root@node01 ~]# hadoop fs -ls /
// :: WARN security.UserGroupInformation: Exception encountered while running the renewal command for cloudera-scm/admin@LOCAL.DOMAIN. (TGT end time:, renewalFailures: org.apache.hadoop.metrics2.lib.MutableGaugeInt@66f06ac9,renewalFailuresTotal: org.apache.hadoop.metrics2.lib.MutableGaugeLong@23f2e873)
ExitCodeException exitCode=: kinit: Ticket expired while renewing credentials at org.apache.hadoop.util.Shell.runCommand(Shell.java:)
at org.apache.hadoop.util.Shell.run(Shell.java:)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:)
at org.apache.hadoop.security.UserGroupInformation$.run(UserGroupInformation.java:)
at java.lang.Thread.run(Thread.java:)
// :: ERROR security.UserGroupInformation: TGT is expired. Aborting renew thread for cloudera-scm/admin@LOCAL.DOMAIN.
// :: WARN security.UserGroupInformation: PriviledgedActionException as:cloudera-scm/admin@LOCAL.DOMAIN (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
// :: WARN security.UserGroupInformation: PriviledgedActionException as:cloudera-scm/admin@LOCAL.DOMAIN (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
// :: WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than seconds before. Last Login=
^Z
[]+ Stopped hadoop fs -ls /
[root@node01 ~]# kinit cloudera-scm/admin@LOCAL.DOMAIN
Password for cloudera-scm/admin@LOCAL.DOMAIN:
[root@node01 ~]# hadoop fs -ls /
Found items
drwxrwxrwt - hdfs supergroup -- : /tmp
drwxr-xr-x - hdfs supergroup -- : /user
kafka
安装
配置
[root@node01 zookeeper]# /opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/kafka-topics --zookeeper node02:2181 --list
19/11/08 10:50:46 INFO utils.Log4jControllerRegistration$: Registered kafka:type=kafka.Log4jController MBean
19/11/08 10:50:46 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Initializing a new session to node02:2181.
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-cdh5.14.2--1, built on 03/27/2018 20:39 GMT
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:host.name=node01
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_231
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.home=/bigdata/jdk1.8.0_231/jre
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/activation-1.1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/activation-1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/aopalliance-1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/aopalliance-repackaged-2.5.0-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/apacheds-i18n-2.0.0-M15.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/apacheds-jdbm1-2.0.0-M2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/api-asn1-api-1.0.0-M20.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/api-util-1.0.0-M20.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/argparse4j-0.7.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/asm-3.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/avro-1.7.6-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/caffeine-2.7.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/cglib-2.2.1-v20090111.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/checker-qual-2.6.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-beanutils-1.8.3.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-beanutils-core-1.8.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-cli-1.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-codec-1.9.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-collections-3.2.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-compress-1.4.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-configuration-1.6.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-digester-1.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-el-1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-httpclient-3.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-io-2.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-lang-2.6.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-lang3-3.5.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-logging-1.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-math3-3.1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-net-3.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/commons-pool2-2.4.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/compileScala.mapping:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/connect-api-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/connect-basic-auth-extension-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/connect-file-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/connect-json-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/connect-runtime-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/connect-transforms-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/error_prone_annotations-2.3.3.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/gson-2.2.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/guava-20.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/guice-3.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/guice-servlet-3.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-annotations-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-archives-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-auth-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-common-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-common-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-core-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-mapreduce-client-shuffle-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-yarn-api-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-yarn-client-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-yarn-common-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-yarn-server-common-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hadoop-yarn-server-nodemanager-2.6.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hive-hcatalog-core-1.1.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hive-hcatalog-server-extensions-1.1.0-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hk2-api-2.5.0-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hk2-locator-2.5.0-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/hk2-utils-2.5.0-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/htrace-core4-4.0.1-incubating.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/httpclient-4.4.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/httpcore-4.4.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-annotations-2.9.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-annotations-2.9.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-core-2.9.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-core-asl-1.9.13-cloudera.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-databind-2.9.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-jaxrs-1.8.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-jaxrs-base-2.9.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-jaxrs-json-provider-2.9.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-mapper-asl-1.9.13-cloudera.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-module-jaxb-annotations-2.9.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jackson-xc-1.8.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javassist-3.22.0-CR2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javax.annotation-api-1.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javax.inject-1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javax.inject-2.5.0-b42.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/java-xmlbuilder-0.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/javax.ws.rs-api-2.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jaxb-api-2.2.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jaxb-api-2.3.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-client-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-common-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-container-servlet-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-container-servlet-core-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-guice-1.9.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-hk2-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-media-jaxb-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jersey-server-2.27.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jets3t-0.9.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jettison-1.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-client-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-continuation-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-http-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-io-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-security-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-server-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-servlet-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-servlets-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-util-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jetty-util-9.4.12.v20180830.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jopt-simple-5.0.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jsch-0.1.42.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jsp-api-2.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/jsr305-3.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka_2.11-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka_2.11-2.1.0-kafka-4.0.0-sources.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-clients-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-log4j-appender-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-streams-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-streams-examples-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-streams-scala_2.11-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-streams-test-utils-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/kafka-tools-2.1.0-kafka-4.0.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/leveldbjni-all-1.8.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/libthrift-0.9.3.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/log4j-1.2.17.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/lz4-java-1.5.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/maven-artifact-3.5.4.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/metrics-core-2.2.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/metrics-servlet-2.2.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/netty-3.10.5.Final.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/osgi-resource-locator-1.0.1.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/paranamer-2.3.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/plexus-utils-3.1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/reflections-0.9.11.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/rocksdbjni-5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/scala-library-2.11.12.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/scala-logging_2.11-3.9.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/scala-reflect-2.11.12.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-binding-hive-conf-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-binding-hive-follower-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-binding-kafka-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-core-common-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-core-model-db-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-core-model-indexer-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-core-model-kafka-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-hdfs-common-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-policy-common-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-policy-indexer-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-policy-kafka-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-provider-common-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-provider-db-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/sentry-provider-file-1.5.1-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/servlet-api-2.5.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/shiro-core-1.2.3.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/slf4j-api-1.7.25.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/slf4j-api-1.7.5.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/slf4j-log4j12-1.7.5.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/snappy-java-1.1.7.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/stax-api-1.0-2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/validation-api-1.1.0.Final.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/xmlenc-0.52.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/xz-1.0.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/zkclient-0.10.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/zookeeper-3.4.5-cdh5.14.2.jar:/opt/cloudera/parcels/KAFKA-4.0.0-1.4.0.0.p0.1/bin/../lib/kafka/bin/../libs/zstd-jni-1.3.5-4.jar:/etc/kafka/conf/sentry-conf
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-696.16.1.el6.x86_64
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:user.name=root
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:user.home=/root
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Client environment:user.dir=/etc/zookeeper
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=node02:2181 sessionTimeout=30000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@67c27493
19/11/08 10:50:46 INFO zookeeper.ClientCnxn: Opening socket connection to server node02/xxxxxx:2181. Will not attempt to authenticate using SASL (unknown error)
19/11/08 10:50:46 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Waiting until connected.
19/11/08 10:50:46 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /172.16.221.xx:35396, server: node02/172.16.237.xx:2181
19/11/08 10:50:46 INFO zookeeper.ClientCnxn: Session establishment complete on server node02/172.16.237.xx:2181, sessionid = 0x16e46647cc30394, negotiated timeout = 30000
19/11/08 10:50:46 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Connected.
topic_start
19/11/08 10:50:46 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Closing.
19/11/08 10:50:46 INFO zookeeper.ZooKeeper: Session: 0x16e46647cc30394 closed
19/11/08 10:50:46 INFO zookeeper.ClientCnxn: EventThread shut down
19/11/08 10:50:46 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Closed.
启用Kerberos
修改security.inter.broker.protocol
重启kafka服务完成以上配置，Kafka集群已启用Kerberos认证
在各个节点上：
配置jaas.conf文件
[root@node01 kafka_client]# pwd
/usr/local/kafka_client #创建文件
[root@node01 kafka_client]# vi jaas.conf
内容如下
KafkaClient{
com.sun.security.auth.module.Krb5LoginModule required
useTicketCache=true;
};
配置client.properties文件
[root@node01 kafka_client]# vi client.properties
内容如下
security.protocol=SASL_PLAINTEXT
sasl.kerberos.service.name=kafka
初始化kerberos账号
[root@node01 kafka_client]# kinit cloudera-scm/admin@LOCAL.DOMAIN
Password for cloudera-scm/admin@LOCAL.DOMAIN:
不要忘了导入变量，否则会报错
Caused by: java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set
找不到jaas配置文件
在KAFKA_OPTS变量里加上" -Djava.security.auth.login.config=/path/to/kafka_server_jaas.conf"
export KAFKA_OPTS="-Djava.security.auth.login.config=/usr/local/kafka_client/jaas.conf"
根据所配置的配置文件启动client
启动生产端
[root@node02 kafka_client]# kafka-console-producer --broker-list node01:,node02:,node03: --topic kerbero --producer.config client.properties
// :: INFO utils.Log4jControllerRegistration$: Registered kafka:type=kafka.Log4jController MBean
// :: INFO producer.ProducerConfig: ProducerConfig values:
acks =
batch.size =
bootstrap.servers = [node01:, node02:, node03:]
buffer.memory =
client.dns.lookup = default
client.id = console-producer
compression.type = none
connections.max.idle.ms =
delivery.timeout.ms =
enable.idempotence = false
interceptor.classes = []
key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
linger.ms =
max.block.ms =
max.in.flight.requests.per.connection =
max.request.size =
metadata.max.age.ms =
metric.reporters = []
metrics.num.samples =
metrics.recording.level = INFO
metrics.sample.window.ms =
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes =
reconnect.backoff.max.ms =
reconnect.backoff.ms =
request.timeout.ms =
retries =
retry.backoff.ms =
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin =
sasl.kerberos.service.name = kafka
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds =
sasl.login.refresh.min.period.seconds =
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = SASL_PLAINTEXT
send.buffer.bytes =
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1., TLSv1., TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
transaction.timeout.ms =
transactional.id = null
value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer // :: INFO authenticator.AbstractLogin: Successfully logged in.
// :: INFO kerberos.KerberosLogin: [Principal=null]: TGT refresh thread started.
// :: INFO kerberos.KerberosLogin: [Principal=null]: TGT valid starting at: Fri Nov :: CST
// :: INFO kerberos.KerberosLogin: [Principal=null]: TGT expires: Sat Nov :: CST
// :: WARN kerberos.KerberosLogin: The TGT cannot be renewed beyond the next expiry date: Sat Nov :: CST .This process will not be able to authenticate new SASL connections after that time (for example, it will not be able to authenticate a new connection with a Kafka Broker). Ask your system administrator to either increase the 'renew until' time by doing : 'modprinc -maxrenewlife null ' within kadmin, or instead, to generate a keytab for null. Because the TGT's expiry cannot be further extended by refreshing, exiting refresh thread now.
// :: INFO utils.AppInfoParser: Kafka version : 2.1.-kafka-4.0.
// :: INFO utils.AppInfoParser: Kafka commitId : unknown
>// :: INFO clients.Metadata: Cluster ID: 9EbFfkQERomQdy0wrndVjQ >hello
>python
>hello
启动消费端
[root@node03 kafka_client]# kafka-console-consumer --topic kerbero --from-beginning --bootstrap-server node01:,node01:,node03: --consumer.config client.properties
// :: INFO utils.Log4jControllerRegistration$: Registered kafka:type=kafka.Log4jController MBean
// :: INFO consumer.ConsumerConfig: ConsumerConfig values:
auto.commit.interval.ms =
auto.offset.reset = earliest
bootstrap.servers = [node01:, node01:, node03:]
check.crcs = true
client.dns.lookup = default
client.id =
connections.max.idle.ms =
default.api.timeout.ms =
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes =
fetch.max.wait.ms =
fetch.min.bytes =
group.id = console-consumer-
heartbeat.interval.ms =
interceptor.classes = []
internal.leave.group.on.close = true
isolation.level = read_uncommitted
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes =
max.poll.interval.ms =
max.poll.records =
metadata.max.age.ms =
metric.reporters = []
metrics.num.samples =
metrics.recording.level = INFO
metrics.sample.window.ms =
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes =
reconnect.backoff.max.ms =
reconnect.backoff.ms =
request.timeout.ms =
retry.backoff.ms =
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin =
sasl.kerberos.service.name = kafka
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds =
sasl.login.refresh.min.period.seconds =
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = SASL_PLAINTEXT
send.buffer.bytes =
session.timeout.ms =
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1., TLSv1., TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer // :: INFO authenticator.AbstractLogin: Successfully logged in.
// :: INFO kerberos.KerberosLogin: [Principal=null]: TGT refresh thread started.
// :: INFO kerberos.KerberosLogin: [Principal=null]: TGT valid starting at: Fri Nov :: CST
// :: INFO kerberos.KerberosLogin: [Principal=null]: TGT expires: Sat Nov :: CST
// :: WARN kerberos.KerberosLogin: The TGT cannot be renewed beyond the next expiry date: Sat Nov :: CST .This process will not be able to authenticate new SASL connections after that time (for example, it will not be able to authenticate a new connection with a Kafka Broker). Ask your system administrator to either increase the 'renew until' time by doing : 'modprinc -maxrenewlife null ' within kadmin, or instead, to generate a keytab for null. Because the TGT's expiry cannot be further extended by refreshing, exiting refresh thread now.
// :: INFO utils.AppInfoParser: Kafka version : 2.1.-kafka-4.0.
// :: INFO utils.AppInfoParser: Kafka commitId : unknown
// :: INFO clients.Metadata: Cluster ID: 9EbFfkQERomQdy0wrndVjQ
// :: INFO internals.AbstractCoordinator: [Consumer clientId=consumer-, groupId=console-consumer-] Discovered group coordinator node02: (id: rack: null)
// :: INFO internals.ConsumerCoordinator: [Consumer clientId=consumer-, groupId=console-consumer-] Revoking previously assigned partitions []
// :: INFO internals.AbstractCoordinator: [Consumer clientId=consumer-, groupId=console-consumer-] (Re-)joining group
// :: INFO internals.AbstractCoordinator: [Consumer clientId=consumer-, groupId=console-consumer-] Successfully joined group with generation
// :: INFO internals.ConsumerCoordinator: [Consumer clientId=consumer-, groupId=console-consumer-] Setting newly assigned partitions [kerbero-]
// :: INFO internals.Fetcher: [Consumer clientId=consumer-, groupId=console-consumer-] Resetting offset for partition kerbero- to offset .
hello
python
hello
JAAS 是个什么梗
hue启动报Kerberos Ticket Renewer已停止
解决:
原因:kerberos凭证过期;
先进入kerberos模式:
kadmin.local命令然后,然后操作下面的
kadmin.local: modprinc -maxrenewlife 90day krbtgt/YOUR_REALM.COM
kadmin.local: modprinc -maxrenewlife 90day +allow_renewable hue/<hostname>@YOUR-REALM.COM
命令出处:http://t.cn/R8ttGKM
http://web.mit.edu/kerberos/krb5-1.12/doc/admin/admin_commands/kadmin_local.html
kadmin [-O|-N] [-r realm] [-p principal] [-q query] [[-c cache_name]|[-k [-t keytab]]|-n] [-w password] [-s admin_server[:port]]
kadmin.local [-r realm] [-p principal] [-q query] [-d dbname] [-e enc:salt ...] [-m] [-x db_args]
DESCRIPTION
kadmin and kadmin.local are command-line interfaces to the Kerberos V5 administration system. They provide nearly identical functionalities; the difference is that kadmin.local directly accesses the KDC database, while kadmin performs operations using kadmind. Except as explicitly noted otherwise, this man page will use “kadmin” to refer to both versions. kadmin provides for the maintenance of Kerberos principals, password policies, and service key tables (keytabs).
The remote kadmin client uses Kerberos to authenticate to kadmind using the service principal kadmin/ADMINHOST (where ADMINHOST is the fully-qualified hostname of the admin server) or kadmin/admin. If the credentials cache contains a ticket for one of these principals, and the -c credentials_cache option is specified, that ticket is used to authenticate to kadmind. Otherwise, the -p and -k options are used to specify the client Kerberos principal name used to authenticate. Once kadmin has determined the principal name, it requests a service ticket from the KDC, and uses that service ticket to authenticate to kadmind.
Since kadmin.local directly accesses the KDC database, it usually must be run directly on the master KDC with sufficient permissions to read the KDC database. If the KDC database uses the LDAP database module, kadmin.local can be run on any host which can access the LDAP server.
kadmin.local //以超管身份进入kadmin kadmin //进入kadmin模式，需输入密码 kdb5_util create -r JENKIN.COM -s //创建数据库 service krb5kdc start //启动kdc服务 service kadmin start //启动kadmin服务 service kprop start //启动kprop服务 kdb5_util dump /var/kerberos/krb5kdc/slave_data //生成dump文件 kprop -f /var/kerberos/krb5kdc/slave_data master2.com //将master数据库同步是slave kadmin模式下： addprinc -randkey root/master1@JENKIN.COM //生成随机key的principal addprinc admin/admin //生成指定key的principal listprincs //查看principal change_password -pw xxxx admin/admin //修改admin/admin的密码 delete_principal admin/admin //删除principal kinit admin/admin //验证principal是否可用 xst -norandkey -k /var/kerberos/krb5kdc/keytab/root.keytab root/master1@JENKIN.COM host/master1@JENKIN.COM //为principal生成keytab，可同时添加多个 ktadd -k /etc/krb5.keytab host/master1@JENKIN.COM //ktadd也可生成keytab kinit -k -t /var/kerberos/krb5kdc/keytab/root.keytab root/master1@JENKIN.COM //测试keytab是否可用 klist -e -k -t /var/kerberos/krb5kdc/keytab/root.keytab //查看keytab
Cloudera Hadoop启用Kerberos认证的更多相关文章
- yarn 用户导致的被挖矿 启用Kerberos认证功能，禁止匿名访问修改8088端口
用户为dr.who,问下内部使用人员,都没有任务在跑: 结论: 恭喜你,你中毒了,攻击者利用Hadoop Yarn资源管理系统REST API未授权漏洞对服务器进行攻击,攻击者可以在未授权的情况下远程 ...
- hadoop的kerberos认证
言归正传,介绍过hadoop的simple认证和kerberos后,我们在这一章介绍hadoop的kerberos认证 我们还使用hadoop集群的机器. OS 版本: Centos6.4 Kerbe ...
- Java Api Consumer 连接启用Kerberos认证的Kafka
java程序连接到一个需要Kerberos认证的kafka集群上,消费生产者生产的信息,kafka版本是2.10-0.10.0.1: Java程序以maven构建,(怎么构建maven工程,可去问下度 ...
- kafka集群安全化之启用kerberos与acl
一.背景 在我们部署完kafka之后,虽然我们已经可以“肆意”的用kafka了,但是在一个大公司的实际生产环境中,kafka集群往往十分庞大,每个使用者都应该只关心自己所负责的Topic,并且对其他人 ...
- 配置两个Hadoop集群Kerberos认证跨域互信
两个Hadoop集群开启Kerberos验证后,集群间不能够相互访问,需要实现Kerberos之间的互信,使用Hadoop集群A的客户端访问Hadoop集群B的服务(实质上是使用Kerberos Re ...
- Cloudera公司主要提供Apache Hadoop开发工程师认证
Cloudera Cloudera公司主要提供Apache Hadoop开发工程师认证(Cloudera CertifiedDeveloper for Apache Hadoop ,CCDH)和Apa ...
- hadoop KerberosUtil 做Kerberos认证
网上找了一下,自己写了个KerberosUtil工具类,测试过可以用. 注意这个不是 org.apache.hadoop.security.authentication.util.KerberosUt ...
- cloudera集群开启kerberos认证后，删除zk中的/hbase目录
问题 在cdh集群中开启了kerberos认证,hbase集群出现一点问题,需要通过zookeeper-client访问zookeeper,删除/hbase节点时候报错:Authentication ...
- window 环境下jdbc访问启用kerberos的impala
最近,公司生产集群添加kerberos安全认证后,访问集群的任何组件都需要进行认证,这样问题来了,对于impala,未配置kerberos安全认证之前通过impala的jdbc驱动(impala-jd ...
随机推荐
- SQL必知必会学习笔记
2.5 select SELECT 要返回的列或表达式 是FROM 从中检索数据的表 仅在从表选择数据时使用WHERE 行级过滤 ...
- 每天一个linux命令：head(15)
head head命令用于显示文件的开头的内容.在默认情况下,head命令显示文件的头10行内容. 格式 head [参数] [文件] 参数选项 参数 备注 -q 不显示文件名的头信息 -v 总是 ...
- 关于vue2.x使用axios以及http-proxy-middleware代理处理跨域的问题
axios现在以及是尤大大推荐使用的了,官方不在维护vue-reresource. 由于是地第一次使用axios, 在使用过程中猜了很大的坑 首先我们使用vue-cli创建的项目, 访问接口肯定是跨域 ...
- linux0.11内核源码——用户级线程及内核级线程
参考资料:哈工大操作系统mooc 用户级线程 1.每个进程执行时会有一套自己的内存映射表,即我们所谓的资源,当执行多进程时切换要切换这套内存映射表,即所谓的资源切换 2.但是如果在这个进程中创建线程, ...
- 【Java】java获取json中某个字段
import com.alibaba.fastjson.JSONObject; public class JsonTest { public static void main(String[] arg ...
- 在python3.7下怎么安装matplotlib与numpy
一.安装matplotlib 1.在Matplotlib的官网下载电脑对应的版本,网址为:https://pypi.org/project/matplotlib/#files 2.将在下载的.whl文 ...
- paper 150：GCC--GNU Compiler Collection（GNU编译器套件）
gcc命令 编程开发 gcc命令使用GNU推出的基于C/C++的编译器,是开放源代码领域应用最广泛的编译器,具有功能强大,编译代码支持性能优化等特点.现在很多程序员都应用GCC, ...
- 【靶场练习_upload-labs复现】Pass01-020
文件上传本是要命,挂马成功率更是随缘,我太难了Orz Pass-01:JS <?php phpinfo();?> 1.函数重写: 2.禁用js: Pass-02:MIME Type 修改M ...
- DZY Loves Math
DZY Loves Math 对于正整数 $n$,定义 $f(n)$ 为 $n$ 所含质因子的最大幂指数. 例如 $f(1960)=f(2^3 * 5^1 * 7^2)=3, f(10007)=1, ...
- codeforces 557D Vitaly and Cycle
题意简述 给定一个图 求至少添加多少条边使得它存在奇环 并求出添加的方案数 (注意不考虑自环) ---------------------------------------------------- ...