ERROR:"org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /config/topics/test" when creating or deleting Kafka operations authorized through the Ranger policies

PROBLEM DESCRIPTION

When creating or deleting topics in Kafka, they cannot be authorized through the Ranger policies. The following errors are displayed while creating the topics:

[ADM_xxxx@xxxx-oc-had102 ~]$ /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper xxxx-oc-had101.example.local:,
xxxx-oc-had201.example.local:,xxxx-oc-had102.example.local:  --create --topic test --partition  --replication-factor
Error while executing topic command : org.apache.zookeeper.KeeperException$NoAuthException:
KeeperErrorCode = NoAuth for /config/topics/test
[-- ::,] ERROR org.I0Itec.zkclient.exception.ZkException:
org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /config/topics/test
.
.
Caused by: org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /config/topics/test

ROOT CAUSE

Kafka with Ranger is only recommended in secure clusters. The above errors are displayed while creating or deleting topic from an ordinary user. This is because only the process owner of Kafka service such as root, can write to Zookeeper znodes (/configs/topics). Ranger policies do not get enforced when a non privileged user creates a topic. This is because kafka-topics.sh script talks directly to Zookeeper to create the topic. It will add entries into the Zookeeper nodes and the watchers on the broker side will monitor and create topics accordingly. Due to the script talking to Zookeeper directly, the authorization cannot be done through the ranger plugin.

RESOLUTION

For the users to create topics, run a script called kafka-acls.sh which will allow or deny users on topics and provide other options.

Note

This is applicable only in secure environment only. The more details on this see the Authorizing Access when Kerberos is Enabled documentation.

About:
This article created by Hortonworks Support (Article: 000005366) on 2017-06-27 05:17
OS: Linux
Type: Configuration, Cluster_Administration
Version: HDP

Support ID: 000005366