Configuring Kerberos for Cloudera’s Hadoop distribution is not as hard as it may seem, but it calls upon very careful configuration of DNS and required packages. If carried out incorrectly, such a process may result in failed configuration, and what’s worse, usually there is not much logging information, which complicates the troubleshooting process.

Here is a working set of step by step commands to configure Kerberos in Cloudera’s Hadoop Distribution (CDH) deployed into AWS using West-2 zone and Centos 6.5 images.

DNS

Kerberos depends on DNS configuration when it verifies requests from hosts, i.e. DNS configuration presupposes 90% of success, so it’s important to have all the configuration changes accomplished on the DNS side (and NOT /etc/hosts).

So, before moving any further make sure all host names are configured in DNS, e.g. commands below return the same values:

hostname

nslookup <current IP address>

Configure Kerberos

Run the following command on host selected as a Kerberos Master Server (usually it’s Cloudera Manager or edge-server) to install Kerberos Server and client (workstati on):

yum -y install krb5-libs krb5-server krb5-workstation

Then, run this command on all slave nodes (as a rule, computation nodes) to install Kerberos client (workstation):

yum -y install krb5-workstation

Configure Kerberos on Kerberos Master Server:

[root@ip-172-31-34-200 cloudera-scm-server]# vi /var/kerberos/krb5kdc/kdc.conf



[kdcdefaults]

 kdc_ports = 88

 kdc_tcp_ports = 88



[realms]

 CMRealm = {

  #master_key_type = aes256-cts

  acl_file = /var/kerberos/krb5kdc/kadm5.acl

  dict_file = /usr/share/dict/words

  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab

  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal

      max_life = 24h 0m 0s

      max_renewable_life = 7d 0h 0m 0s

 }

Configure Kerberos client (workstation) on all servers (including master and slaves):

echo "[logging]

 default = FILE:/var/log/krb5libs.log

 kdc = FILE:/var/log/krb5kdc.log

 admin_server = FILE:/var/log/kadmind.log



[libdefaults]

 default_realm = CMRealm

 dns_lookup_realm = false

 dns_lookup_kdc = false

 ticket_lifetime = 24h

 renew_lifetime = 7d

 forwardable = true



[realms]

 CMRealm = {

  kdc = ip-172-31-37-43.us-west-2.compute.internal

  admin_server = ip-172-31-37-43.us-west-2.compute.internal

 }



[d omain_realm]

us-west-2.compute.internal = CMRealm

.us-west-2.compute.internal = CMRealm" > /etc/krb5.conf

Initialize Kerberos database on Master Server:

[root@ip-172-31-34-200 cloudera-scm-server]# kdb5_util create -s

Loading random data

Initializing database '/var/kerberos/krb5kdc/principal' for realm 'CMRealm',

master key name 'K/M@CMRealm'

You will be prompted for the database Master Password.

It is important that you NOT FORGET this password.

Enter KDC database master key: welcome1

Re-enter KDC database master key to verify:





echo "*/admin@CMRealm *" > /var/kerberos/krb5kdc/kadm5.acl

Add the administrator principle (on Kerberos Master Server):

[root@ip-172-31-34-200 cloudera-scm-server]# kadmin.local -q "addprinc tunde/admin"

Authenticating as principal root/admin@CMRealm with password.

WARNING: no policy specified for tunde/admin@CMRealm; defaulting to no policy

Enter password for principal "tunde/admin@CMRealm":

Re-enter password for principal "tunde/admin@CMRealm":

Principal "tunde/admin@CMRealm" created.

Run Kerberos and make sure it starts automatically when the system starts:

service krb5kdc start

service kadmin start

chkconfig krb5kdc on

chkconfig kadmin on

Verify the list of principles:

[root@ip-172-31-34-200 cloudera-scm-server]# kadmin -p tunde/admin -q "list_principals"

Initialize ticket for principle “tunde/admin”

[root@ip-172-31-34-200 cloudera-scm-server]# kinit tunde/admin

[root@ip-172-31-34-200 cloudera-scm-server]# klist

[root@ip-172-31-34-200 cloudera-scm-server]# kinit -R

Create administrator account on Kerberos Master:

kadmin -p tunde/admin -q "addprinc -pw welcome1 cloudera-scm/admin@CMRealm"

kadmin -p tunde/admin -q "xst -k cmf.keytab cloudera-scm/admin@CMRealm"

Copy Kerberos configuration files to CDH location so that CDH can pick it up:

mv cmf.keytab /etc/cloudera-scm-server/

chown cloudera-scm:cloudera-scm /etc/cloudera-scm-server/cmf.keytab

chmod 600 /etc/cloudera-scm-server/cmf.keytab



echo "cloudera-scm/admin@CMRealm" > /etc/cloudera-scm-server/cmf.principal

cat /etc/cloudera-scm-server/cmf.principal

chown cloudera-scm:cloudera-scm /etc/cloudera-scm-server/cmf.principal

chmod 600 /etc/cloudera-scm-server/cmf.principal

Configuring Kerberos with Cloudera Manager UI

Once all the hosts (master and slaves) are configured for Kerberos, configure Kerberos in Cloudera Manager where you need to specify Kerberos Master host, realm, and administrator principal.

For more details, check out these blogs:

Configuring Kerberos for Cloudera’s Hadoop Distribution (CDH)