In this article, we will explain how to setup Kafka with SSL configurations. Here we provided Kafka commands for Linux and RHEL.
Kafka Installation with SSL Configurations
Kafka ssl setup (encryption between Kafka client and Kafka broker)
Step 1:First, we will create a CA Authority using below command
openssl req -new -newkey rsa:4096 -days 365 -x509 -subj "/CN=kafka-security-CA" -keyout ca-key -out ca-cert -nodes
Step 2: Setting up SSL for Kafka brokers in the cluster in Zookeeper:
I) Configure keystore for Kafka broker
keytool -genkey -keystore kafka1ks.jks -validity 365 -storepass sreekanth@123 -keypass sreekanth@123 -dname "CN=kafka1.training.com,OU=I/T Infrastructure,O=IBM INA Holdings Inc,L=Philadelphia,ST=Pennsylvania,C=US" -storetype pkcs12 --keyalg RSA -keysize 2048
b) Here to view the content of keystore:
keytool -list -keystore kafka1ks.jks -v
c) Get the signed version of keystore certificate (it is a two step process):
c.1) Do a certificate request from our keystore
keytool -keystore kafka1ks.jks -certreq -file kafka1ks.csr
c.2) Sign the requested certificate
openssl x509 -req -CA /home/kafka/ca/ca-cert -CAkey /home/kafka/ca/ca-key -in kafka1ks.csr -out kafka1ks.crt -days 365 -CAcreateserial
To list the signed certificate details:
keytool -printcert -v -file kafka1ks.crt
d) Now add the ca-cert (public certficate) and ca-signed(signed certificate) in kafka1ks.jks
keytool -keystore kafka1ks.jks -alias CARoot -import -file /home/kafka/ca/ca-cert -storepass sreekanth@123 (now ca-cert is added to the key store with alias CARoot)
keytool -keystore kafka1ks.jks -import -file kafka1ks.crt -storepass kafka1ks@123
(Now ca-signed cert is added to the key store)
II) create a trust store and import ca-cert (which is CA public certificate):
keytool -keystore kafka1ts.jks -alias CARoot -import -file /home/kafka/ca/ca-cert -storepass sreets@123
III) Using the certificates we have generated, we will setup kafka broker to use SSL on the port 9093
Alter server.properties file with below details:
listeners=PLAINTEXT://broker-host-name:non-ssl-port,SSL://broker-host-name:ssl-port advertised.listeners=PLAINTEXT://broker-host-name:non-ssl-port,SSL://broker-host-name:ssl-port ssl.keystore.location=/home/apps/kafka/private/ssl/server.keystore.jks ssl.keystore.password=<keystore-password> ssl.key.password=<keystore-password> ssl.truststore.location=/home/apps/kafka/private/ssl/server.truststore.jks ssl.truststore.password=<keystore-password>
IV) Once kafka is rebooted, we will test the SSL certificates once again to make sure they are correctly applied
To test the SSL connectivity from your kafka client to brokers:
openssl s_client -connect <broker-host-name>:<broker-ssl-port>
Step 3:
Kafka Client side setup:
————————-
I) Get the ca-cert (CA public certificate) from kafka broker server to client server (application server)
II) Create a trust store for kafka client server and import ca-cert file in trust store
keytool -keystore maheshts.jks -alias caroot -import -file ca-cert
III) Update the client.properties file with below values
security.protocol=SSL ssl.truststore.location=<location-of-client-truststore-file-name>.jks ssl.truststore.password=<truststore-password>
After completed above steps then will send messages from kafka client in secure mode.