How to resolve ERROR: Fatal error during KafkaServer startup. Prepare to shutdown | Error| Resolution

In this article, we will explain how to resolve the “Found directory /mnt/kafkalogs/lost+found, ‘lost+found’ is not in the form of topic-partition”.



[ 2020-11-2415:06:19,687] INFO [ThrottledChannelReaper-Fetch]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[ 2020-11-2415:06:19,689] INFO [ThrottledChannelReaper-Produce]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[ 2020-11-2415:06:19,693] INFO [ThrottledChannelReaper-Request]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[ 2020-11-2415:06:19,798] INFO Loading logs. (kafka.log.LogManager)
[ 2020-11-2415:06:19,816] ERROR There was an error in one of the threads during logs loading: org.apache.kafka.common.KafkaException: Found directory /mnt/kafkalogs/lost+found, 'lost+found' is not in the form of topic-partition or topic-partition.uniqueId-delete (if marked for deletion).
Kafka's log directories (and children) should only contain Kafka topic data. (kafka.log.LogManager)
[ 2020-11-2415:06:19,819] ERROR [KafkaServer id=2] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.kafka.common.KafkaException: Found directory /mnt/kafkalogs/lost+found, 'lost+found' is not in the form of topic-partition or topic-partition.uniqueId-delete (if marked for deletion).
Kafka's log directories (and children) should only contain Kafka topic data.
at kafka.log.Log$.exception$1(Log.scala:2150)
at kafka.log.Log$.parseTopicPartitionName(Log.scala:2157)
at kafka.log.LogManager.kafka$log$LogManager$$loadLog(LogManager.scala:260)
at kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$11$$anonfun$apply$15$$anonfun$apply$2.apply$mcV$sp(LogManager.scala:345)
at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:63)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[ 2020-11-2415:06:19,823] INFO [KafkaServer id=2] shutting down (kafka.server.KafkaServer)
[ 2020-11-2415:06:19,826] INFO [ZooKeeperClient] Closing. (kafka.zookeeper.ZooKeeperClient)
[ 2020-11-2415:06:19,879] INFO Session: 0x1000000c6c60001 closed (org.apache.zookeeper.ZooKeeper)
[ 2020-11-2415:06:19,881] INFO [ZooKeeperClient] Closed. (kafka.zookeeper.ZooKeeperClient)
[ 2020-11-2415:06:19,881] INFO EventThread shut down for session: 0x1000000c6c60001 (org.apache.zookeeper.ClientCnxn)
[ 2020-11-2415:06:19,882] INFO [ThrottledChannelReaper-Fetch]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[ 2020-11-24 11:34:15,688] INFO [ThrottledChannelReaper-Fetch]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[ 2020-11-24 11:34:15,688] INFO [ThrottledChannelReaper-Fetch]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[ 2020-11-24 11:34:15,688] INFO [ThrottledChannelReaper-Produce]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[ 2020-11-24 11:34:15,689] INFO [ThrottledChannelReaper-Produce]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[ 2020-11-24 11:34:15,689] INFO [ThrottledChannelReaper-Produce]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[ 2020-11-24 11:34:15,689] INFO [ThrottledChannelReaper-Request]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[ 2020-11-24 11:34:15,693] INFO [ThrottledChannelReaper-Request]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[ 2020-11-24 11:34:15,693] INFO [ThrottledChannelReaper-Request]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[ 2020-11-24 11:34:15,699] INFO [KafkaServer id=2] shut down completed (kafka.server.KafkaServer)
[ 2020-11-24 11:34:15,700] INFO Shutting down SupportedServerStartable (io.confluent.support.metrics.SupportedServerStartable)
[ 2020-11-24 11:34:15,700] INFO Closing BaseMetricsReporter (io.confluent.support.metrics.BaseMetricsReporter)
[ 2020-11-24 11:34:15,700] INFO Waiting for metrics thread to exit (io.confluent.support.metrics.SupportedServerStartable)
[ 2020-11-24 11:34:15,701] INFO Shutting down KafkaServer (io.confluent.support.metrics.SupportedServerStartable)
[ 2020-11-24 11:34:15,701] INFO [KafkaServer id=2] shutting down (kafka.server.KafkaServer)

The above error belongs log directory related issue, here we provided simple solution for the Kafka log directory issue.
Solution:




Step 1: First, check the Kafka log directory in the below properties file

vi /etc/kafak/server.proeprties

Step 2:  In the above file it showing like below

logs.dir = /etc/kafka/logs

Step 3:  After that execute the below command

kafka-server-start /etc/kafka/server.properties \
--override log.dirs=/etc/kafka/logs

Step 4: Once completed all the steps then try to restart Confluent Kafka service using below command

systemctl start confluent-kafka