January 28, 2023


Technology Room

Resolve Apache Kafka Beginning Situation Put in on Single/Multi-Node Cluster

2 min read

 This brief article explains learn how to resolve the error: ERROR Deadly error throughout KafkaServer startup. Put together to shutdown (kafka.server.KafkaServer)kafka.widespread.InconsistentClusterIdException once we begin the Apache Kafka put in and configured on a multi-node cluster. You possibly can learn the steps for organising a multi-node Apache Kafka cluster here.

 With out integrating Apache Zookeeper, Kafka alone gained’t have the ability to kind the entire Kafka cluster. As a result of ZooKeeper handles the management election of Kafka brokers and manages service discovery in addition to cluster topology. It additionally tracks when subjects are created or deleted from the cluster and maintains a subject checklist. General, ZooKeeper supplies an in-sync view of the Kafka cluster.

The above-mentioned error is sort of widespread and might be encountered whereas we’re updating the server.propertiesfile on every Kafka Dealer within the present cluster so as to add an extra zookeeper server entry due to the brand new Kafka Dealer addition into the cluster. It may be attainable to face this situation on a single node Kafka Dealer, too, after a system reboot if carried out any configuration modifications on the Zookeeper’s zoo.cfg or Kafka’s server. properties file to change/replace the /tmp listing location.

The foundation reason behind the problem is when Zookeeper information are saved within the momentary folder and Kafka logs are saved within the persistent folder or vice versa. Ideally, the default worth of dataDir is /tmp/zookeeper  in zoo.cfg  or Zookeeper to retailer the snapshots, and equally, the default worth of log.dirs in server.properties for Kafka Dealer is /tmp/kafka-logs. After the system restarts, recordsdata which might be saved in that momentary listing get cleaned, and a brand new cluster ID could be generated because of the full new registration of the Kafka cluster. Ultimately, this configuration mismatch or cluster id quantity variation causes the above point out error.

There are two methods to resolve the above error shortly, however they aren’t really helpful for the manufacturing surroundings.

Possibility 1: Replace the Cluster ID within the file meta.properties 

The meta.properties file might be situated at /tmp/kafka-logs if the worth of log.dirskey in server.properties was not up to date with a persistent folder location. Open the meta.properties, the contents might be related as proven under. Change the cluster.id worth with the brand new cluster ID current within the error log and restart the Kafka server.


Possibility 2: Delete /tmp/kafka-logs/meta.properties which saved the unsuitable cluster.id of final/failed session.

Hope you might have loved this learn. Please like and share in case you really feel this composition is efficacious.

Copyright © All rights reserved. | Newsphere by AF themes.