zookeeper is the dependency of kafka

install java and zookeeper

apt-get update
apt-get install default-jre -y
apt-get install zookeeperd -y

test zookeeper

telnet 127.0.0.1 2181

enter:

ruok

it should respond with:

imok

create a user for kafka and login as kafka user

adduser kafka
adduser kafka sudo
su -l kafka

download and install apache kafka:

mkdir ~/Downloads
curl "https://downloads.apache.org/kafka/2.8.1/kafka-2.8.1-src.tgz" -o ~/Downloads/kafka.tgz
mkdir ~/kafka && cd ~/kafka
tar -zxvf ~/Downloads/kafka.tgz --strip 1

Configuring Kafka Server:

vi ~/kafka/config/server.properties

Add a setting that allows us to delete Kafka topics first. Add the following to the file’s bottom:

delete.topic.enable = true

create services:

sudo vi /etc/systemd/system/zookeeper.service

Next, you need to add the below content:

[Unit]
Requires=network.target remote-fs.target
After=network.target remote-fs.target

[Service]
Type=simple
User=kafka
ExecStart=/home/kafka/kafka/bin/zookeeper-server-start.sh /home/kafka/kafka/config/zookeeper.properties
ExecStop=/home/kafka/kafka/bin/zookeeper-server-stop.sh
Restart=on-abnormal

[Install]
WantedBy=multi-user.target
sudo vi /etc/systemd/system/kafka.service

Next, you need to add the below content:

[Unit]
Requires=zookeeper.service
After=zookeeper.service

[Service]
Type=simple
User=kafka
ExecStart=/bin/sh -c '/home/kafka/kafka/bin/kafka-server-start.sh /home/kafka/kafka/config/server.properties > /home/kafka/kafka/kafka.log 2>&1'
ExecStop=/home/kafka/kafka/bin/kafka-server-stop.sh
Restart=on-abnormal

[Install]
WantedBy=multi-user.target

enable and start services:

sudo systemctl start kafka
sudo systemctl status kafka
sudo systemctl enable zookeeper
sudo systemctl enable kafka

Testing kafka installation:

be sure executing this commands as kafka user:

~/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic TutorialTopic

echo "Hello, World" | ~/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic TutorialTopic > /dev/null

the consumer can connect and read messages from another terminal:

~/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic TutorialTopic --from-beginning