Apache Spark-Kafka API

  • date 15th March, 2019 |
  • by Prwatech |
  • 0 Comments
Step 1: Download Kafka

Download the kafka_2.11-2.0.0 release and un-tar it.

https://archive.apache.org/dist/kafka/2.0.0/kafka_2.11-2.0.0.tgz

And get into Kafka directory

Step 2: Start the server

Start a ZooKeeper server. and kafka server

Step 3: Create the topics for the example programs

We need two topics for the example program

These can be listed

Step 4: Now visit to below given site and download the package (maven package)

https://github.com/mapr-demos/kafka-sample-programs

And save it on your desktop

Step 5: Compile and package up the example programs

Go back to the directory where you have the example programs and compile and build the example programs.

Note : You need to Install Maven first on your ubuntu system to read the package

Command :   sudo apt install maven

Step 6: Run the example producer

Step 7: Start the example consumer

After running the consumer once, however, if we run the producer again and then run the consumer again, we will see the consumer pick up and start processing messages shortly after it starts.

Leave a Reply

Your email address will not be published. Required fields are marked *

Quick Support

image image