Hadoop High Availability Setup

How to set up a Hadoop Cluster with high availability

Step 1: Download the zookeeper from the browser.

Step 2: Untar the zookeeper using the terminal by a given command.

 Step 3: Download the Hadoop from the browser.

Step 4: Untar Hadoop using the below command.

Step 5: Change the Host Name.

Step 6: Update your system.

Step 7: Install Java

Step 8: Install SSH Server

Step 9: Add Java, Hadoop and Zookeeper path in bashrc file.

Step 10: Update bashrc file.

Step 11: Open core-site.xml inactive name node and add below properties.

Step 12: Open hdfs-site.xml inactive name node and add below properties.

Step 13: Change the directory and go to zookeeper/conf

Step 14 : Copy zoo_sample.cfg to zoo.cfg

Step 15: Edit zoo.cfg file and give the dataDir path

Step 16: Make the Clone of this and rename them as NN2 and DN1.

Step 17: Add each node IP in etc/hosts in every node

Step 18: Generate ssh key and copy in each other.

Step 19: Go to DN1 and open hdfs-site.xml and this property

Step 20: Go to NN1 and in zookeeper metadata directory create one file myid and add number 1.

Step 21: Go to NN2 and in zookeeper metadata directory create one file myid and add number 2.

Step 22: Go to NN2 and in zookeeper metadata directory create one file myid and add number 3

Step 23: Start the journal node in all nodes.

Step 24:  write jps in all node

Step 25: Format name node in NN1

Step 26: Start name node in NN1

Step 27: Go to NN2 and copy metadata from NN1 to NN2

Step 28: Start name node in NN2

Step 29: Now Start Zookeeper service in all three nodes.

Step 30: Run Jps in NN1

Step 31: Kill the NameNode process