Install Hadoop 3.2.1 on Windows 10 Step by Step Guide
Prerequisites:
Hardware requirements:-
RAM: 8 GB or above
Software requirements:-
vmware workstation – https://www.vmware.com/in/products/workstation-pro/workstation-pro-evaluation.html
ubuntu 10.x or above – https://ubuntu.com/download/desktop
Programming languages: Linux
Learn Linux : https://prwatech.in/blog/linux/linux-architecture/
1.Install ubuntu in virtual machine
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image.png)
2. Power on this virtual machine
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-1.png)
3. Open the Terminal
Select As marked in the diagram or follow the arrow
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-2.png)
You will see this
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-3.png)
4. Check your hostname is ubuntu
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-4.png)
5. Set up a single node hadoop cluster
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-5.png)
6. Create a group called hadoop
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-6.png)
7. Create an user called hduser
root@ubuntu:/home/user# sudo adduser hduser
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-28.png)
It will ask password two times followed by some details, press enter and yes.
Give password which you can remember or else enter default password as “password”
8. Add hduser to hadoop group
# sudo adduser hduser hadoop
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-29.png)
9. Add the ‘hduser’ to ‘sudoers’ so that hduser can do some admin task
#sudo visudo
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-31.png)
It will open another file where you have to enter the below command:
Save -> Ctrl+s and then Exit -> ctrl+x
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-27.png)
10. Logout of your system and login as a hduser Add hduser to hadoop group
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-12.png)
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-13.png)
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-14.png)
Password: password you have set earlier
11. Open the Terminal
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-15.png)
12. Configure ssh
#sudo apt-get install openssh-server
“Enter password and Y to continue”
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-16.png)
13. Generate ‘SSH’ for communication
#ssh-keygen
Press ‘enter’ whenever it asks.
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-32.png)
14. Copy public key to Authorized key file & edit the permission
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-18.png)
15. Give permission of the Authorized key
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-34.png)
16. Start ‘SSH’
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-35.png)
17. Test your ‘SSH’ connectivity
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-36.png)
18. Disable IPV6
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-37.png)
Enter “i” for –INSERT—mode and pres “enter” at the end of last line.
And write the following lines at bottom after # disable ipv6:
“net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1“
To exit press “esc” then “wq” then “enter”
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-38.png)
19. Check if ipv6 is disabled
If it gives ‘1’ as output then its disabled
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-24.png)
20. Now install hadoop 3.2.1 from the following website:
https://archive.apache.org/dist/hadoop/common/hadoop-3.2.1/
Download hadoop 3.2.1.tar.gz and save it to hduser/desktop
21. Move the above file to /usr/local/
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-39.png)
22. Then go to local directory
#cd /usr/local
and then ls
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-26.png)
23. Now untar(unzip) the file using the command: $ sudo tar –xvf hadoop-3.2.1.tar.gz
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-41.png)
24. Remove the tar file now
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-42.png)
25. Now lets create a shortcut name of hadoop-3.2.1 to hadoop
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-44.png)
26. Check the file hadoop by entering ‘ls’ and follow below commands
![](https://prwatech.in/blog/wp-content/uploads/2021/03/h9-1-1024x400.png)
27. Change the ownership of hadoop-3.2.1 to hduser group
![](https://prwatech.in/blog/wp-content/uploads/2021/03/h10.png)
28. Now give all the permissions to hadoop-3.2.1 folder
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-45.png)
29. Edit hadoop-env.sh file
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-46.png)
Go to bottom of the file press ‘i’ And add those lines at end which are under red circle
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-47.png)
#export JAVA_HOME={JAVA_HOME}
30. Update HOME ~/.bashrc
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-48.png)
Now go to the bottom of the file press ‘i’ and then enter the lines
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-49.png)
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-50.png)
31. Update yarn-site.xml
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-51.png)
# Enter the lines under <configuration>
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-52.png)
32. Update core-site.xml
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-53.png)
Enter the following lines under <configuration> files
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-54.png)
33.Create the above temp folder and give appropriate permission using following commands
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-55.png)
34. update mapred-site.xml
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-56.png)
Enter the following lines under <configuration> files
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-57.png)
35. Create a temporary directory which will be used as base location for DFSAlso create the directory and set the required ownership and permissions by adding following three lines
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-58.png)
36. Update hdfs-site.xml
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-59.png)
Enter the following lines under <configuration> files
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-60.png)
37. Format Namenode
Close the terminal and run the command in the new terminal
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-61.png)
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-62.png)
38. Start your single-node cluster
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-63.png)
39. Type “jps”
![](https://prwatech.in/blog/wp-content/uploads/2021/03/image-64.png)
If you get the above nodes after writing ‘jps’ it means you have successfully installed hadoop-3.2.1 on single node cluster.