{"id":2032,"date":"2019-06-19T09:43:41","date_gmt":"2019-06-19T09:43:41","guid":{"rendered":"https:\/\/prwatech.in\/blog\/?p=2032"},"modified":"2024-03-29T06:50:29","modified_gmt":"2024-03-29T06:50:29","slug":"steps-for-single-node-cluster-with-hadoop-on-aws","status":"publish","type":"post","link":"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/","title":{"rendered":"Single Node Cluster With Hadoop on AWS"},"content":{"rendered":"<h1><span style=\"color: #ff6600;\">Steps For Single Node Cluster With Hadoop on AWS<\/span><\/h1>\n<p>&nbsp;<\/p>\n<p>Steps For Single Node Cluster With Hadoop on <a href=\"https:\/\/prwatech.in\/aws-training-institute-in-pune\/\">AWS<\/a>, in this Tutorial one, you can learn Steps For Single Node Cluster With Hadoop on AWS. Are you the one who is looking for the best platform which provides information about Steps For Single Node Cluster With Hadoop on AWS? Or the one who is looking forward to taking the advanced Certification Course from India\u2019s Leading <a href=\"https:\/\/prwatech.in\/aws-training-institute-in-pune\/\">AWS Training institute<\/a>? Then you\u2019ve landed on the Right Path.<\/p>\n<p>The Below mentioned Tutorial will help to Understand the detailed information about Steps To Launch An Amazon EC2 Instance, so Just Follow All the Tutorials of India\u2019s Leading <a href=\"https:\/\/prwatech.in\/aws-training-institute-in-pune\/\">Best AWS Training institute<\/a> and Be a Pro AWS Developer.<\/p>\n<p><span style=\"color: #000000;\">To set up a single node cluster with Hadoop in EC2 instance on AWS, we have to first login to EC2 instance in <a href=\"https:\/\/prwatech.in\/aws-training-institute-in-pune\/\">AWS Management<\/a> Console by using Login Id and Password<\/span><\/p>\n<p><span style=\"color: #000000;\">After login to EC2 click on Instances option available at the left side of the dashboard, you will see an instance state in stopped mode<\/span><\/p>\n<p><span style=\"color: #000000;\">Start instance by right-clicking on stopped instance &gt; instance state &gt; start &gt; pop-up will appear asking for to start to click YES<\/span><\/p>\n<p><span style=\"color: #000000;\">After clicking Yes instance state will start and in running state<\/span><\/p>\n<p><span style=\"color: #000000;\">Open PuTTY tool where you saved and run it, PuTTY Configuration window will open as given below:<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2037\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw17.jpg\" alt=\"\" width=\"850\" height=\"703\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw17.jpg 643w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw17-300x248.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<ul>\n<li><span style=\"color: #000000;\">Copy Public DNS address in the Host Name field<\/span><\/li>\n<li><span style=\"color: #000000;\">Expand SSH &gt; click Auth &gt; Browse for .ppk file &gt; click open<\/span><\/li>\n<li><span style=\"color: #000000;\">The pop-up will appear to ask permission click, yes, then you will get in Ubuntu Linux instance window in EC2<\/span><\/li>\n<\/ul>\n<p><strong><span style=\"color: #000000;\">Type ubuntu &gt; Enter<\/span><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2502\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/enter.png\" alt=\"\" width=\"27\" height=\"27\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/enter.png 200w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/enter-150x150.png 150w\" sizes=\"auto, (max-width: 27px) 100vw, 27px\" \/><\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2039\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw18.jpg\" alt=\"\" width=\"850\" height=\"649\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw18.jpg 639w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw18-300x229.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p><span style=\"color: #000000;\">Now start to set up a single node cluster by downloading <strong>Jdk 1.8.0 <\/strong>and <strong>Hadoop 2.9.0<\/strong> version<\/span><\/p>\n<h3><strong><span style=\"color: #000000;\">Log in as the root user to install base packages<\/span><\/strong><\/h3>\n<p><strong>sudo su<\/strong><\/p>\n<p><strong>sudo apt-get update<\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2040\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw19.jpg\" alt=\"\" width=\"850\" height=\"208\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw19.jpg 502w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw19-300x74.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p>Type exit from root user in the terminal and Enter\u00a0<img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2502\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/enter.png\" alt=\"\" width=\"27\" height=\"27\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/enter.png 200w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/enter-150x150.png 150w\" sizes=\"auto, (max-width: 27px) 100vw, 27px\" \/><\/p>\n<p><strong>Download<span style=\"color: #000000;\"> jdk 1.8.0 <\/span>using<span style=\"color: #000000;\"> wget <\/span>command<\/strong><\/p>\n<p>wget -c &#8211;header &#8220;Cookie: oraclelicense=accept-secure backup-cookie&#8221; http:\/\/<a href=\"http:\/\/download.oracle.com\/otn-pub\/java\/jdk\/8u131-b11\/d54c1d3a095b4ff2b6607d096fa80163\/jdk-8u131-linux-x64.tar.gz\" target=\"_blank\" rel=\"noopener\" data-saferedirecturl=\"https:\/\/www.google.com\/url?q=http:\/\/download.oracle.com\/otn-pub\/java\/jdk\/8u131-b11\/d54c1d3a095b4ff2b6607d096fa80163\/jdk-8u131-linux-x64.tar.gz&amp;source=gmail&amp;ust=1567838433366000&amp;usg=AFQjCNF1WJcmMFnikXV9bQY4uym36sq0kw\">download.oracle.com\/<wbr \/>otn-pub\/java\/jdk\/8u131-b11\/<wbr \/>d54c1d3a095b4ff2b6607d096fa801<wbr \/>63\/jdk-8u131-linux-x64.tar.gz<\/a><\/p>\n<p>Untar downloaded file(tar.gz) format by using<\/p>\n<p><span style=\"color: #000000;\"><strong>tar -xvf \u00a0jdk-8u131-linux-x64.tar.gz<\/strong><\/span><\/p>\n<p>Download the latest stable Hadoop using wget from one of the\u00a0Apache mirrors.<\/p>\n<p><strong>wget\u00a0<\/strong>https:\/\/archive.apache.org\/dist\/hadoop\/core\/hadoop-2.9.0\/hadoop-2.9.0.tar.gz<\/p>\n<p>After download untar Hadoop-2.9.0-tar.gz<\/p>\n<p><span style=\"color: #000000;\"><strong>tar -xvf hadoop-2.9.0.tar.gz<\/strong><\/span><\/p>\n<p>Create a dir for Hadoop and jdk using below command :<\/p>\n<p>$<span style=\"color: #000000;\"><strong>sudo mkdir \/usr\/lib\/hadoop<\/strong><\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2044\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw20.jpg\" alt=\"\" width=\"850\" height=\"659\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw20.jpg 649w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw20-300x233.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p>$<strong><span style=\"color: #000000;\">sudo mkdir \/usr\/lib\/jvm<\/span><\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2045\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw21.jpg\" alt=\"\" width=\"850\" height=\"634\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw21.jpg 656w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw21-300x224.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p>Move extracted hadoop and jdk folder to newly created dir :<\/p>\n<p>$<strong><span style=\"color: #000000;\">mv hadoop-2.9.0 \/usr\/lib\/hadoop<\/span><\/strong><\/p>\n<p>$<span style=\"color: #000000;\"><strong>mv jdk-1.8.0_131 \/usr\/lib\/jvm<\/strong><\/span><\/p>\n<p>Set jdk and hadoop path in .bashrc file, which is present at <span style=\"color: #000000;\"><strong>\/home\/ubuntu<\/strong><\/span>\u00a0dir :<\/p>\n<p>$<strong><span style=\"color: #000000;\">ls \u2013al<\/span><\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2046\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw22.jpg\" alt=\"\" width=\"850\" height=\"416\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw22.jpg 639w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw22-300x147.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p>$<strong><span style=\"color: #000000;\">vi .bashrc<\/span><\/strong><\/p>\n<p>After opening .bashrc file press <span style=\"color: #000000;\"><strong>ESC<\/strong><\/span> button<\/p>\n<p>To edit .bashrc file press <strong><span style=\"color: #000000;\">i<\/span><\/strong> to INSERT<\/p>\n<p>After configure .bashrc file press : (colon) and<strong><span style=\"color: #000000;\"> wq!<\/span><\/strong> To save,<span style=\"color: #000000;\"><strong>quit<\/strong><\/span> and overwrite<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2047\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw23.jpg\" alt=\"\" width=\"850\" height=\"132\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw23.jpg 643w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw23-300x47.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p>Execute .bashrc using below command:<\/p>\n<p><strong>$<span style=\"color: #000000;\">. .bashrc<\/span><\/strong><\/p>\n<p>Check java and hadoop version :<\/p>\n<p>$<span style=\"color: #000000;\"><strong>java -version<\/strong><\/span><\/p>\n<p>$<strong><span style=\"color: #000000;\">hadoop version<\/span><\/strong><\/p>\n<p><strong>Go to hadoop sbin dir :<\/strong><\/p>\n<p>$<strong><span style=\"color: #000000;\">cd $HADOOP_HOME\/sbin<\/span><\/strong><\/p>\n<p>$<strong><span style=\"color: #000000;\">ls \u2013lrt<\/span><\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2048\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw24.jpg\" alt=\"\" width=\"850\" height=\"831\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw24.jpg 570w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw24-300x293.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p><span style=\"color: #000000;\">Setting a hadoop environment for passwordless ssh access. Passwordless SSH Configuration is a mandatory installation requirement for a single node cluster. However, it is more useful in a multi-node cluster environment.<\/span><\/p>\n<h3>Enter command $<strong><span style=\"color: #000000;\">ssh-keygen -t rsa<\/span><\/strong><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2049\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw25.jpg\" alt=\"\" width=\"850\" height=\"489\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw25.jpg 615w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw25-300x173.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<h3>$<strong><span style=\"color: #000000;\"> ls \u2013alr<\/span><\/strong><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2050\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw26.jpg\" alt=\"\" width=\"850\" height=\"424\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw26.jpg 632w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw26-300x150.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<h3>$<strong><span style=\"color: #000000;\">ls \u2013lrt<\/span><\/strong><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2051\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw27.jpg\" alt=\"\" width=\"850\" height=\"169\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw27.jpg 504w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw27-300x60.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<h3><span style=\"color: #000000;\">cat .ssh\/id_rsa.pub &gt;&gt; .ssh\/authorized_keys<\/span><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2052\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw28.jpg\" alt=\"\" width=\"850\" height=\"79\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw28.jpg 572w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw28-300x28.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<h3><span style=\"color: #000000;\">ssh-copy-id -i .ssh\/id_rsa.pub ubuntu@localhost<\/span><\/h3>\n<ol>\n<li>\n<h3><strong>check password less ssh access to localhost<\/strong><\/h3>\n<\/li>\n<\/ol>\n<p>$ <strong><span style=\"color: #000000;\">ssh localhost<\/span><\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2053\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw29.jpg\" alt=\"\" width=\"850\" height=\"769\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw29.jpg 577w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw29-300x271.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p><span style=\"color: #000000;\">Exit from inner localhost shell by using below command:<\/span><\/p>\n<p><span style=\"color: #000000;\"><strong>\u00a0<\/strong><strong>$exit<\/strong><\/span><\/p>\n<p><span style=\"color: #000000;\">Set the Hadoop config files for a single node cluster. We need to set the below files in order to Hadoop a single node cluster to function properly.<\/span><\/p>\n<p><span style=\"color: #000000;\"><strong>\u2666\u00a0core-site.xml<\/strong><\/span><\/p>\n<p><strong><span style=\"color: #000000;\">hadoop-env.sh<\/span><\/strong><\/p>\n<p><strong><span style=\"color: #000000;\">\u2666\u00a0yarn-site.xml<\/span><\/strong><\/p>\n<p><strong><span style=\"color: #000000;\">hdfs-site.xml<\/span><\/strong><\/p>\n<p><strong><span style=\"color: #000000;\">\u2666\u00a0mapred-site.xml<\/span><\/strong><\/p>\n<p><span style=\"color: #000000;\">Create a directory where the Hadoop will store its data. We will set this directory path in hdfs-site.<\/span><\/p>\n<h3>$<strong><span style=\"color: #000000;\">mkdir\u00a0 hadoopdata<\/span><\/strong><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2054\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw30.jpg\" alt=\"\" width=\"850\" height=\"114\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw30.jpg 680w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw30-300x40.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p>To Configuration in core-site.xml<\/p>\n<p><strong>The command<\/strong>\u00a0is $ <strong><span style=\"color: #000000;\">vi core-site.xml<\/span><\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2055\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw31.jpg\" alt=\"\" width=\"850\" height=\"342\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw31.jpg 412w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw31-300x121.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<h3><strong><span style=\"color: #000000;\">vi hadoop-env.sh<\/span><\/strong><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2056\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw32.jpg\" alt=\"\" width=\"850\" height=\"332\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw32.jpg 697w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw32-300x117.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<h3>$ <strong><span style=\"color: #000000;\">vi yarn-site.xml<\/span><\/strong><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2057\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw33.jpg\" alt=\"\" width=\"850\" height=\"396\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw33.jpg 674w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw33-300x140.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<h3><strong><span style=\"color: #000000;\">vi hdfs-site.xml<\/span><\/strong><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2058\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw34.jpg\" alt=\"\" width=\"850\" height=\"546\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw34.jpg 700w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw34-300x193.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p><span style=\"color: #0000ff;\">NOTE: Make mapred-site.xml file from mapred-site.xml.template.<\/span><\/p>\n<h3>$ <strong><span style=\"color: #000000;\">mapred-site.xml<\/span><\/strong><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2059\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw35.jpg\" alt=\"\" width=\"850\" height=\"521\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw35.jpg 463w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw35-300x184.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p><span style=\"color: #000000;\"><strong>\u25ba\u00a0<\/strong>Formatting the HDFS file system via NameNode (after installing Hadoop, for the\u00a0first time we have to format the HDFS file system to make it work)<\/span><\/p>\n<p><span style=\"color: #000000;\">Enter the command as given below:<\/span><br \/>\n<span style=\"color: #00ccff;\"><strong><span style=\"color: #000000;\">$<\/span><span style=\"color: #000000;\">hadoop namenode \u2013format<\/span><\/strong><\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2060\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw36.jpg\" alt=\"\" width=\"850\" height=\"370\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw36.jpg 786w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw36-300x131.jpg 300w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw36-768x334.jpg 768w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2061\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw37.jpg\" alt=\"\" width=\"850\" height=\"317\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw37.jpg 791w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw37-300x112.jpg 300w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw37-768x286.jpg 768w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p>Go to sbin directory of hadoop :<br \/>\n$<strong><span style=\"color: #000000;\">cd $HADOOP_HOME\/sbin<\/span><\/strong><\/p>\n<p>$<strong><span style=\"color: #000000;\">ls \u2013lrt<\/span><\/strong><\/p>\n<h3><strong>\u25ba<\/strong>Start One service at a time :<\/h3>\n<h3>$<strong><span style=\"color: #000000;\">hadoop-daemon.sh start namenode<\/span><\/strong><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2062\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw38.jpg\" alt=\"\" width=\"850\" height=\"144\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw38.jpg 798w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw38-300x51.jpg 300w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw38-768x130.jpg 768w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p>Start service from HDFS components :$<strong><span style=\"color: #000000;\">start-dfs.sh<\/span><\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2063\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw39.jpg\" alt=\"\" width=\"850\" height=\"269\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw39.jpg 793w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw39-300x95.jpg 300w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw39-768x243.jpg 768w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p>Start all service:<\/p>\n<p>$<strong><span style=\"color: #000000;\">start-all.sh<\/span><\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2064\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw40.jpg\" alt=\"\" width=\"850\" height=\"202\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw40.jpg 790w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw40-300x71.jpg 300w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw40-768x183.jpg 768w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p><strong>\u25ba<\/strong>Check for Hadoop processes \/daemons running on Hadoop with Java Virtual Machine Process Status\u00a0Tool.<\/p>\n<p>$<strong><span style=\"color: #000000;\">jps<\/span><\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-2065\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw41.jpg\" alt=\"\" width=\"850\" height=\"226\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw41.jpg 652w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw41-300x80.jpg 300w\" sizes=\"auto, (max-width: 850px) 100vw, 850px\" \/><\/p>\n<p><strong>Port number of Hadoop-daemons given below:<\/strong><\/p>\n<p><span style=\"color: #0000ff;\">NameNode <\/span>&#8211; localhost: 50070<\/p>\n<p><span style=\"color: #0000ff;\">DataNode<\/span> &#8211; localhost: 50075<\/p>\n<p><span style=\"color: #0000ff;\">SecondaryNameNode<\/span> &#8211; localhost: 50090<\/p>\n<p><span style=\"color: #0000ff;\">ResourceManager<\/span> &#8211;\u00a0 localhost: 8088<\/p>\n<p><strong><span style=\"color: #993300;\"># Single Node Cluster with Hadoop in EC2 Instance on AWS <\/span><\/strong>setup here successfully, now you can Copy file from LFS to HDFS and HDFS to LFS by using various Hadoop commands like:<\/p>\n<p><strong>\u25ba Following commands are used for copy files from local file system to Hadoop distributed file system:<\/strong><\/p>\n<p>$<strong><span style=\"color: #000000;\">hadoop fs -copyFromLocal LFS HDFS<\/span><\/strong>\u00a0\u00a0 Or<\/p>\n<p>$<strong><span style=\"color: #000000;\">hadoop fs -put LFS HDFS<\/span><\/strong><\/p>\n<p><strong><span style=\"color: #000000;\">mkdir dir_name<\/span><\/strong><\/p>\n<p><strong>\u25ba Following commands are used for Hadoop distributed file system (HDFS):<\/strong><\/p>\n<p>$<strong><span style=\"color: #000000;\">hadoop fs -mkdir \/dir_name<\/span><\/strong><\/p>\n<p><strong><span style=\"color: #000000;\">hadoop fs -cp src dst<\/span><\/strong><\/p>\n<p>$<strong><span style=\"color: #000000;\"> hadoop fs -mv src dst<\/span><\/strong><\/p>\n<h5 style=\"text-align: center;\"><span style=\"color: #0000ff;\"><strong># Last but not least, always asks for help!<\/strong><\/span><\/h5>\n","protected":false},"excerpt":{"rendered":"<p>Steps For Single Node Cluster With Hadoop on AWS &nbsp; Steps For Single Node Cluster With Hadoop on AWS, in this Tutorial one, you can learn Steps For Single Node Cluster With Hadoop on AWS. Are you the one who is looking for the best platform which provides information about Steps For Single Node Cluster [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1648,31],"tags":[356],"class_list":["post-2032","post","type-post","status-publish","format-standard","hentry","category-amazon-elastic-compute-cloud-ec2","category-aws","tag-steps-for-single-node-cluster-with-hadoop-on-aws"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.7 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Setting Up Single Node Cluster With Hadoop on AWS | Prwatech<\/title>\n<meta name=\"description\" content=\"Steps For Single Node Cluster With Hadoop on AWS, To set up a single node cluster with Hadoop in EC2 instance on AWS.Enroll today with us\" \/>\n<meta name=\"robots\" content=\"noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Setting Up Single Node Cluster With Hadoop on AWS | Prwatech\" \/>\n<meta property=\"og:description\" content=\"Steps For Single Node Cluster With Hadoop on AWS, To set up a single node cluster with Hadoop in EC2 instance on AWS.Enroll today with us\" \/>\n<meta property=\"og:url\" content=\"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/\" \/>\n<meta property=\"og:site_name\" content=\"Prwatech\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prwatech.in\/\" \/>\n<meta property=\"article:published_time\" content=\"2019-06-19T09:43:41+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-03-29T06:50:29+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw17.jpg\" \/>\n<meta name=\"author\" content=\"Prwatech\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Eduprwatech\" \/>\n<meta name=\"twitter:site\" content=\"@Eduprwatech\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Prwatech\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/\",\"url\":\"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/\",\"name\":\"Setting Up Single Node Cluster With Hadoop on AWS | Prwatech\",\"isPartOf\":{\"@id\":\"https:\/\/prwatech.in\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw17.jpg\",\"datePublished\":\"2019-06-19T09:43:41+00:00\",\"dateModified\":\"2024-03-29T06:50:29+00:00\",\"author\":{\"@id\":\"https:\/\/prwatech.in\/blog\/#\/schema\/person\/db90baff7744090b2288bbc98fea87f3\"},\"description\":\"Steps For Single Node Cluster With Hadoop on AWS, To set up a single node cluster with Hadoop in EC2 instance on AWS.Enroll today with us\",\"breadcrumb\":{\"@id\":\"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/#primaryimage\",\"url\":\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw17.jpg\",\"contentUrl\":\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw17.jpg\",\"width\":643,\"height\":532},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/prwatech.in\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Single Node Cluster With Hadoop on AWS\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/prwatech.in\/blog\/#website\",\"url\":\"https:\/\/prwatech.in\/blog\/\",\"name\":\"Prwatech\",\"description\":\"Share Ideas, Start Something Good.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/prwatech.in\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/prwatech.in\/blog\/#\/schema\/person\/db90baff7744090b2288bbc98fea87f3\",\"name\":\"Prwatech\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/prwatech.in\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/c00bafc1b04045f31eda917de39891456c44fa47c092b9bb6be0f860a3a30a2f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/c00bafc1b04045f31eda917de39891456c44fa47c092b9bb6be0f860a3a30a2f?s=96&d=mm&r=g\",\"caption\":\"Prwatech\"},\"url\":\"https:\/\/prwatech.in\/blog\/author\/prwatech123\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Setting Up Single Node Cluster With Hadoop on AWS | Prwatech","description":"Steps For Single Node Cluster With Hadoop on AWS, To set up a single node cluster with Hadoop in EC2 instance on AWS.Enroll today with us","robots":{"index":"noindex","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_US","og_type":"article","og_title":"Setting Up Single Node Cluster With Hadoop on AWS | Prwatech","og_description":"Steps For Single Node Cluster With Hadoop on AWS, To set up a single node cluster with Hadoop in EC2 instance on AWS.Enroll today with us","og_url":"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/","og_site_name":"Prwatech","article_publisher":"https:\/\/www.facebook.com\/prwatech.in\/","article_published_time":"2019-06-19T09:43:41+00:00","article_modified_time":"2024-03-29T06:50:29+00:00","og_image":[{"url":"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw17.jpg","type":"","width":"","height":""}],"author":"Prwatech","twitter_card":"summary_large_image","twitter_creator":"@Eduprwatech","twitter_site":"@Eduprwatech","twitter_misc":{"Written by":"Prwatech","Est. reading time":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/","url":"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/","name":"Setting Up Single Node Cluster With Hadoop on AWS | Prwatech","isPartOf":{"@id":"https:\/\/prwatech.in\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/#primaryimage"},"image":{"@id":"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/#primaryimage"},"thumbnailUrl":"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw17.jpg","datePublished":"2019-06-19T09:43:41+00:00","dateModified":"2024-03-29T06:50:29+00:00","author":{"@id":"https:\/\/prwatech.in\/blog\/#\/schema\/person\/db90baff7744090b2288bbc98fea87f3"},"description":"Steps For Single Node Cluster With Hadoop on AWS, To set up a single node cluster with Hadoop in EC2 instance on AWS.Enroll today with us","breadcrumb":{"@id":"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/#primaryimage","url":"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw17.jpg","contentUrl":"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/06\/aw17.jpg","width":643,"height":532},{"@type":"BreadcrumbList","@id":"https:\/\/prwatech.in\/blog\/aws\/amazon-elastic-compute-cloud-ec2\/steps-for-single-node-cluster-with-hadoop-on-aws\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/prwatech.in\/blog\/"},{"@type":"ListItem","position":2,"name":"Single Node Cluster With Hadoop on AWS"}]},{"@type":"WebSite","@id":"https:\/\/prwatech.in\/blog\/#website","url":"https:\/\/prwatech.in\/blog\/","name":"Prwatech","description":"Share Ideas, Start Something Good.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/prwatech.in\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/prwatech.in\/blog\/#\/schema\/person\/db90baff7744090b2288bbc98fea87f3","name":"Prwatech","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/prwatech.in\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/c00bafc1b04045f31eda917de39891456c44fa47c092b9bb6be0f860a3a30a2f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c00bafc1b04045f31eda917de39891456c44fa47c092b9bb6be0f860a3a30a2f?s=96&d=mm&r=g","caption":"Prwatech"},"url":"https:\/\/prwatech.in\/blog\/author\/prwatech123\/"}]}},"_links":{"self":[{"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/posts\/2032","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/comments?post=2032"}],"version-history":[{"count":27,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/posts\/2032\/revisions"}],"predecessor-version":[{"id":11175,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/posts\/2032\/revisions\/11175"}],"wp:attachment":[{"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/media?parent=2032"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/categories?post=2032"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/tags?post=2032"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}