{"id":1354,"date":"2026-10-12T12:12:08","date_gmt":"2026-10-12T12:12:08","guid":{"rendered":"https:\/\/prwatech.in\/blog\/?p=1354"},"modified":"2026-10-14T15:40:58","modified_gmt":"2026-10-14T15:40:58","slug":"hdfs-questions-and-answers","status":"publish","type":"post","link":"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/","title":{"rendered":"HDFS Interview Questions and Answers"},"content":{"rendered":"<h1><strong><b>Most Asked HDFS Interview Questions and Answers<\/b><\/strong><\/h1>\n<p>HDFS interviews test whether you understand how a distributed file system stores data safely and serves it quickly, and whether you can reason about failures. This guide gives you 45 HDFS interview questions and answers, from architecture to scenario questions, for freshers and experienced candidates.<\/p>\n<p>Every answer starts with a short, direct response you can say aloud, followed by a table, a command or a small calculation you can run. The page uses current Hadoop 3 details, such as the new port numbers, erasure coding and multiple standby NameNodes, and corrects the common mistake of calling the Secondary NameNode a backup.<\/p>\n<p>The guide has six parts: 8 on HDFS architecture, 8 on the read and write path and replication, 8 on HDFS commands, 10 on high availability, federation and advanced features, 5 on capacity and performance and 6 scenario questions. For the wider platform, read the main Hadoop interview page.<\/p>\n<h3><strong><b>Key points<\/b><\/strong><\/h3>\n<ul>\n<li>45 questions in six parts, from architecture to scenarios.<\/li>\n<li>Read and write paths explained step by step.<\/li>\n<li>Current commands and Hadoop 3 details.<\/li>\n<li>High availability, federation, erasure coding, snapshots and quotas.<\/li>\n<li>Executed capacity and memory calculations.<\/li>\n<\/ul>\n<h3><strong><b>In this guide<\/b><\/strong><\/h3>\n<div style=\"border: 1px solid #d9e0ea; border-radius: 8px; overflow-x: auto; margin: 20px 0; -webkit-overflow-scrolling: touch;\">\n<table style=\"width: 100%; min-width: 560px; border-collapse: collapse; font-family: Arial,Helvetica,sans-serif; font-size: 16px; line-height: 1.6; color: #1f2937;\">\n<caption style=\"caption-side: top; text-align: left; padding: 14px 16px; font-weight: bold; font-size: 17px; color: #0f2a4a;\">What this guide covers<\/caption>\n<thead>\n<tr>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38;\" scope=\"col\">Section<\/th>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38; width: 150px;\" scope=\"col\">Questions<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a;\" scope=\"row\">HDFS Architecture (Freshers)<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0; white-space: nowrap;\">Q1 to Q8<\/td>\n<\/tr>\n<tr style=\"background: #f5f8fc;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a;\" scope=\"row\">Read and Write Path and Replication (Intermediate)<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0; white-space: nowrap;\">Q9 to Q16<\/td>\n<\/tr>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a;\" scope=\"row\">HDFS Commands (Freshers and Intermediate)<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0; white-space: nowrap;\">Q17 to Q24<\/td>\n<\/tr>\n<tr style=\"background: #f5f8fc;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a;\" scope=\"row\">High Availability, Federation and Advanced Features (Experienced)<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0; white-space: nowrap;\">Q25 to Q34<\/td>\n<\/tr>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a;\" scope=\"row\">Capacity and Performance (Experienced)<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0; white-space: nowrap;\">Q35 to Q39<\/td>\n<\/tr>\n<tr style=\"background: #f5f8fc;\">\n<th style=\"text-align: left; padding: 12px 16px; color: #0f2a4a;\" scope=\"row\">Scenario Questions (Experienced)<\/th>\n<td style=\"padding: 12px 16px; white-space: nowrap;\">Q40 to Q45<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<h2><strong><b>HDFS Architecture Interview Questions for Freshers<\/b><\/strong><\/h2>\n<p>HDFS interviews begin with the architecture. Give crisp answers on the NameNode, DataNodes and blocks, because every later question builds on them. For a wider view of Hadoop, see the main Hadoop interview page.<\/p>\n<h3><strong><b>Q1. What is HDFS, and what is it designed for?<\/b><\/strong><\/h3>\n<p>The Hadoop Distributed File System stores very large files across many machines, with each file split into blocks that are copied to several nodes. It is designed for high throughput on large sequential reads, for streaming access, for hardware that fails often and for a write once, read many pattern. It is not designed for low latency or many small files.<\/p>\n<h3><strong><b>Q2. Describe the HDFS architecture.<\/b><\/strong><\/h3>\n<p>HDFS has a master and worker design. The NameNode holds the namespace and the map from files to blocks. DataNodes store the blocks on local disks, and send heartbeats and block reports to the NameNode. Clients ask the NameNode where data is, then read and write the data directly with the DataNodes.<\/p>\n<div style=\"border: 1px solid #d9e0ea; border-radius: 8px; overflow-x: auto; margin: 20px 0; -webkit-overflow-scrolling: touch;\">\n<table style=\"width: 100%; min-width: 700px; border-collapse: collapse; font-family: Arial,Helvetica,sans-serif; font-size: 16px; line-height: 1.6; color: #1f2937;\">\n<caption style=\"caption-side: top; text-align: left; padding: 14px 16px; font-weight: bold; font-size: 17px; color: #0f2a4a;\">Parts of an HDFS cluster<\/caption>\n<thead>\n<tr>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38; width: 160px;\" scope=\"col\">Part<\/th>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38;\" scope=\"col\">Job<\/th>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38; width: 200px;\" scope=\"col\">Holds data blocks?<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Client<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Reads and writes files through the HDFS API<\/td>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">No<\/td>\n<\/tr>\n<tr style=\"background: #f5f8fc;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">NameNode<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Namespace, permissions, block map<\/td>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">No<\/td>\n<\/tr>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">DataNode<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Stores and serves blocks<\/td>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Yes<\/td>\n<\/tr>\n<tr style=\"background: #f5f8fc;\">\n<th style=\"text-align: left; padding: 12px 16px; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">JournalNode<\/th>\n<td style=\"padding: 12px 16px;\">Shares the edit log in an HA setup<\/td>\n<td style=\"padding: 12px 16px;\">No<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>&nbsp;<\/p>\n<h3><strong><b>Q3. What is a block, and what is the default block size?<\/b><\/strong><\/h3>\n<p>A block is the unit in which HDFS splits and stores a file. The default size is 128 megabytes, controlled by dfs.blocksize, and you can set it per file. A file smaller than a block takes only its actual size on disk, but it still counts as one block object in NameNode memory.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">python<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">import math\r\n\r\nsizes_mb = [1, 128, 129, 1000]\r\n\r\nfor s in sizes_mb:\r\n    print(f\"{s} MB file -&gt; {math.ceil(s \/ 128)} block(s)\")<\/pre>\n<div style=\"background: #1f2937; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">output<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #ffffff; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">1 MB file -&gt; 1 block(s)\r\n128 MB file -&gt; 1 block(s)\r\n129 MB file -&gt; 2 block(s)\r\n1000 MB file -&gt; 8 block(s)<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<h3><strong><b>Q4. What does the NameNode store, and where?<\/b><\/strong><\/h3>\n<p>The NameNode keeps the whole namespace and the block map in memory for fast lookups. For durability it writes every change to an edit log on disk, and periodically saves a checkpoint of the namespace as an fsimage file. Block locations are not saved on disk. They are rebuilt from the block reports sent by DataNodes after a restart.<\/p>\n<p><strong><b>What the interviewer is listening for: <\/b><\/strong><em><i>You say that block locations are rebuilt from block reports and not stored on disk.<\/i><\/em><\/p>\n<h3><strong><b>Q5. What are the fsimage and the edit log?<\/b><\/strong><\/h3>\n<p>The fsimage is a snapshot of the file system namespace at a point in time. The edit log is a record of every change made since that snapshot. On startup, the NameNode loads the fsimage and replays the edit log to reach the current state. A periodic checkpoint merges the edits into a new fsimage, which keeps the log short.<\/p>\n<h3><strong><b>Q6. What does the Secondary NameNode do?<\/b><\/strong><\/h3>\n<p>The Secondary NameNode periodically downloads the fsimage and edit log, merges them into a new fsimage and sends it back. This keeps the edit log small and shortens restart time. It is not a failover node and cannot take over when the NameNode fails. In a high availability setup, the standby NameNode does this work instead.<\/p>\n<p><strong><b>What the interviewer is listening for: <\/b><\/strong><em><i>You say clearly that it is a checkpointing helper and not a backup.<\/i><\/em><\/p>\n<h3><strong><b>Q7. What are heartbeats and block reports?<\/b><\/strong><\/h3>\n<p>Each DataNode sends a heartbeat to the NameNode every three seconds by default to say it is alive, and the reply can carry commands such as replicate a block. A block report lists all the blocks a DataNode holds, and is sent at start and then periodically. A DataNode silent for about ten and a half minutes is marked dead.<\/p>\n<div style=\"border: 1px solid #d9e0ea; border-radius: 8px; overflow-x: auto; margin: 20px 0; -webkit-overflow-scrolling: touch;\">\n<table style=\"width: 100%; min-width: 700px; border-collapse: collapse; font-family: Arial,Helvetica,sans-serif; font-size: 16px; line-height: 1.6; color: #1f2937;\">\n<caption style=\"caption-side: top; text-align: left; padding: 14px 16px; font-weight: bold; font-size: 17px; color: #0f2a4a;\">DataNode heartbeats and block reports<\/caption>\n<thead>\n<tr>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38; width: 190px;\" scope=\"col\">Message<\/th>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38; width: 210px;\" scope=\"col\">Default interval<\/th>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38;\" scope=\"col\">Purpose<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Heartbeat<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Every 3 seconds<\/td>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Shows the DataNode is alive, carries commands<\/td>\n<\/tr>\n<tr style=\"background: #f5f8fc;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Block report<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Every 6 hours<\/td>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Full list of blocks on the DataNode<\/td>\n<\/tr>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Dead node timeout<\/th>\n<td style=\"padding: 12px 16px;\">About 10.5 minutes<\/td>\n<td style=\"padding: 12px 16px;\">Then re replication starts<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>&nbsp;<\/p>\n<h3><strong><b>Q8. What are the default ports of the HDFS services?<\/b><\/strong><\/h3>\n<p>In Hadoop 3 the NameNode web interface is on port 9870, and its file system RPC port is commonly 8020 or 9000 depending on configuration. The DataNode web interface is on 9864 and the Secondary NameNode on 9868. Hadoop 2 used different numbers, such as 50070 for the NameNode, so say which version you mean.<\/p>\n<p><strong><b>What the interviewer is listening for: <\/b><\/strong><em><i>You know the Hadoop 3 ports changed from Hadoop 2.<\/i><\/em><\/p>\n<h2><strong><b>Read and Write Path and Replication Interview Questions for Intermediate<\/b><\/strong><\/h2>\n<p>Expect to be asked to explain a write or read step by step, so rehearse them.<\/p>\n<h3><strong><b>Q9. How does HDFS write a file?<\/b><\/strong><\/h3>\n<p>The client asks the NameNode to create the file, and the NameNode checks permissions and returns a list of DataNodes for the first block. The client streams data in packets to the first DataNode, which forwards it to the second, and then to the third, in a pipeline. Acknowledgements flow back, and the process repeats for each block.<\/p>\n<ul>\n<li>The client contacts the NameNode to create the file entry.<\/li>\n<li>For each block, the NameNode chooses the DataNodes for the pipeline.<\/li>\n<li>Packets go through the pipeline and acknowledgements return to the client.<\/li>\n<li>The client tells the NameNode when the file is closed.<\/li>\n<\/ul>\n<h3><strong><b>Q10. How does HDFS read a file?<\/b><\/strong><\/h3>\n<p>The client asks the NameNode for the block locations of the file. The NameNode returns, for each block, the DataNodes that hold it, sorted by distance from the client. The client then reads each block directly from the nearest DataNode. The NameNode is not in the data path, which keeps it from becoming a bottleneck.<\/p>\n<h3><strong><b>Q11. What is rack awareness, and what is the default replica placement?<\/b><\/strong><\/h3>\n<p>Rack awareness means HDFS knows which rack each DataNode sits in, through a topology script or mapping. With replication factor 3, the default policy puts the first replica on the writer node, the second on a node in a different rack, and the third on another node in that second rack. It balances safety with network cost.<\/p>\n<div style=\"border: 1px solid #d9e0ea; border-radius: 8px; overflow-x: auto; margin: 20px 0; -webkit-overflow-scrolling: touch;\">\n<table style=\"width: 100%; min-width: 640px; border-collapse: collapse; font-family: Arial,Helvetica,sans-serif; font-size: 16px; line-height: 1.6; color: #1f2937;\">\n<caption style=\"caption-side: top; text-align: left; padding: 14px 16px; font-weight: bold; font-size: 17px; color: #0f2a4a;\">Default HDFS replica placement<\/caption>\n<thead>\n<tr>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38; width: 150px;\" scope=\"col\">Replica<\/th>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38;\" scope=\"col\">Placement<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">First<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">The node running the writer, or a random node<\/td>\n<\/tr>\n<tr style=\"background: #f5f8fc;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Second<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">A node on a different rack<\/td>\n<\/tr>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Third<\/th>\n<td style=\"padding: 12px 16px;\">A different node on the same rack as the second<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>&nbsp;<\/p>\n<p><strong><b>What the interviewer is listening for: <\/b><\/strong><em><i>You can name the three placements and say why: one rack failure never loses all copies.<\/i><\/em><\/p>\n<h3><strong><b>Q12. What happens if a DataNode fails during a write?<\/b><\/strong><\/h3>\n<p>The pipeline is rebuilt without the failed node, the client continues with the remaining DataNodes, and the NameNode later notices the block is under replicated and schedules a new copy. The write does not fail unless too few nodes remain to meet the minimum number of replicas.<\/p>\n<h3><strong><b>Q13. What happens when a replica is corrupted?<\/b><\/strong><\/h3>\n<p>HDFS keeps a checksum for each chunk of a block. A client or DataNode that finds a mismatch while reading reports the bad replica to the NameNode and reads another copy. The NameNode then arranges a new healthy copy from a good replica and removes the corrupt one. DataNodes also scan their blocks in the background.<\/p>\n<h3><strong><b>Q14. How do you change the replication factor of a file?<\/b><\/strong><\/h3>\n<p>Use the setrep command, or set dfs.replication for new files. Increasing it makes the NameNode schedule extra copies, and decreasing it makes it delete extras. The factor is a per file setting, so important or hot data can have a higher factor than cold data.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">bash<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">hdfs dfs -setrep -w 5 \/data\/hot\/lookup.csv\r\nhdfs fsck \/data\/hot\/lookup.csv -files -blocks -locations<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<h3><strong><b>Q15. How is a block stored on a DataNode?<\/b><\/strong><\/h3>\n<p>Each block replica is saved as a file on the local file system of the DataNode, together with a small metadata file holding its checksums. DataNodes spread blocks over all their configured disks. HDFS does not alter blocks after they are written, which makes caching, replication and recovery simple.<\/p>\n<h3><strong><b>Q16. Can you modify a file in HDFS?<\/b><\/strong><\/h3>\n<p>HDFS is write once and read many. You can create, delete, rename, append to and truncate a file, but you cannot change bytes in the middle. To edit data, rewrite the file or use a layer on top, such as Hive ACID tables, HBase or a table format such as Iceberg or Delta, which manage the changes for you.<\/p>\n<h2><strong><b>HDFS Commands Interview Questions for Freshers and Intermediate<\/b><\/strong><\/h2>\n<p>Commands are tested in practical rounds. Note that the current form is hdfs dfs, and the older hadoop fs form works but is generic.<\/p>\n<h3><strong><b>Q17. What are the most used HDFS file commands?<\/b><\/strong><\/h3>\n<p>Use hdfs dfs with a subcommand: ls to list, mkdir to create a directory, put or copyFromLocal to upload, get or copyToLocal to download, cat to show content, rm to delete, du for sizes, and df for capacity. The syntax follows Linux commands, but all paths refer to HDFS.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">bash<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">hdfs dfs -mkdir -p \/user\/asha\/input\r\nhdfs dfs -put sales.csv \/user\/asha\/input\/\r\nhdfs dfs -ls \/user\/asha\/input\r\nhdfs dfs -cat \/user\/asha\/input\/sales.csv | head\r\nhdfs dfs -du -h \/user\/asha\r\nhdfs dfs -get \/user\/asha\/input\/sales.csv .\/sales_copy.csv\r\nhdfs dfs -rm -r \/user\/asha\/input<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<h3><strong><b>Q18. What is the difference between put, copyFromLocal and moveFromLocal?<\/b><\/strong><\/h3>\n<p>The put and copyFromLocal commands both copy a local file into HDFS and keep the local copy. The copyFromLocal command only accepts a local source, while put can also read from standard input. The moveFromLocal command copies the file and then deletes the local copy.<\/p>\n<h3><strong><b>Q19. How do you check the health of the file system?<\/b><\/strong><\/h3>\n<p>Run hdfs fsck on a path. It reports missing, corrupt and under replicated blocks and, with options, lists files, blocks and their locations. Run hdfs dfsadmin with the report option for capacity and the state of each DataNode. The fsck command is read only and reports problems without repairing them.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">bash<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">hdfs fsck \/ -files -blocks -locations\r\nhdfs dfsadmin -report<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<p>Note that fsck is its own command: hdfs fsck, not a subcommand of dfs.<\/p>\n<h3><strong><b>Q20. How do you format the NameNode, and when should you?<\/b><\/strong><\/h3>\n<p>Run hdfs namenode with the format option, once, when you create a new cluster. It initialises the namespace directories. Never format an existing cluster, because it erases the metadata and every file in HDFS becomes unreachable. The old hadoop namenode command is deprecated.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">bash<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">hdfs namenode -format<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<h3><strong><b>Q21. How does the trash work in HDFS?<\/b><\/strong><\/h3>\n<p>If trash is enabled with fs.trash.interval, set to a number of minutes, deleted files move to a hidden trash folder in the user home directory and are purged after that interval. The setting is off by default in stock Hadoop, so a delete is permanent unless an administrator enabled it. The skipTrash option bypasses it.<\/p>\n<h3><strong><b>Q22. How do you copy data between clusters?<\/b><\/strong><\/h3>\n<p>Use DistCp, a MapReduce based tool that copies large amounts of data in parallel between clusters or within one. The update and delete options make repeated copies incremental. It is the standard tool for backups, migrations and moving data to cloud storage.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">bash<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">hadoop distcp -update hdfs:\/\/nn1:8020\/data\/sales hdfs:\/\/nn2:8020\/backup\/sales<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<h3><strong><b>Q23. What is safe mode?<\/b><\/strong><\/h3>\n<p>Safe mode is a read only state the NameNode enters at start up while it collects block reports. It leaves when enough blocks, by default 99.9 percent, have reached their minimum replication. In safe mode, you cannot write or delete. An administrator can check or leave it manually with the dfsadmin command, though that should be done with care.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">bash<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">hdfs dfsadmin -safemode get\r\nhdfs dfsadmin -safemode leave<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<h3><strong><b>Q24. How do HDFS permissions work?<\/b><\/strong><\/h3>\n<p>HDFS uses a POSIX like model with owner, group and others, and read, write and execute bits. The execute bit on a file has no meaning, and on a directory it allows access to children. A umask sets default permissions, and access control lists add finer rules. In secure clusters, identity comes from Kerberos.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">bash<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">hdfs dfs -chmod 750 \/user\/asha\r\nhdfs dfs -chown asha:analytics \/user\/asha\r\nhdfs dfs -setfacl -m user:ravi:r-x \/user\/asha<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<h2><strong><b>High Availability, Federation and Advanced Features Interview Questions for Experienced<\/b><\/strong><\/h2>\n<p>Experienced interviews dig into availability, scale and the features added in Hadoop 3.<\/p>\n<h3><strong><b>Q25. How does NameNode high availability work?<\/b><\/strong><\/h3>\n<p>Two or more NameNodes run, one active and the others standby. They share the edit log through a quorum of JournalNodes, so a standby stays almost up to date, and DataNodes send block reports to all of them. A ZooKeeper based failover controller detects failure and promotes a standby. Fencing prevents two active nodes.<\/p>\n<p><strong><b>What the interviewer is listening for: <\/b><\/strong><em><i>You name JournalNodes, ZooKeeper failover controllers and fencing.<\/i><\/em><\/p>\n<h3><strong><b>Q26. What is split brain, and how does HDFS prevent it?<\/b><\/strong><\/h3>\n<p>Split brain means two NameNodes both believe they are active and accept writes, which would corrupt the namespace. HDFS prevents it with fencing: the JournalNodes accept edits from only one writer at a time, and the failover controller can also fence the old active node by shutting it down or cutting its access.<\/p>\n<h3><strong><b>Q27. What is HDFS federation?<\/b><\/strong><\/h3>\n<p>Federation allows several independent NameNodes, each owning a part of the namespace, to share one pool of DataNodes. It scales the number of files beyond one NameNode memory, and separates workloads. Each namespace has its own block pool. Router based federation presents one view over many namespaces.<\/p>\n<h3><strong><b>Q28. What is erasure coding in HDFS?<\/b><\/strong><\/h3>\n<p>Erasure coding, introduced in Hadoop 3, stores data blocks with parity blocks instead of full copies. A common policy, Reed Solomon 6 plus 3, stores 6 data and 3 parity blocks, using 1.5 times the data size against 3 times for triple replication, and survives the loss of any 3 blocks. It suits cold data and costs more to read and recover.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">python<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">data_gb = 600\r\n\r\nprint(\"3x replication:\", data_gb * 3, \"GB\")\r\nprint(\"RS(6,3) erasure coding:\", data_gb * 9 \/ 6, \"GB\")<\/pre>\n<div style=\"background: #1f2937; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">output<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #ffffff; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">3x replication: 1800 GB\r\nRS(6,3) erasure coding: 900.0 GB<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<h3><strong><b>Q29. How do you apply an erasure coding policy?<\/b><\/strong><\/h3>\n<p>Enable the policy on the cluster, then set it on a directory. Files written into that directory afterwards are erasure coded. Existing files keep their replication until they are rewritten. Pick the policy by how many racks or nodes you have, since more parity blocks need more failure domains.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">bash<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">hdfs ec -listPolicies\r\nhdfs ec -enablePolicy -policy RS-6-3-1024k\r\nhdfs ec -setPolicy -path \/data\/cold -policy RS-6-3-1024k<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<h3><strong><b>Q30. What are HDFS snapshots?<\/b><\/strong><\/h3>\n<p>A snapshot is a read only, point in time copy of a directory. It is created fast because it records only the changes made after it, without copying blocks. Snapshots protect against accidental deletes and support backups. A directory must be made snapshottable by an administrator first.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">bash<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">hdfs dfsadmin -allowSnapshot \/data\/sales\r\nhdfs dfs -createSnapshot \/data\/sales before_cleanup\r\nhdfs dfs -ls \/data\/sales\/.snapshot<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<h3><strong><b>Q31. How do quotas work in HDFS?<\/b><\/strong><\/h3>\n<p>A name quota limits the number of files and directories under a directory, and a space quota limits the raw bytes. The space quota counts all replicas, so a file with three replicas uses three times its size. A write that would pass the quota fails.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">python<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">file_gb, replication, quota_gb = 30, 3, 100\r\n\r\nused = file_gb * replication\r\nprint(\"space quota used by one file:\", used, \"GB of\", quota_gb, \"GB\")\r\nprint(\"another file of the same size fits:\", used + file_gb * replication &lt;= quota_gb)<\/pre>\n<div style=\"background: #1f2937; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">output<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #ffffff; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">space quota used by one file: 90 GB of 100 GB\r\nanother file of the same size fits: False<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<h3><strong><b>Q32. What is HDFS transparent encryption?<\/b><\/strong><\/h3>\n<p>HDFS can encrypt data at rest in encryption zones, which are directories whose files are encrypted. Each file has its own data key, which is itself encrypted with a zone key held in an external key management server. Clients encrypt and decrypt, so the data is never seen as plain text by DataNodes or the NameNode.<\/p>\n<h3><strong><b>Q33. What are the storage policies and the balancer?<\/b><\/strong><\/h3>\n<p>Storage policies let you place data on different media, such as hot on disk, warm, cold on archive disks, or on SSD, chosen per directory. The balancer is a tool that moves blocks between DataNodes to even out disk use after nodes are added or after uneven writes. Run it with a threshold, in a quiet period.<\/p>\n<h3><strong><b>Q34. How does HDFS differ from cloud object storage such as S3?<\/b><\/strong><\/h3>\n<p>HDFS has a real directory tree, fast atomic renames, strong consistency and data locality on its own disks, but you manage the servers. Object storage is a managed, durable and cheaper store with no true directories, where a rename is a copy and delete, and access always goes over the network. Modern platforms often separate compute from storage.<\/p>\n<div style=\"border: 1px solid #d9e0ea; border-radius: 8px; overflow-x: auto; margin: 20px 0; -webkit-overflow-scrolling: touch;\">\n<table style=\"width: 100%; min-width: 700px; border-collapse: collapse; font-family: Arial,Helvetica,sans-serif; font-size: 16px; line-height: 1.6; color: #1f2937;\">\n<caption style=\"caption-side: top; text-align: left; padding: 14px 16px; font-weight: bold; font-size: 17px; color: #0f2a4a;\">HDFS vs object storage<\/caption>\n<thead>\n<tr>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38; width: 170px;\" scope=\"col\">Point<\/th>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38;\" scope=\"col\">HDFS<\/th>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38;\" scope=\"col\">Object storage<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Management<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">You run the cluster<\/td>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Managed service<\/td>\n<\/tr>\n<tr style=\"background: #f5f8fc;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Rename<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Atomic and fast<\/td>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Copy then delete<\/td>\n<\/tr>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Data locality<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Yes<\/td>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">No, network access<\/td>\n<\/tr>\n<tr style=\"background: #f5f8fc;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Cost model<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Hardware you own, three copies<\/td>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Pay for what you store<\/td>\n<\/tr>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Best for<\/th>\n<td style=\"padding: 12px 16px;\">On premise clusters, high throughput local reads<\/td>\n<td style=\"padding: 12px 16px;\">Cloud data lakes<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>&nbsp;<\/p>\n<h2><strong><b>Capacity and Performance Interview Questions for Experienced Professionals<\/b><\/strong><\/h2>\n<p>Interviewers like numbers. Be ready to size a cluster and to explain what limits it.<\/p>\n<h3><strong><b>Q35. How do you estimate the HDFS capacity you need?<\/b><\/strong><\/h3>\n<p>Start with the data size, multiply by the replication factor, then add headroom for growth and for temporary and intermediate data, commonly 20 to 30 percent of disk kept free. For example, 100 terabytes of data with three copies needs 300 terabytes, and about 400 terabytes of raw disk once headroom is included.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">python<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">data_tb, replication, free_fraction = 100, 3, 0.25\r\n\r\nneeded = data_tb * replication \/ (1 - free_fraction)\r\nprint(\"raw disk to buy, TB:\", round(needed))<\/pre>\n<div style=\"background: #1f2937; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">output<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #ffffff; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">raw disk to buy, TB: 400<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<h3><strong><b>Q36. What limits the size of an HDFS cluster?<\/b><\/strong><\/h3>\n<p>The main limit is the NameNode memory, because it holds every file and block object, so the number of files is limited more than the total bytes. Large blocks and fewer, larger files reduce the load. Beyond that, federation scales the namespace, and the network and the number of DataNodes limit throughput.<\/p>\n<h3><strong><b>Q37. How do you improve HDFS read and write performance?<\/b><\/strong><\/h3>\n<p>Use larger files and columnar formats, keep enough DataNodes and disks per node for parallel reads, and use short circuit local reads so a client on the same node reads the block directly from disk. Compress with a splittable or columnar friendly codec, and keep the network non blocking within and between racks.<\/p>\n<h3><strong><b>Q38. What is short circuit local read?<\/b><\/strong><\/h3>\n<p>Normally a client reads a block through the DataNode over a socket. With short circuit local reads, a client on the same machine as the block opens the block file directly through a domain socket, skipping the DataNode process. This reduces latency and CPU use for local reads, and is used by systems such as HBase and Impala.<\/p>\n<h3><strong><b>Q39. Why are large numbers of small files a problem, and how do you fix it?<\/b><\/strong><\/h3>\n<p>Every file and block takes memory in the NameNode, and each small file may start a separate task. Millions of small files can exhaust NameNode memory and slow jobs. Merge files with a compaction job, store data in container formats such as Parquet, ORC or Avro, or archive with HAR files. Prevent the problem upstream.<\/p>\n<div style=\"margin: 20px 0; border: 1px solid #d9e0ea; border-radius: 8px; overflow: hidden;\">\n<div style=\"background: #0f2a4a; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">python<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #f5f8fc; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">files, bytes_per_object = 50_000_000, 150\r\nobjects = files * 2            # file object plus one block object\r\n\r\nprint(\"approx NameNode memory:\", round(objects * bytes_per_object \/ 1024**3), \"GB\")<\/pre>\n<div style=\"background: #1f2937; color: #ffffff; font-family: Arial,Helvetica,sans-serif; font-size: 14px; font-weight: bold; padding: 8px 16px;\">output<\/div>\n<pre style=\"margin: 0; padding: 16px; background: #ffffff; color: #1f2937; font-family: 'Courier New',Courier,monospace; font-size: 14px; line-height: 1.7; overflow-x: auto; white-space: pre;\">approx NameNode memory: 14 GB<\/pre>\n<\/div>\n<p>&nbsp;<\/p>\n<h2><strong><b>Scenario Questions for Experienced Professionals<\/b><\/strong><\/h2>\n<p>Describe what you check first and which command you run.<\/p>\n<h3><strong><b>Q40. The NameNode is stuck in safe mode after a restart. What do you do?<\/b><\/strong><\/h3>\n<p>First check why with the dfsadmin safemode get command and the NameNode log. It usually waits for DataNodes to report blocks, so confirm that DataNodes are running. Run fsck to see missing blocks. If blocks are truly missing, restore the DataNodes or the data before leaving safe mode, because forcing it out can hide data loss.<\/p>\n<p><strong><b>What the interviewer is listening for: <\/b><\/strong><em><i>You investigate the cause before forcing safe mode off.<\/i><\/em><\/p>\n<h3><strong><b>Q41. fsck shows missing blocks. What now?<\/b><\/strong><\/h3>\n<p>Identify the affected files with fsck, then find out why the blocks are missing, such as failed disks or DataNodes that are down. Bring the DataNodes back if possible, since the blocks may return in block reports. If the data is gone, restore from backup or the source, and delete the damaged files so that jobs do not keep failing.<\/p>\n<h3><strong><b>Q42. One DataNode disk is full while others are nearly empty. What do you do?<\/b><\/strong><\/h3>\n<p>This is data skew. Run the balancer with a suitable threshold so blocks move to emptier nodes, and check the disk balancer for disks inside one node. Look for the cause, such as a node added recently, or writes that always start from one client node. Set up alerts on disk use.<\/p>\n<h3><strong><b>Q43. You must retain 5 years of cold data cheaply on Hadoop. What is your approach?<\/b><\/strong><\/h3>\n<p>Move old data to a cold storage tier with an archive storage policy and erasure coding, which cuts the storage cost to about half of triple replication. Keep the data in columnar compressed formats. Or move it to object storage with archive classes if the cluster is moving to cloud. Document access and recovery time.<\/p>\n<h3><strong><b>Q44. Users report that HDFS reads are slow at peak times. How do you investigate?<\/b><\/strong><\/h3>\n<p>Check the NameNode RPC queue and latency, since a busy NameNode slows every request. Look for a few jobs listing huge directories or opening many small files, and look at DataNode disk and network use, and at stragglers. Fixes include fewer and larger files, caching of listings, more handler threads and separating heavy workloads.<\/p>\n<h3><strong><b>Q45. How do you recover from accidentally deleting an important directory?<\/b><\/strong><\/h3>\n<p>First check the trash, if it was enabled, and restore from the trash folder of the user. If a snapshot exists, copy the data from the snapshot path. Otherwise restore from a DistCp backup. To prevent this, enable trash, create regular snapshots of key directories and restrict delete rights.<\/p>\n<h2><strong><b>Common questions about HDFS interviews<\/b><\/strong><\/h2>\n<h3><strong><b>What topics should I prepare for an HDFS interview?<\/b><\/strong><\/h3>\n<p>Prepare the architecture, blocks and replication, the read and write path, rack awareness and the commands first. Then learn high availability, federation, erasure coding, snapshots and the small files problem. Experienced roles add capacity planning and troubleshooting scenarios.<\/p>\n<h3><strong><b>Is HDFS still used in 2026?<\/b><\/strong><\/h3>\n<p>Yes, mainly in on premise clusters and in existing Hadoop platforms. Many new platforms use cloud object storage instead, but the ideas of blocks, replication, locality and a metadata server appear everywhere, so the knowledge transfers.<\/p>\n<h3><strong><b>What is the difference between HDFS and a normal file system?<\/b><\/strong><\/h3>\n<p>A normal file system manages one machine and small blocks, while HDFS spreads very large files across many machines using large blocks and keeps several copies for fault tolerance. HDFS is built for large sequential reads and is not meant for low latency or many small files.<\/p>\n<h3><strong><b>Which HDFS commands should I know by heart?<\/b><\/strong><\/h3>\n<p>Know ls, mkdir, put, get, cat, rm, du, setrep and chmod with hdfs dfs, plus hdfs fsck, hdfs dfsadmin report and safemode, hdfs namenode format and hadoop distcp. Practise them on a small cluster until you can type them without help.<\/p>\n<h3><strong><b>How can I practise HDFS at home?<\/b><\/strong><\/h3>\n<p>Run a single node or a small multi node cluster in containers, put files, check blocks with fsck, stop a DataNode and watch re replication. Try snapshots, quotas and an erasure coding policy. Note what you see, because interviewers ask about hands on experience.<\/p>\n<h2><strong><b>A four week plan to prepare<\/b><\/strong><\/h2>\n<p>Use this plan if you have about a month before your interview.<\/p>\n<div style=\"border: 1px solid #d9e0ea; border-radius: 8px; overflow-x: auto; margin: 20px 0; -webkit-overflow-scrolling: touch;\">\n<table style=\"width: 100%; min-width: 640px; border-collapse: collapse; font-family: Arial,Helvetica,sans-serif; font-size: 16px; line-height: 1.6; color: #1f2937;\">\n<caption style=\"caption-side: top; text-align: left; padding: 14px 16px; font-weight: bold; font-size: 17px; color: #0f2a4a;\">Four week study plan<\/caption>\n<thead>\n<tr>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38; width: 90px;\" scope=\"col\">Week<\/th>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38; width: 260px;\" scope=\"col\">Focus<\/th>\n<th style=\"background: #0f2a4a; color: #ffffff; text-align: left; padding: 12px 16px; border-bottom: 2px solid #0b1f38;\" scope=\"col\">What to do<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Week 1<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Architecture<\/td>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Draw the architecture from memory. Explain the NameNode, DataNodes, blocks and checkpoints.<\/td>\n<\/tr>\n<tr style=\"background: #f5f8fc;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Week 2<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Read, write and commands<\/td>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Trace a read and a write. Practise the commands on a test cluster, including fsck.<\/td>\n<\/tr>\n<tr style=\"background: #ffffff;\">\n<th style=\"text-align: left; padding: 12px 16px; border-bottom: 1px solid #e5e9f0; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Week 3<\/th>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Availability and features<\/td>\n<td style=\"padding: 12px 16px; border-bottom: 1px solid #e5e9f0;\">Study high availability, federation, erasure coding, snapshots, quotas and encryption.<\/td>\n<\/tr>\n<tr style=\"background: #f5f8fc;\">\n<th style=\"text-align: left; padding: 12px 16px; color: #0f2a4a; white-space: nowrap;\" scope=\"row\">Week 4<\/th>\n<td style=\"padding: 12px 16px;\">Capacity and scenarios<\/td>\n<td style=\"padding: 12px 16px;\">Do capacity calculations, and practise the scenario questions aloud.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>&nbsp;<\/p>\n<h2><strong><b>Practise with a trainer<\/b><\/strong><\/h2>\n<p>If you want to practise these questions with a trainer and work on hands on projects, you can book a free demo class for the <a href=\"https:\/\/prwatech.in\/big-data-online-course\/\">Prwatech Big Data course<\/a> and Hadoop course. It covers HDFS, YARN, MapReduce, Hive, Spark and hands on projects.<\/p>\n<p>Contact the learning team for current batch dates and fees.<\/p>\n<blockquote><p><em><strong>Written by Mithun Ray, data engineering trainer at Prwatech. Reviewed by Avishekh Verma. Last updated October,2026.<\/strong><\/em><\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"<p>Most Asked HDFS Interview Questions and Answers HDFS interviews test whether you understand how a distributed file system stores data safely and serves it quickly, and whether you can reason about failures. This guide gives you 45 HDFS interview questions and answers, from architecture to scenario questions, for freshers and experienced candidates. Every answer starts [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":2933,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[36,1709],"tags":[39],"class_list":["post-1354","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-interview-questions","category-interview-questions-interview-questions","tag-hdfs-questions-and-answers"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.7 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Top 45 HDFS Interview Questions with Answers | Prwatech<\/title>\n<meta name=\"description\" content=\"Most asked HDFS interview questions with answers on NameNode, blocks, replication, read and write path, commands, HA &amp; more for freshers, intermediates &amp; professionals\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Top 45 HDFS Interview Questions with Answers | Prwatech\" \/>\n<meta property=\"og:description\" content=\"Most asked HDFS interview questions with answers on NameNode, blocks, replication, read and write path, commands, HA &amp; more for freshers, intermediates &amp; professionals\" \/>\n<meta property=\"og:url\" content=\"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/\" \/>\n<meta property=\"og:site_name\" content=\"Prwatech\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prwatech.in\/\" \/>\n<meta property=\"article:published_time\" content=\"2026-10-12T12:12:08+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-10-14T15:40:58+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/04\/opening-for.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"768\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Prwatech\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Eduprwatech\" \/>\n<meta name=\"twitter:site\" content=\"@Eduprwatech\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Prwatech\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"18 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/\",\"url\":\"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/\",\"name\":\"Top 45 HDFS Interview Questions with Answers | Prwatech\",\"isPartOf\":{\"@id\":\"https:\/\/prwatech.in\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/04\/opening-for.jpg\",\"datePublished\":\"2026-10-12T12:12:08+00:00\",\"dateModified\":\"2026-10-14T15:40:58+00:00\",\"author\":{\"@id\":\"https:\/\/prwatech.in\/blog\/#\/schema\/person\/db90baff7744090b2288bbc98fea87f3\"},\"description\":\"Most asked HDFS interview questions with answers on NameNode, blocks, replication, read and write path, commands, HA & more for freshers, intermediates & professionals\",\"breadcrumb\":{\"@id\":\"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/#primaryimage\",\"url\":\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/04\/opening-for.jpg\",\"contentUrl\":\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/04\/opening-for.jpg\",\"width\":1024,\"height\":768,\"caption\":\"Hdfs interview questions and answers\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/prwatech.in\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"HDFS Interview Questions and Answers\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/prwatech.in\/blog\/#website\",\"url\":\"https:\/\/prwatech.in\/blog\/\",\"name\":\"Prwatech\",\"description\":\"Share Ideas, Start Something Good.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/prwatech.in\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/prwatech.in\/blog\/#\/schema\/person\/db90baff7744090b2288bbc98fea87f3\",\"name\":\"Prwatech\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/prwatech.in\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/c00bafc1b04045f31eda917de39891456c44fa47c092b9bb6be0f860a3a30a2f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/c00bafc1b04045f31eda917de39891456c44fa47c092b9bb6be0f860a3a30a2f?s=96&d=mm&r=g\",\"caption\":\"Prwatech\"},\"url\":\"https:\/\/prwatech.in\/blog\/author\/prwatech123\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Top 45 HDFS Interview Questions with Answers | Prwatech","description":"Most asked HDFS interview questions with answers on NameNode, blocks, replication, read and write path, commands, HA & more for freshers, intermediates & professionals","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/","og_locale":"en_US","og_type":"article","og_title":"Top 45 HDFS Interview Questions with Answers | Prwatech","og_description":"Most asked HDFS interview questions with answers on NameNode, blocks, replication, read and write path, commands, HA & more for freshers, intermediates & professionals","og_url":"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/","og_site_name":"Prwatech","article_publisher":"https:\/\/www.facebook.com\/prwatech.in\/","article_published_time":"2026-10-12T12:12:08+00:00","article_modified_time":"2026-10-14T15:40:58+00:00","og_image":[{"width":1024,"height":768,"url":"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/04\/opening-for.jpg","type":"image\/jpeg"}],"author":"Prwatech","twitter_card":"summary_large_image","twitter_creator":"@Eduprwatech","twitter_site":"@Eduprwatech","twitter_misc":{"Written by":"Prwatech","Est. reading time":"18 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/","url":"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/","name":"Top 45 HDFS Interview Questions with Answers | Prwatech","isPartOf":{"@id":"https:\/\/prwatech.in\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/#primaryimage"},"image":{"@id":"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/#primaryimage"},"thumbnailUrl":"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/04\/opening-for.jpg","datePublished":"2026-10-12T12:12:08+00:00","dateModified":"2026-10-14T15:40:58+00:00","author":{"@id":"https:\/\/prwatech.in\/blog\/#\/schema\/person\/db90baff7744090b2288bbc98fea87f3"},"description":"Most asked HDFS interview questions with answers on NameNode, blocks, replication, read and write path, commands, HA & more for freshers, intermediates & professionals","breadcrumb":{"@id":"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/#primaryimage","url":"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/04\/opening-for.jpg","contentUrl":"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2019\/04\/opening-for.jpg","width":1024,"height":768,"caption":"Hdfs interview questions and answers"},{"@type":"BreadcrumbList","@id":"https:\/\/prwatech.in\/blog\/interview-questions\/hdfs-questions-and-answers\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/prwatech.in\/blog\/"},{"@type":"ListItem","position":2,"name":"HDFS Interview Questions and Answers"}]},{"@type":"WebSite","@id":"https:\/\/prwatech.in\/blog\/#website","url":"https:\/\/prwatech.in\/blog\/","name":"Prwatech","description":"Share Ideas, Start Something Good.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/prwatech.in\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/prwatech.in\/blog\/#\/schema\/person\/db90baff7744090b2288bbc98fea87f3","name":"Prwatech","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/prwatech.in\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/c00bafc1b04045f31eda917de39891456c44fa47c092b9bb6be0f860a3a30a2f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c00bafc1b04045f31eda917de39891456c44fa47c092b9bb6be0f860a3a30a2f?s=96&d=mm&r=g","caption":"Prwatech"},"url":"https:\/\/prwatech.in\/blog\/author\/prwatech123\/"}]}},"_links":{"self":[{"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/posts\/1354","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/comments?post=1354"}],"version-history":[{"count":9,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/posts\/1354\/revisions"}],"predecessor-version":[{"id":11992,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/posts\/1354\/revisions\/11992"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/media\/2933"}],"wp:attachment":[{"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/media?parent=1354"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/categories?post=1354"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/tags?post=1354"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}