Most Asked HDFS Interview Questions and Answers

HDFS interviews test whether you understand how a distributed file system stores data safely and serves it quickly, and whether you can reason about failures. This guide gives you 45 HDFS interview questions and answers, from architecture to scenario questions, for freshers and experienced candidates.

Every answer starts with a short, direct response you can say aloud, followed by a table, a command or a small calculation you can run. The page uses current Hadoop 3 details, such as the new port numbers, erasure coding and multiple standby NameNodes, and corrects the common mistake of calling the Secondary NameNode a backup.

The guide has six parts: 8 on HDFS architecture, 8 on the read and write path and replication, 8 on HDFS commands, 10 on high availability, federation and advanced features, 5 on capacity and performance and 6 scenario questions. For the wider platform, read the main Hadoop interview page.

Key points

• 45 questions in six parts, from architecture to scenarios.
• Read and write paths explained step by step.
• Current commands and Hadoop 3 details.
• High availability, federation, erasure coding, snapshots and quotas.
• Executed capacity and memory calculations.

In this guide

HDFS Architecture Interview Questions for Freshers

HDFS interviews begin with the architecture. Give crisp answers on the NameNode, DataNodes and blocks, because every later question builds on them. For a wider view of Hadoop, see the main Hadoop interview page.

Q1. What is HDFS, and what is it designed for?

The Hadoop Distributed File System stores very large files across many machines, with each file split into blocks that are copied to several nodes. It is designed for high throughput on large sequential reads, for streaming access, for hardware that fails often and for a write once, read many pattern. It is not designed for low latency or many small files.

Q2. Describe the HDFS architecture.

HDFS has a master and worker design. The NameNode holds the namespace and the map from files to blocks. DataNodes store the blocks on local disks, and send heartbeats and block reports to the NameNode. Clients ask the NameNode where data is, then read and write the data directly with the DataNodes.

Q3. What is a block, and what is the default block size?

A block is the unit in which HDFS splits and stores a file. The default size is 128 megabytes, controlled by dfs.blocksize, and you can set it per file. A file smaller than a block takes only its actual size on disk, but it still counts as one block object in NameNode memory.

import math

sizes_mb = [1, 128, 129, 1000]

for s in sizes_mb:
    print(f"{s} MB file -> {math.ceil(s / 128)} block(s)")

1 MB file -> 1 block(s)
128 MB file -> 1 block(s)
129 MB file -> 2 block(s)
1000 MB file -> 8 block(s)

Q4. What does the NameNode store, and where?

The NameNode keeps the whole namespace and the block map in memory for fast lookups. For durability it writes every change to an edit log on disk, and periodically saves a checkpoint of the namespace as an fsimage file. Block locations are not saved on disk. They are rebuilt from the block reports sent by DataNodes after a restart.

What the interviewer is listening for: You say that block locations are rebuilt from block reports and not stored on disk.

Q5. What are the fsimage and the edit log?

The fsimage is a snapshot of the file system namespace at a point in time. The edit log is a record of every change made since that snapshot. On startup, the NameNode loads the fsimage and replays the edit log to reach the current state. A periodic checkpoint merges the edits into a new fsimage, which keeps the log short.

Q6. What does the Secondary NameNode do?

The Secondary NameNode periodically downloads the fsimage and edit log, merges them into a new fsimage and sends it back. This keeps the edit log small and shortens restart time. It is not a failover node and cannot take over when the NameNode fails. In a high availability setup, the standby NameNode does this work instead.

What the interviewer is listening for: You say clearly that it is a checkpointing helper and not a backup.

Q7. What are heartbeats and block reports?

Each DataNode sends a heartbeat to the NameNode every three seconds by default to say it is alive, and the reply can carry commands such as replicate a block. A block report lists all the blocks a DataNode holds, and is sent at start and then periodically. A DataNode silent for about ten and a half minutes is marked dead.

Q8. What are the default ports of the HDFS services?

In Hadoop 3 the NameNode web interface is on port 9870, and its file system RPC port is commonly 8020 or 9000 depending on configuration. The DataNode web interface is on 9864 and the Secondary NameNode on 9868. Hadoop 2 used different numbers, such as 50070 for the NameNode, so say which version you mean.

What the interviewer is listening for: You know the Hadoop 3 ports changed from Hadoop 2.

Read and Write Path and Replication Interview Questions for Intermediate

Expect to be asked to explain a write or read step by step, so rehearse them.

Q9. How does HDFS write a file?

The client asks the NameNode to create the file, and the NameNode checks permissions and returns a list of DataNodes for the first block. The client streams data in packets to the first DataNode, which forwards it to the second, and then to the third, in a pipeline. Acknowledgements flow back, and the process repeats for each block.

• The client contacts the NameNode to create the file entry.
• For each block, the NameNode chooses the DataNodes for the pipeline.
• Packets go through the pipeline and acknowledgements return to the client.
• The client tells the NameNode when the file is closed.

Q10. How does HDFS read a file?

The client asks the NameNode for the block locations of the file. The NameNode returns, for each block, the DataNodes that hold it, sorted by distance from the client. The client then reads each block directly from the nearest DataNode. The NameNode is not in the data path, which keeps it from becoming a bottleneck.

Q11. What is rack awareness, and what is the default replica placement?

Rack awareness means HDFS knows which rack each DataNode sits in, through a topology script or mapping. With replication factor 3, the default policy puts the first replica on the writer node, the second on a node in a different rack, and the third on another node in that second rack. It balances safety with network cost.

What the interviewer is listening for: You can name the three placements and say why: one rack failure never loses all copies.

Q12. What happens if a DataNode fails during a write?

The pipeline is rebuilt without the failed node, the client continues with the remaining DataNodes, and the NameNode later notices the block is under replicated and schedules a new copy. The write does not fail unless too few nodes remain to meet the minimum number of replicas.

Q13. What happens when a replica is corrupted?

HDFS keeps a checksum for each chunk of a block. A client or DataNode that finds a mismatch while reading reports the bad replica to the NameNode and reads another copy. The NameNode then arranges a new healthy copy from a good replica and removes the corrupt one. DataNodes also scan their blocks in the background.

Q14. How do you change the replication factor of a file?

Use the setrep command, or set dfs.replication for new files. Increasing it makes the NameNode schedule extra copies, and decreasing it makes it delete extras. The factor is a per file setting, so important or hot data can have a higher factor than cold data.

hdfs dfs -setrep -w 5 /data/hot/lookup.csv
hdfs fsck /data/hot/lookup.csv -files -blocks -locations

Q15. How is a block stored on a DataNode?

Each block replica is saved as a file on the local file system of the DataNode, together with a small metadata file holding its checksums. DataNodes spread blocks over all their configured disks. HDFS does not alter blocks after they are written, which makes caching, replication and recovery simple.

Q16. Can you modify a file in HDFS?

HDFS is write once and read many. You can create, delete, rename, append to and truncate a file, but you cannot change bytes in the middle. To edit data, rewrite the file or use a layer on top, such as Hive ACID tables, HBase or a table format such as Iceberg or Delta, which manage the changes for you.

HDFS Commands Interview Questions for Freshers and Intermediate

Commands are tested in practical rounds. Note that the current form is hdfs dfs, and the older hadoop fs form works but is generic.

Q17. What are the most used HDFS file commands?

Use hdfs dfs with a subcommand: ls to list, mkdir to create a directory, put or copyFromLocal to upload, get or copyToLocal to download, cat to show content, rm to delete, du for sizes, and df for capacity. The syntax follows Linux commands, but all paths refer to HDFS.

hdfs dfs -mkdir -p /user/asha/input
hdfs dfs -put sales.csv /user/asha/input/
hdfs dfs -ls /user/asha/input
hdfs dfs -cat /user/asha/input/sales.csv | head
hdfs dfs -du -h /user/asha
hdfs dfs -get /user/asha/input/sales.csv ./sales_copy.csv
hdfs dfs -rm -r /user/asha/input

Q18. What is the difference between put, copyFromLocal and moveFromLocal?

The put and copyFromLocal commands both copy a local file into HDFS and keep the local copy. The copyFromLocal command only accepts a local source, while put can also read from standard input. The moveFromLocal command copies the file and then deletes the local copy.

Q19. How do you check the health of the file system?

Run hdfs fsck on a path. It reports missing, corrupt and under replicated blocks and, with options, lists files, blocks and their locations. Run hdfs dfsadmin with the report option for capacity and the state of each DataNode. The fsck command is read only and reports problems without repairing them.

hdfs fsck / -files -blocks -locations
hdfs dfsadmin -report

Note that fsck is its own command: hdfs fsck, not a subcommand of dfs.

Q20. How do you format the NameNode, and when should you?

Run hdfs namenode with the format option, once, when you create a new cluster. It initialises the namespace directories. Never format an existing cluster, because it erases the metadata and every file in HDFS becomes unreachable. The old hadoop namenode command is deprecated.

hdfs namenode -format

Q21. How does the trash work in HDFS?

If trash is enabled with fs.trash.interval, set to a number of minutes, deleted files move to a hidden trash folder in the user home directory and are purged after that interval. The setting is off by default in stock Hadoop, so a delete is permanent unless an administrator enabled it. The skipTrash option bypasses it.

Q22. How do you copy data between clusters?

Use DistCp, a MapReduce based tool that copies large amounts of data in parallel between clusters or within one. The update and delete options make repeated copies incremental. It is the standard tool for backups, migrations and moving data to cloud storage.

hadoop distcp -update hdfs://nn1:8020/data/sales hdfs://nn2:8020/backup/sales

Q23. What is safe mode?

Safe mode is a read only state the NameNode enters at start up while it collects block reports. It leaves when enough blocks, by default 99.9 percent, have reached their minimum replication. In safe mode, you cannot write or delete. An administrator can check or leave it manually with the dfsadmin command, though that should be done with care.

hdfs dfsadmin -safemode get
hdfs dfsadmin -safemode leave

Q24. How do HDFS permissions work?

HDFS uses a POSIX like model with owner, group and others, and read, write and execute bits. The execute bit on a file has no meaning, and on a directory it allows access to children. A umask sets default permissions, and access control lists add finer rules. In secure clusters, identity comes from Kerberos.

hdfs dfs -chmod 750 /user/asha
hdfs dfs -chown asha:analytics /user/asha
hdfs dfs -setfacl -m user:ravi:r-x /user/asha

High Availability, Federation and Advanced Features Interview Questions for Experienced

Experienced interviews dig into availability, scale and the features added in Hadoop 3.

Q25. How does NameNode high availability work?

Two or more NameNodes run, one active and the others standby. They share the edit log through a quorum of JournalNodes, so a standby stays almost up to date, and DataNodes send block reports to all of them. A ZooKeeper based failover controller detects failure and promotes a standby. Fencing prevents two active nodes.

What the interviewer is listening for: You name JournalNodes, ZooKeeper failover controllers and fencing.

Q26. What is split brain, and how does HDFS prevent it?

Split brain means two NameNodes both believe they are active and accept writes, which would corrupt the namespace. HDFS prevents it with fencing: the JournalNodes accept edits from only one writer at a time, and the failover controller can also fence the old active node by shutting it down or cutting its access.

Q27. What is HDFS federation?

Federation allows several independent NameNodes, each owning a part of the namespace, to share one pool of DataNodes. It scales the number of files beyond one NameNode memory, and separates workloads. Each namespace has its own block pool. Router based federation presents one view over many namespaces.

Q28. What is erasure coding in HDFS?

Erasure coding, introduced in Hadoop 3, stores data blocks with parity blocks instead of full copies. A common policy, Reed Solomon 6 plus 3, stores 6 data and 3 parity blocks, using 1.5 times the data size against 3 times for triple replication, and survives the loss of any 3 blocks. It suits cold data and costs more to read and recover.

data_gb = 600

print("3x replication:", data_gb * 3, "GB")
print("RS(6,3) erasure coding:", data_gb * 9 / 6, "GB")

3x replication: 1800 GB
RS(6,3) erasure coding: 900.0 GB

Q29. How do you apply an erasure coding policy?

Enable the policy on the cluster, then set it on a directory. Files written into that directory afterwards are erasure coded. Existing files keep their replication until they are rewritten. Pick the policy by how many racks or nodes you have, since more parity blocks need more failure domains.

hdfs ec -listPolicies
hdfs ec -enablePolicy -policy RS-6-3-1024k
hdfs ec -setPolicy -path /data/cold -policy RS-6-3-1024k

Q30. What are HDFS snapshots?

A snapshot is a read only, point in time copy of a directory. It is created fast because it records only the changes made after it, without copying blocks. Snapshots protect against accidental deletes and support backups. A directory must be made snapshottable by an administrator first.

hdfs dfsadmin -allowSnapshot /data/sales
hdfs dfs -createSnapshot /data/sales before_cleanup
hdfs dfs -ls /data/sales/.snapshot

Q31. How do quotas work in HDFS?

A name quota limits the number of files and directories under a directory, and a space quota limits the raw bytes. The space quota counts all replicas, so a file with three replicas uses three times its size. A write that would pass the quota fails.

file_gb, replication, quota_gb = 30, 3, 100

used = file_gb * replication
print("space quota used by one file:", used, "GB of", quota_gb, "GB")
print("another file of the same size fits:", used + file_gb * replication <= quota_gb)

space quota used by one file: 90 GB of 100 GB
another file of the same size fits: False

Q32. What is HDFS transparent encryption?

HDFS can encrypt data at rest in encryption zones, which are directories whose files are encrypted. Each file has its own data key, which is itself encrypted with a zone key held in an external key management server. Clients encrypt and decrypt, so the data is never seen as plain text by DataNodes or the NameNode.

Q33. What are the storage policies and the balancer?

Storage policies let you place data on different media, such as hot on disk, warm, cold on archive disks, or on SSD, chosen per directory. The balancer is a tool that moves blocks between DataNodes to even out disk use after nodes are added or after uneven writes. Run it with a threshold, in a quiet period.

Q34. How does HDFS differ from cloud object storage such as S3?

HDFS has a real directory tree, fast atomic renames, strong consistency and data locality on its own disks, but you manage the servers. Object storage is a managed, durable and cheaper store with no true directories, where a rename is a copy and delete, and access always goes over the network. Modern platforms often separate compute from storage.

Capacity and Performance Interview Questions for Experienced Professionals

Interviewers like numbers. Be ready to size a cluster and to explain what limits it.

Q35. How do you estimate the HDFS capacity you need?

Start with the data size, multiply by the replication factor, then add headroom for growth and for temporary and intermediate data, commonly 20 to 30 percent of disk kept free. For example, 100 terabytes of data with three copies needs 300 terabytes, and about 400 terabytes of raw disk once headroom is included.

data_tb, replication, free_fraction = 100, 3, 0.25

needed = data_tb * replication / (1 - free_fraction)
print("raw disk to buy, TB:", round(needed))

raw disk to buy, TB: 400

Q36. What limits the size of an HDFS cluster?

The main limit is the NameNode memory, because it holds every file and block object, so the number of files is limited more than the total bytes. Large blocks and fewer, larger files reduce the load. Beyond that, federation scales the namespace, and the network and the number of DataNodes limit throughput.

Q37. How do you improve HDFS read and write performance?

Use larger files and columnar formats, keep enough DataNodes and disks per node for parallel reads, and use short circuit local reads so a client on the same node reads the block directly from disk. Compress with a splittable or columnar friendly codec, and keep the network non blocking within and between racks.

Q38. What is short circuit local read?

Normally a client reads a block through the DataNode over a socket. With short circuit local reads, a client on the same machine as the block opens the block file directly through a domain socket, skipping the DataNode process. This reduces latency and CPU use for local reads, and is used by systems such as HBase and Impala.

Q39. Why are large numbers of small files a problem, and how do you fix it?

Every file and block takes memory in the NameNode, and each small file may start a separate task. Millions of small files can exhaust NameNode memory and slow jobs. Merge files with a compaction job, store data in container formats such as Parquet, ORC or Avro, or archive with HAR files. Prevent the problem upstream.

files, bytes_per_object = 50_000_000, 150
objects = files * 2            # file object plus one block object

print("approx NameNode memory:", round(objects * bytes_per_object / 1024**3), "GB")

approx NameNode memory: 14 GB

Scenario Questions for Experienced Professionals

Describe what you check first and which command you run.

Q40. The NameNode is stuck in safe mode after a restart. What do you do?

First check why with the dfsadmin safemode get command and the NameNode log. It usually waits for DataNodes to report blocks, so confirm that DataNodes are running. Run fsck to see missing blocks. If blocks are truly missing, restore the DataNodes or the data before leaving safe mode, because forcing it out can hide data loss.

What the interviewer is listening for: You investigate the cause before forcing safe mode off.

Q41. fsck shows missing blocks. What now?

Identify the affected files with fsck, then find out why the blocks are missing, such as failed disks or DataNodes that are down. Bring the DataNodes back if possible, since the blocks may return in block reports. If the data is gone, restore from backup or the source, and delete the damaged files so that jobs do not keep failing.

Q42. One DataNode disk is full while others are nearly empty. What do you do?

This is data skew. Run the balancer with a suitable threshold so blocks move to emptier nodes, and check the disk balancer for disks inside one node. Look for the cause, such as a node added recently, or writes that always start from one client node. Set up alerts on disk use.

Q43. You must retain 5 years of cold data cheaply on Hadoop. What is your approach?

Move old data to a cold storage tier with an archive storage policy and erasure coding, which cuts the storage cost to about half of triple replication. Keep the data in columnar compressed formats. Or move it to object storage with archive classes if the cluster is moving to cloud. Document access and recovery time.

Q44. Users report that HDFS reads are slow at peak times. How do you investigate?

Check the NameNode RPC queue and latency, since a busy NameNode slows every request. Look for a few jobs listing huge directories or opening many small files, and look at DataNode disk and network use, and at stragglers. Fixes include fewer and larger files, caching of listings, more handler threads and separating heavy workloads.

Q45. How do you recover from accidentally deleting an important directory?

First check the trash, if it was enabled, and restore from the trash folder of the user. If a snapshot exists, copy the data from the snapshot path. Otherwise restore from a DistCp backup. To prevent this, enable trash, create regular snapshots of key directories and restrict delete rights.

Common questions about HDFS interviews

What topics should I prepare for an HDFS interview?

Prepare the architecture, blocks and replication, the read and write path, rack awareness and the commands first. Then learn high availability, federation, erasure coding, snapshots and the small files problem. Experienced roles add capacity planning and troubleshooting scenarios.

Is HDFS still used in 2026?

Yes, mainly in on premise clusters and in existing Hadoop platforms. Many new platforms use cloud object storage instead, but the ideas of blocks, replication, locality and a metadata server appear everywhere, so the knowledge transfers.

What is the difference between HDFS and a normal file system?

A normal file system manages one machine and small blocks, while HDFS spreads very large files across many machines using large blocks and keeps several copies for fault tolerance. HDFS is built for large sequential reads and is not meant for low latency or many small files.

Which HDFS commands should I know by heart?

Know ls, mkdir, put, get, cat, rm, du, setrep and chmod with hdfs dfs, plus hdfs fsck, hdfs dfsadmin report and safemode, hdfs namenode format and hadoop distcp. Practise them on a small cluster until you can type them without help.

How can I practise HDFS at home?

Run a single node or a small multi node cluster in containers, put files, check blocks with fsck, stop a DataNode and watch re replication. Try snapshots, quotas and an erasure coding policy. Note what you see, because interviewers ask about hands on experience.

A four week plan to prepare

Use this plan if you have about a month before your interview.

Practise with a trainer

If you want to practise these questions with a trainer and work on hands on projects, you can book a free demo class for the Prwatech Big Data course and Hadoop course. It covers HDFS, YARN, MapReduce, Hive, Spark and hands on projects.

Contact the learning team for current batch dates and fees.

Written by Mithun Ray, data engineering trainer at Prwatech. Reviewed by Avishekh Verma. Last updated October,2026.