Hadoop HDFS Operations - Commands with examples | W3School

Starting HDFS

To start HDFS, the HDFS file system has to be configured. To do that open NameNode and execute the below command:

$ hadoop namenode -format

Also Read: Hadoop Mapreduce

Once the formatting of the HDFS system is complete then the distributed file system can be started. To start the data nodes and name nodes as the cluster in the distributed file system, execute the below command:

$ start-dfs.sh

Listing Files in HDFS

Once all the information has been entered into the system, the list of files in the directories can be found out by using the command ‘ls’. The syntax to execute the ‘ls’ command to pass to a directory or filename in the form of an argument is below:

$ $HADOOP_HOME/bin/hadoop fs -ls <args>

[post_middile_section_ad]

Inserting Data into HDFS

Let us assume that we have some data in a file called file.txt that has to be saved in the HDFS system. Currently, the file.txt is on the local system. Following the below steps will help you to insert this file into the Hadoop file system:

A.      Step 1: An input directory must be created by using the command below:

$ $HADOOP_HOME/bin/hadoop fs -mkdir /user/input

B.      Step 2: Data must be transferred and stored in the Hadoop file system from the local systems. The ‘put’ command can be used for this step as shown below:

$ $HADOOP_HOME/bin/hadoop fs -put /home/file.txt /user/input

C.      Step 3: Use the ‘ls’ command to verify the file in the Hadoop file system. Us the below command for verifying:

$ $HADOOP_HOME/bin/hadoop fs -ls /user/input

Retrieving Data from HDFS

Let us assume that the code has generated a file called output.txt in Hadoop file system that has to be retrieved. Following the below steps will help you to retrieve this file from the Hadoop file system:

A.      Step 1: To view the file from the HDFS file system use the ‘cat’ command as shown below:

$ $HADOOP_HOME/bin/hadoop fs -cat /user/output/outfile

B.      Step 2: Data must be retrieved from Hadoop file system to the local system. The ‘get’ command can be used for this step as shown below:

$ $HADOOP_HOME/bin/hadoop fs -get /user/output/ /home/hadoop_tp/

Shutting Down the HDFS

Once the file is retrieved from the HDFS file system then it can be shut down by using the below command – $ stop-dfs.sh

[post_middile_section_ad]

Conclusion

The above tutorial will help you in understanding the HDFS operations in detail. You can start your interacting with the Hadoop file system by following above tutorial.