How to see Hadoop's Configuration details if property is known

Some times you may want to see any specific property in your hadoop cluster and you might feel it would be good if you have any command line option that gives value stored in configuration. and Yes, there is an option to view most of the hadoop's configuration details.

This is important feature which will be useful when you entered into any new organization and you have been given hadoop cluster, but you are not sure of the hadoop installation path to check the properties by your self in hadoop .xml files, such as core-site.xml, yarn-site.xml, hdfs-site.xml etc.

Let me give you some useful commands which can come in handy.

1) hadoop org.apache.hadoop.mapred.JobConf

-- This gives properties of all mapreduce jobs. It takes properties from mapreduce-site and mapreduce-defaults. It works any where and gives all the properties.

(It works in both MR1 and MR2, dont confuse by the class package.)

This will throw complete xml like properties on the screen, you can use grep to filter on your requirement basis.

2) hadoop org.apache.hadoop.hdfs.tools.GetConf

-- By running this, with option -confKey we can see the value of any specific property..

eg: hadoop org.apache.hadoop.hdfs.tools.GetConf -confKey mapreduce.task.io.sort.mb

3) hadoop org.apache.hadoop.conf.Configuration -- By running this we can see the properties from core-site and core-defaults..

If you want to alter any specific property for your mapreduce job, then you can do with the help of GenericOptionsParser or ToolRunner.

Thank You...