使用GenericOptionsParser简化参数解析

GenericOptionsParser可以让你的Map/Reduce程序具有Hadoop常用的属性。
一般无需直接用GenericOptionsParser,继承Configured即可。

public class ConfigurationPrinter extends Configured implements Tool {

//添加你需要的配置文件
static {
    Configuration.addDefaultResource("hdfs-default.xml");
    Configuration.addDefaultResource("hdfs-site.xml");
    Configuration.addDefaultResource("mapred-default.xml");
    Configuration.addDefaultResource("mapred-site.xml");
}

//run而不是在main中驱动。
@Override
public int run(String[] args) throws Exception {
    //conf已经支持Hadoop参数的解析了。
    Configuration conf = getConf();
    for (Entry<String, String> entry: conf) {
        System.out.printf("%s=%s\n", entry.getKey(), entry.getValue());
    }
    return 0;
}

public static void main(String[] args) throws Exception {
int exitCode = ToolRunner.run(new ConfigurationPrinter(), args);
System.exit(exitCode);
}

Hadoop支持的属性如下:

     -conf      specify a configuration file
     -D
            use value for given property
     -fs       specify a namenode
     -jt     specify a job tracker
     -files     specify comma separated
                            files to be copied to the map reduce cluster
     -libjars    specify comma separated
                            jar files to include in the classpath.
     -archives     specify comma
             separated archives to be unarchived on the compute machines.

Leave a Reply

Your email address will not be published. Required fields are marked *