configuration {hive}

Managing the Hadoop configuration

Description

Functions for showing/changing Hadoop configuration.

Usage

hive_get_parameter( x, henv = hive() )
hive_get_masters( henv = hive() )
hive_get_workers( henv = hive() )
hive_get_nreducer( henv = hive() )
hive_set_nreducer( n, henv = hive() )

Arguments

henv

An object containing the local Hadoop configuration.

x

A character string naming the parameter in the Hadoop configuration.

n

An integer specifying the number of reducers to be used in hive_stream().

Details

The function hive_get_parameter() is used to get parameters from the Hadoop cluster configuration.

The functions hive_get_workers() and hive_get_masters() return the hostnames of the configured nodes in the cluster.

The functions hive_get_nreducer() and hive_set_nreducer() are used to get/set the number of reducers which are used in Hadoop Streaming using hive_stream().

Value

hive_get_parameter() returns the specified parameter as a character string.

hive_get_workers() returns a character vector naming the hostnames of the configured worker nodes in the cluster.

hive_get_masters() returns a character vector of the hostnames of the configured master nodes in the cluster.

hive_get_nreducer() returns an integer representing the number of configured reducers.

Author(s)

Stefan Theussl

References

Apache Hadoop cluster configuration (https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html#Configuring_Hadoop_in_Non-Secure_Mode).

Examples

## Which tmp directory is set in the Hadoop configuration?
## Not run: hive_get_parameter("hadoop.tmp.dir")

## The master nodes of the cluster
## Not run: hive_get_masters()

## The worker nodes of the cluster
## Not run: hive_get_workers()

## The number of configured reducers
## Not run: hive_get_nreducer()