Holdout-class {performanceEstimation}

Class "Holdout"

Description

This class of objects contains the information describing a hold out experiment, i.e. its settings.

Objects from the Class

Objects can be created by calls of the form Holdout(...) providing the values for the class slots. The objects contain information on the number of repetitions of the hold out experiment, the percentage of the given data to set as hold out test set, the random number generator seed, information on whether stratified sampling should be used and optionally the concrete data splits to use on each iteration of the holdout experiment. Note that most of the times you will not supply these data splits as the holdout routines in this infra-structure will take care of building them. Still, this allows you to replicate some experiment carried out with specific train/test splits.

Slots

nReps:

Object of class numeric indicating the number of repetitions of the N folds CV experiment (defaulting to 1).

hldSz:

Object of class numeric with the percentage (a number between 0 and 1) of cases to use as hold out (defaulting to 0.3).

strat:

Object of class logical indicating whether the sampling should or not be stratefied (defaulting to FALSE).

seed:

Object of class numeric with the random number generator seed (defaulting to 1234).

dataSplits:

Object of class list containing the data splits to use on each repetition of a nReps Holdout experiment (defaulting to NULL). This list should contain nReps elements. Each element should be a vector with the row ids of the test set of the respective iteration. On all these iterations the training set will be formed by the ids not appearing in the test set.

Extends

Class EstCommon, directly. Class EstimationMethod, directly.

Methods

show

signature(object = "Holdout"): method used to show the contents of a Holdout object.

Author(s)

Luis Torgo [e-mail]

References

Torgo, L. (2014) An Infra-Structure for Performance Estimation and Experimental Comparison of Predictive Models in R. arXiv:1412.0436 [cs.MS] http://arxiv.org/abs/1412.0436

See Also

MonteCarlo, LOOCV, Bootstrap, CV, EstimationMethod, EstimationTask

Examples

showClass("Holdout")

## 10 repetitions of a holdout experiment leaving on each repetition
## 20% of the cases randomly chosen as test set (the holdout)
h1 <- Holdout(nReps=10,hldSz=0.2,strat=TRUE)
h1

## Small example illustrating the format of user supplied data splits
## in this case for 3 repetitions of a Holdout process where each test
## set has 10 cases
h2 <- Holdout(dataSplits=list(1:10,11:20,21:30))
h2