testMultiplePairwise {exreport}

Multiple Comparison Statistical Test (Friedman + Pairwise Shaffer PostHoc)

Description

This function perfoms a multiple comparison statistical test for the given experiment. First of all it performs a Friedman Test over all methods. In the case this test is rejected, meaning that significant differences are present among the methods a post-hoc test is then executed. For that, each pair of methods are compared between each other, and finally a Shaffer familywise error correction is applied to the resulting p-values.

Usage

testMultiplePairwise(e, output, rankOrder = "max", alpha = 0.05)

Arguments

e

Input experiment

output

The output for which the tet will be performed.

rankOrder

The optimization strategy, can be either maximizing "max" or minimizing "min" the target output variable.

alpha

The significance level used for the whole testing procedure.

Value

an testMultiplePairwise object

Examples

# First we create an experiment from the wekaExperiment problem and prepare
# it to apply the test:
experiment <- expCreate(wekaExperiment, name="test", parameter="fold")
experiment <- expReduce(experiment, "fold", mean)
experiment <- expSubset(experiment, list(featureSelection = "yes"))
experiment <- expInstantiate(experiment, removeUnary=TRUE)

# Then we perform a testMultiplePairwise test procedure
test <- testMultiplePairwise(experiment, "accuracy", "max")

summary(test)