se_boot {CAISEr}

Bootstrap standard errors

Description

Calculates the standard errors of a given statistic using bootstrap

Usage

se_boot(Xk, dif = "simple", comparisons = "all.vs.all", boot.R = 999, ...)

Arguments

Xk

list object where each position contains a vector of observations of algorithm k on a given problem instance.

dif

name of the difference for which the SEs are desired. Accepts "perc" (for percent differences) or "simple" (for simple differences)

comparisons

standard errors to be calculated. Accepts "all.vs.first" (in which cases the first object in algorithms is considered to be the reference algorithm) or "all.vs.all" (if there is no reference and all pairwise SEs are desired).

boot.R

(optional) number of bootstrap resamples (if method == "boot")

...

other parameters (used only for compatibility with calls to se_param(), unused in this function)

Value

Data frame containing, for each pair of interest, the estimated difference (column "Phi") and the sample standard error (column "SE")

References

• A.C. Davison, D.V. Hinkley: Bootstrap methods and their application. Cambridge University Press (1997)

• F. Campelo, F. Takahashi: Sample size estimation for power and accuracy in the experimental comparison of algorithms. Journal of Heuristics 25(2):305-338, 2019.

Author(s)

Felipe Campelo ([e-mail], [e-mail])

Examples

# three vectors of normally distributed observations
set.seed(1234)
Xk <- list(rnorm(10, 5, 1),  # mean = 5, sd = 1,
           rnorm(20, 10, 2), # mean = 10, sd = 2,
           rnorm(20, 15, 5)) # mean = 15, sd = 3

se_boot(Xk, dif = "simple", comparisons = "all.vs.all")
se_boot(Xk, dif = "perc", comparisons = "all.vs.first")
se_boot(Xk, dif = "perc", comparisons = "all.vs.all")