run_test {paramtest}

Run a function iteratively, with options for parallel processing.

Description

run_test runs a user-defined function iteratively. This function is intentionally kept general and flexible, to allow for a wide variety of applications. This function is the general-purpose function called by functions such as grid_search and random_search, which provide different methods for generating the parameters to be tested.

Usage

run_test(func, params = NULL, n.iter = 1, output = c("list",
  "data.frame"), boot = FALSE, bootParams = NULL, parallel = c("no",
  "multicore", "snow"), ncpus = 1, cl = NULL, beep = NULL, ...)

Arguments

func

A user-defined function. The first argument to this function will be the iteration number.

params

A list or data frame of parameters to be passed to func. Each set of parameters will be passed to func in turn.

n.iter

Number of iterations (per set of params).

output

Specifies how run_test provides the ultimate output from func: can return a "list" or a "data.frame". Note that if "data.frame" is specified, the supplied function must return a vector, matrix, or data frame, so it can be coerced into the data frame format. The "list" option will accept any type of output.

boot

Whether or not to use bootstrapped data to pass along to func. Using this option instead of bootstrapping within func is preferable to take advantage of parallelization.

bootParams

If boot=TRUE, then use bootParams to pass along a named list of arguments to the boot function. The statistic and R parameters will be filled automatically, but at minimum you will need to pass along data. Information about parallel processing will also be passed along automatically.

parallel

The type of parallel operation to be used (if any).

ncpus

Integer: the number of processes to be used in parallel operation.

cl

An optional parallel or snow cluster for use if parallel = 'snow'. If not supplied, a cluster on the local machine is created for the duration of the iterations.

beep

Include a numeric value or character vector indicating the sound you wish to play once the tests are done running. If set to TRUE, a random sound will be played. Requires the 'beepr' package, and information about supported values is available in the documentation for that package.

...

Additional arguments to be passed to func. If you do not need to vary certain parameters in your model, you can pass them to func here.

Value

Returns a list (by default) with one element per iteration. If output is specified as "data.frame", then func must return a (named) vector with the results you wish to capture.

See Also

boot

Examples

lm_test <- function(iter, N, b0, b1) {
    x <- rnorm(N, 0, 1)
    y <- rnorm(N, b0 + b1*x, sqrt(1 - b1^2))
    data <- data.frame(y, x)
    model <- lm(y ~ x, data)

    # capture output from model summary
    est <- coef(summary(model))['x', 'Estimate']
    se <- coef(summary(model))['x', 'Std. Error']
    p <- coef(summary(model))['x', 'Pr(>|t|)']

    return(c(xm=mean(x), xsd=sd(x), ym=mean(y), ysd=sd(y), est=est, se=se, p=p,
        sig=est > 0 & p <= .05))
}

# test power for sample size N=200 and N=300, with 500 iterations for each
power_sim <- run_test(lm_test, params=data.frame(N=c(200, 300)),
    n.iter=500, b0=0, b1=.15)