evaluate {cases}

Evaluate the accuracy of multiple (candidate) classifiers in several subgroups

Description

Assess classification accuracy of multiple classifcation rules stratified by subgroups, e.g. in diseased (sensitivity) and healthy (specificity) individuals.

Usage

evaluate(
  data,
  contrast = define_contrast("raw"),
  benchmark = 0.75,
  alpha = 0.05,
  alternative = c("two.sided", "greater", "less"),
  adjustment = c("none", "bonferroni", "maxt", "bootstrap", "mbeta"),
  transformation = c("none", "logit"),
  analysis = c("co-primary", "full"),
  regu = FALSE,
  pars = list(),
  ...
)

Arguments

data

list of n_g x m binary matrix or data.frame (n_g observations of m binary decisions), g is the index of subgroups/classes, usually created via compare.

contrast

cases_contrast object, specified via define_contrast

benchmark

value to compare against (RHS), should have same length as data.

alpha

numeric, significance level (default: 0.05)

alternative

character, specify alternative hypothesis

adjustment

character, specify type of statistical adjustment taken to address multiplicity

transformation

character, define transformation to ensure results (e.g. point estimates, confidence limits) lie in unit interval ("none" (default) or "logit")

analysis

character, "co-primary" or "full"

regu

numeric vector of length 3, specify type of shrinkage. Alternatively, logical of length one (TRUE := c(2, 1, 1/2), FALSE := c(0, 0, 0))

pars

further parameters given as named list list(type="pairs", nboot=10000)

...

additional named parameters, can be used instead of (in in conjunction with) pars

Details

Adjustment methods (adjustment) and additional parameters (pars or ...):

"none" (default): no adjustment for multiplicity

"bonferroni": Bonferroni adjustment

"maxt": maxT adjustment

"bootstrap": Bootstrap approach

• type: "pairs" (default) or "wild" = type (for adjustment="bootstrap)

• nboot: number of bootstrap draws (default: 5000)

• res_tra: = 0,1,2 or 3 = type of residual transformation of wild boostrap (default = 0: no transformation) (see https://www.math.kth.se/matstat/gru/sf2930/papers/wild.bootstrap.pdf)

"mbeta": A heuristic Bayesian approach which is based on a multivariate beta-binomial model.

• nrep: number of posterior draws (default: 5000)

• lfc_pr: prior probability of 'least-favorable parameter configuration' (default: 1).

Value

cases_results object, which is a list of analysis results

Examples

#
data <- draw_data_roc()
evaluate(data)