error_modelSel {poolABC}

Compute error in model selection with Approximate Bayesian Computation

Description

This function calculates the confusion matrix and the mean misclassification probabilities of models from the output of the sim_modelSel() function.

Usage

error_modelSel(object, threshold = NA, print = TRUE)

Arguments

object

a list created by the sim_modelSel() function, containing results of a simulation study to evaluate the quality of model selection with Approximate Bayesian Computation.

threshold

numeric value between 0 and 1 representing the minimum posterior probability of assignment.

print

logical, if TRUE (default), then this function prints the mean models probabilities.

Details

It is also possible to define a threshold for the posterior model probabilities. This threshold sets the minimum posterior probability of assignment. Thus, a simulation where the posterior probability of any model is below the threshold will not be assigned to a model and will instead be classified as "unclear".

Value

apart from directly displaying the results if print is TRUE, the output object of this function is a list with the following elements:

confusion.matrix

the confusion matrix.

probs

the mean model misclassification probabilities.

postmeans

the mean model misclassification probabilities when each model is correctly or incorrectly estimated.

Examples

# load the matrix with simulated parameter values
data(sumstats)

# select a random simulation to act as target just to test the function
target <- sumstats[10 ,]

# create a "fake" vector of model indices
# this assumes that half the simulations were from one model and the other half from other model
# this is not true but serves as an example of how to use this function
index <- c(rep("model1", nrow(sumstats)/2), rep("model2", nrow(sumstats)/2))

# perform a leave-one-out cross validation of model selection
mysim <- sim_modelSel(index = index, sumstats = sumstats, nval = 10, tol = 0.1)

# compute the confusion matrix and the mean misclassification probabilities
error_modelSel(object = mysim)