TModC {ModTools}

Compare Classification Models

Description

For the comparison of several classification models, the AUC values and BrierScore values of the models are determined and tabulated. Both the absolute values and the relative values are reported, each related to the model with the highest corresponding value.

Usage

TModC(..., newdata = NULL, reference = NULL, ord = NULL)

## S3 method for class 'TModC'
plot(x, col = NULL, args.legend = NULL,...)

Arguments

...

the models to be compared

x

TModC object to plot

newdata

the data to use for predicting. If not provided, the model.frame will be used.

reference

the reference values

ord

character defining the order of the results table, can be any of "auc", "bs", "auc_p", "bs_p", "bs_rnk", "auc_rnk", "ensemble" (using the mean of "auc_p" and "bs_p" for the ranking).

col

the color for the lines in the ROC plot

args.legend

the legend to be placed in the ROC plot

Value

a matrix with the columns

auc

absolute value of area under the ROC curve (AUC)

auc_p

percentage of the auc based on the best observerd AUC

auc_rnk

the rank of the auc

bs

absolute value of the Brier score

bs_p

percentage of the Brier score based on the best observed BS

bs_rnk

the rank of the BS

auc_grnk

character representation of the AUC rank

bs_grnk

character representation of the BS rank

Author(s)

Andri Signorell <[e-mail]>

See Also

TMod, BrierScore, AUC, ROC

Examples

d.pim <- SplitTrainTest(d.pima, p = 0.2)
mdiab <- formula(diabetes ~ pregnant + glucose + pressure + triceps +
                            insulin + mass + pedigree + age)

r.glm <- FitMod(mdiab, data=d.pim$train, fitfn="logit")
r.rp <- FitMod(mdiab, data=d.pim$train, fitfn="rpart")
mods <- list(glm=r.glm, rp=r.rp)

# the table with the measures
(tm <- TModC(mods, ord="auc"))

# plotting the ROC curves
plot(tm, col=c("darkmagenta", "dodgerblue"))