evalm {MLeval}

evalm: Evaluate Machine Learning Models in R

Description

evalm is for machine learning model evaluation in R. The function can accept the Caret 'train' function results to evaluate machine learning predictions or a data frame of probabilities and ground truth labels can be passed in to evaluate. Probability data must be column1: probability group1 (column named as your group name 1), column2: probability group2 (column named as your group name 2), column3: observation labels (column named 'obs'), column4: Group, e.g. different models (column named 'Group'), optional to include if different models are combined horizontally.

Usage

evalm(list1, gnames = NULL, title = "", cols = NULL,
  silent = FALSE, rlinethick = 1.25, fsize = 12.5,
  dlinecol = "grey", dlinethick = 0.75, bins = 6, optimise = "INF",
  percent = 95, showplots = TRUE, positive = NULL, plots = c("prg",
  "pr", "r", "cc"))

Arguments

list1

List or data frame: List of Caret results objects from train, or a single train results object, or a data frame of probabilities and observed labels

gnames

Character vector: A vector of group names for the fit objects

title

Character string: A title for the ROC plot

cols

Character vector: A vector of colours for the group or groups

silent

Logical flag: whether to hide messages (default=FALSE)

rlinethick

Numerical value: Thickness of the ROC curve line

fsize

Numerical value: Font size for the ROC curve plots

dlinecol

Character string: Colour of the diagonal line

dlinethick

Numerical value: Thickness of the diagonal line

bins

Numerical value: Number of bins for calibration curve

optimise

Character string: Metric by which to select the operating point (INF, MCC, or F1)

percent

Numerical value: percentage for the confidence intervals (default = 95)

showplots

Logical flag: whether to show plots or not

positive

Character string: Name of the positive group (will effect PR metrics)

plots

Character vector: which plots to show: r = roc, pr = proc, prg = precision recall gain, cc = calibration curve

Value

List containing: 1) A ggplot2 ROC curve object for printing 2) A ggplot2 PROC object for printing 3) A ggplot2 PRG curve for printing 4) Optimised results according to defined metric 5) P cut-off of 0.5 standard results

Examples

r <- evalm(fit)