accuracy {rcompanion}

Minimum maximum accuracy, mean absolute percent error, median absolute error, root mean square error, coefficient of variation, and Efron's pseudo r-squared

Description

Produces a table of fit statistics for multiple models.

Usage

accuracy(fits, plotit = FALSE, digits = 3, ...)

Arguments

fits

A series of model object names. Must be a list of model objects or a single model object.

plotit

If TRUE, produces plots of the predicted values vs. the actual values for each model.

digits

The number of significant digits in the output.

...

Other arguments passed to plot.

Details

Produces a table of fit statistics for multiple models: minimum maximum accuracy, mean absolute percentage error, median absolute error, root mean square error, normalized root mean square error, Efron's pseudo r-squared, and coefficient of variation.

For minimum maximum accuracy, larger indicates a better fit, and a perfect fit is equal to 1.

For mean absolute error (MAE), smaller indicates a better fit, and a perfect fit is equal to 0. It has the same units as the dependent variable. Note that here, MAE is simply the mean of the absolute values of the differences of predicted values and the observed values (MAE = mean(abs(predy - actual))). There are other definitions of MAE and similar-sounding terms.

Median absolute error (MedAE) is similar, except employing the median rather than the mean.

For mean absolute percent error (MAPE), smaller indicates a better fit, and a perfect fit is equal to 0. The result is reported as a fraction. That is, a result of 0.1 is equal to 10 percent.

Root mean square error (RMSE) has the same units as the predicted values.

Normalized root mean square error (NRMSE) is RMSE divided by the mean or the median of the values of the dependent variable.

Efron's pseudo r-squared is calculated as 1 minus the residual sum of squares divided by the total sum of squares. For linear models (lm model objects), Efron's pseudo r-squared will be equal to r-squared. For other models, it should not be interpreted as r-squared, but can still be useful as a relative measure.

CV.prcnt is the coefficient of variation for the model. Here it is expressed as a percent. That is, a result of 10 = 10 percent.

Model objects currently supported: lm, glm, nls, betareg, gls, lme, lmer, lmerTest, glmmTMB, rq, loess, gam, glm.nb, glmRob, mblm, and rlm.

Value

A list of two objects: The series of model calls, and a data frame of statistics for each model.

Author(s)

Salvatore Mangiafico, [e-mail]

References

https://rcompanion.org/handbook/G_14.html

See Also

compareLM, compareGLM, nagelkerke

Examples

data(BrendonSmall)
BrendonSmall$Calories = as.numeric(BrendonSmall$Calories)
BrendonSmall$Calories2 = BrendonSmall$Calories ^ 2
model.1 = lm(Sodium ~ Calories, data = BrendonSmall)

accuracy(model.1, plotit=FALSE)

model.2 = lm(Sodium ~ Calories + Calories2, data = BrendonSmall)
model.3 = glm(Sodium ~ Calories, data = BrendonSmall, family="Gamma")
quadplat = function(x, a, b, clx) {
          ifelse(x  < clx, a + b * x   + (-0.5*b/clx) * x   * x,
                           a + b * clx + (-0.5*b/clx) * clx * clx)}
model.4 = nls(Sodium ~ quadplat(Calories, a, b, clx),
              data = BrendonSmall,
              start = list(a=519, b=0.359, clx = 2300))
              
accuracy(list(model.1, model.2, model.3, model.4), plotit=FALSE)

### Perfect and poor model fits
X = c(1, 2,  3,  4,  5,  6, 7, 8, 9, 10, 11, 12)
Y = c(1, 2,  3,  4,  5,  6, 7, 8, 9, 10, 11, 12)
Z = c(1, 12, 13, 6, 10, 13, 4, 3, 5,  6, 10, 14)
perfect = lm(Y ~ X)
poor    = lm(Z ~ X)
accuracy(list(perfect, poor), plotit=FALSE)