cv.merMod {cv}

Cross-Validate Mixed-Effects Model

Description

cv() methods for mixed-effect models of class "merMod", fit by the lmer() and glmer() functions in the lme4 package; for models of class "lme" fit by the lme() function in the nlme package; and for models of class "glmmTMB" fit by the glmmTMB() function in the glmmTMB package.

Usage

## S3 method for class 'merMod'
cv(
  model,
  data = insight::get_data(model),
  criterion = mse,
  k = NULL,
  reps = 1L,
  seed,
  details = NULL,
  ncores = 1L,
  clusterVariables,
  blups = coef,
  fixed.effects = lme4::fixef,
  ...
)

## S3 method for class 'lme'
cv(
  model,
  data = insight::get_data(model),
  criterion = mse,
  k = NULL,
  reps = 1L,
  seed,
  details = NULL,
  ncores = 1L,
  clusterVariables,
  blups = coef,
  fixed.effects = nlme::fixef,
  ...
)

## S3 method for class 'glmmTMB'
cv(
  model,
  data = insight::get_data(model),
  criterion = mse,
  k = NULL,
  reps = 1L,
  seed,
  details = NULL,
  ncores = 1L,
  clusterVariables,
  blups = coef,
  fixed.effects = glmmTMB::fixef,
  ...
)

Arguments

model

a mixed-effects model object for which a cv() method is available.

data

data frame to which the model was fit (not usually necessary)

criterion

cross-validation ("cost" or lack-of-fit) criterion function of form f(y, yhat) where y is the observed values of the response and yhat the predicted values; the default is mse (the mean-squared error).

k

perform k-fold cross-validation; k may be a number or "loo" or "n" for n-fold (leave-one-out) cross-validation; the default is 10 if cross-validating individual cases and "loo" if cross-validating clusters.

reps

number of times to replicate k-fold CV (default is 1), or greater), compute a confidence interval for the bias-corrected CV criterion, if the criterion is the average of casewise components.

seed

for R's random number generator; optional, if not supplied a random seed will be selected and saved; not needed for n-fold cross-validation

details

if TRUE (the default if the number of folds k <= 10), save detailed information about the value of the CV criterion for the cases in each fold and the regression coefficients with that fold deleted.

ncores

number of cores to use for parallel computations (default is 1, i.e., computations aren't done in parallel)

clusterVariables

a character vector of names of the variables defining clusters for a mixed model with nested or crossed random effects; if missing, cross-validation is performed for individual cases rather than for clusters

blups

a function to be used to compute BLUPs for case-based CV when details = TRUE.

fixed.effects

a function to be used to compute fixed-effect coefficients for cluster-based CV when details = TRUE.

...

for cv() methods, to match generic, and for cvMixed(), arguments to be passed to update().

Details

For mixed-effects models, cross-validation can be done by "clusters" or by individual observations. If the former, predictions are based only on fixed effects; if the latter, predictions include the random effects (i.e., are the best linear unbiased predictors or "BLUPS").

Value

The methods cv.merMod(), cv.lme(), and cv.glmmTMB(), return objects of class "cv", or, if reps > 1, of class "cvList" (see cv()).

Functions

• cv(merMod): cv() method for lmer() and glmer() models from the lme4 package.

• cv(lme): cv() method for lme() models from the nlme package.

• cv(glmmTMB): cv() method for glmmTMB() models from the glmmTMB package.

See Also

cv, lmer, glmer, lme, glmmTMB

Examples

library("lme4")
# from ?lmer:
(fm1 <- lmer(Reaction ~ Days + (Days | Subject), sleepstudy))
cv(fm1, clusterVariables="Subject") # LOO CV of clusters
cv(fm1, seed=447) # 10-fold CV of cases
cv(fm1, clusterVariables="Subject", k=5,
   seed=834, reps=3) # 5-fold CV of clusters, repeated 3 times

library("nlme")
# from ?lme
(fm2 <- lme(distance ~ age + Sex, data = Orthodont,
            random = ~ 1))
cv(fm2) # LOO CV of cases
cv(fm2, clusterVariables="Subject", k=5, seed=321) # 5-fold CV of clusters

library("glmmTMB")
# from ?glmmTMB
(m1 <- glmmTMB(count ~ mined + (1|site),
               zi=~mined,
               family=poisson, data=Salamanders))
cv(m1, seed=97816, k=5, clusterVariables="site") # 5-fold CV of clusters
cv(m1, seed=34506, k=5) # 5-fold CV of cases