prediction_summary_cv {bayesrules}

Cross-Validated Posterior Predictive Summaries

Description

Given a set of observed data including a quantitative response variable y and an rstanreg model of y, this function returns 4 cross-validated measures of the model's posterior prediction quality: Median absolute prediction error (mae) measures the typical difference between the observed y values and their posterior predictive medians (stable = TRUE) or means (stable = FALSE). Scaled mae (mae_scaled) measures the typical number of absolute deviations (stable = TRUE) or standard deviations (stable = FALSE) that observed y values fall from their predictive medians (stable = TRUE) or means (stable = FALSE). within_50 and within_90 report the proportion of observed y values that fall within their posterior prediction intervals, the probability levels of which are set by the user. For hierarchical models of class lmerMod, the folds are comprised by collections of groups, not individual observations.

Usage

prediction_summary_cv(
  data,
  group,
  model,
  k,
  prob_inner = 0.5,
  prob_outer = 0.95
)

Arguments

data

data frame including the variables in the model, both response y and predictors x

group

a character string representing the name of the factor grouping variable, ie. random effect (only used for hierarchical models)

model

an rstanreg model object with quantitative y

k

the number of folds to use for cross validation

prob_inner

posterior predictive interval probability (a value between 0 and 1)

prob_outer

posterior predictive interval probability (a value between 0 and 1)

Value

list

Examples

example_data <- data.frame(x = sample(1:100, 20))
example_data$y <- example_data$x*3 + rnorm(20, 0, 5)
example_model <- rstanarm::stan_glm(y ~ x,  data = example_data)
prediction_summary_cv(model = example_model, data = example_data, k = 2)