waic {loo} | R Documentation |
Widely applicable information criterion (WAIC)
Description
The waic()
methods can be used to compute WAIC from the pointwise
log-likelihood. However, we recommend LOO-CV using PSIS (as implemented by
the loo()
function) because PSIS provides useful diagnostics as well as
effective sample size and Monte Carlo estimates.
Usage
waic(x, ...)
## S3 method for class 'array'
waic(x, ...)
## S3 method for class 'matrix'
waic(x, ...)
## S3 method for class ''function''
waic(x, ..., data = NULL, draws = NULL)
is.waic(x)
Arguments
x |
A log-likelihood array, matrix, or function. The Methods (by class) section, below, has detailed descriptions of how to specify the inputs for each method. |
draws , data , ... |
For the function method only. See the Methods (by class) section below for details on these arguments. |
Value
A named list (of class c("waic", "loo")
) with components:
estimates
-
A matrix with two columns (
"Estimate"
,"SE"
) and three rows ("elpd_waic"
,"p_waic"
,"waic"
). This contains point estimates and standard errors of the expected log pointwise predictive density (elpd_waic
), the effective number of parameters (p_waic
) and the information criterionwaic
(which is just-2 * elpd_waic
, i.e., converted to deviance scale). pointwise
-
A matrix with three columns (and number of rows equal to the number of observations) containing the pointwise contributions of each of the above measures (
elpd_waic
,p_waic
,waic
).
Methods (by class)
-
waic(array)
: AnI
byC
byN
array, whereI
is the number of MCMC iterations per chain,C
is the number of chains, andN
is the number of data points. -
waic(matrix)
: AnS
byN
matrix, whereS
is the size of the posterior sample (with all chains merged) andN
is the number of data points. -
waic(`function`)
: A functionf()
that takes argumentsdata_i
anddraws
and returns a vector containing the log-likelihood for a single observationi
evaluated at each posterior draw. The function should be written such that, for each observationi
in1:N
, evaluatingf(data_i = data[i,, drop=FALSE], draws = draws)
results in a vector of length
S
(size of posterior sample). The log-likelihood function can also have additional arguments butdata_i
anddraws
are required.If using the function method then the arguments
data
anddraws
must also be specified in the call toloo()
:-
data
: A data frame or matrix containing the data (e.g. observed outcome and predictors) needed to compute the pointwise log-likelihood. For each observationi
, thei
th row ofdata
will be passed to thedata_i
argument of the log-likelihood function. -
draws
: An object containing the posterior draws for any parameters needed to compute the pointwise log-likelihood. Unlikedata
, which is indexed by observation, for each observation the entire objectdraws
will be passed to thedraws
argument of the log-likelihood function. The
...
can be used if your log-likelihood function takes additional arguments. These arguments are used like thedraws
argument in that they are recycled for each observation.
-
References
Watanabe, S. (2010). Asymptotic equivalence of Bayes cross validation and widely application information criterion in singular learning theory. Journal of Machine Learning Research 11, 3571-3594.
Vehtari, A., Gelman, A., and Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing. 27(5), 1413–1432. doi:10.1007/s11222-016-9696-4 (journal version, preprint arXiv:1507.04544).
Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2024). Pareto smoothed importance sampling. Journal of Machine Learning Research, 25(72):1-58. PDF
See Also
The loo package vignettes and Vehtari, Gelman, and Gabry (2017) and Vehtari, Simpson, Gelman, Yao, and Gabry (2024) for more details on why we prefer
loo()
towaic()
.-
loo_compare()
for comparing models on approximate LOO-CV or WAIC.
Examples
### Array and matrix methods
LLarr <- example_loglik_array()
dim(LLarr)
LLmat <- example_loglik_matrix()
dim(LLmat)
waic_arr <- waic(LLarr)
waic_mat <- waic(LLmat)
identical(waic_arr, waic_mat)
## Not run:
log_lik1 <- extract_log_lik(stanfit1)
log_lik2 <- extract_log_lik(stanfit2)
(waic1 <- waic(log_lik1))
(waic2 <- waic(log_lik2))
print(compare(waic1, waic2), digits = 2)
## End(Not run)