kfold.brmsfit {brms} | R Documentation |
Perform exact K-fold cross-validation by refitting the model K times each leaving out one-Kth of the original data. Folds can be run in parallel using the future package.
## S3 method for class 'brmsfit' kfold( x, ..., K = 10, Ksub = NULL, folds = NULL, group = NULL, exact_loo = NULL, compare = TRUE, resp = NULL, model_names = NULL, save_fits = FALSE )
x |
A |
... |
More |
K |
The number of subsets of equal (if possible) size
into which the data will be partitioned for performing
K-fold cross-validation. The model is refit |
Ksub |
Optional number of subsets (of those subsets defined by |
folds |
Determines how the subsets are being constructed.
Possible values are |
group |
Optional name of a grouping variable or factor in the model.
What exactly is done with this variable depends on argument |
exact_loo |
Deprecated! Please use |
compare |
A flag indicating if the information criteria
of the models should be compared to each other
via |
resp |
Optional names of response variables. If specified, predictions are performed only for the specified response variables. |
model_names |
If |
save_fits |
If |
The kfold
function performs exact K-fold
cross-validation. First the data are partitioned into K folds
(i.e. subsets) of equal (or as close to equal as possible) size by default.
Then the model is refit K times, each time leaving out one of the
K
subsets. If K is equal to the total number of observations
in the data then K-fold cross-validation is equivalent to exact
leave-one-out cross-validation (to which loo
is an efficient
approximation). The compare_ic
function is also compatible with
the objects returned by kfold
.
The subsets can be constructed in multiple different ways:
If both folds
and group
are NULL
, the subsets
are randomly chosen so that they have equal (or as close to equal as
possible) size.
If folds
is NULL
but group
is specified, the
data is split up into subsets, each time omitting all observations of one
of the factor levels, while ignoring argument K
.
If folds = "stratified"
the subsets are stratified after
group
using loo::kfold_split_stratified
.
If folds = "grouped"
the subsets are split by
group
using loo::kfold_split_grouped
.
If folds = "loo"
exact leave-one-out cross-validation
will be performed and K
will be ignored. Further, if group
is specified, all observations corresponding to the factor level of the
currently predicted single value are omitted. Thus, in this case, the
predicted values are only a subset of the omitted ones.
If folds
is a numeric vector, it must contain one element per
observation in the data. Each element of the vector is an integer in
1:K
indicating to which of the K
folds the corresponding
observation belongs. There are some convenience functions available in
the loo package that create integer vectors to use for this purpose
(see the Examples section below and also the
kfold-helpers page).
kfold
returns an object that has a similar structure as the
objects returned by the loo
and waic
methods and
can be used with the same post-processing functions.
## Not run: fit1 <- brm(count ~ zAge + zBase * Trt + (1|patient) + (1|obs), data = epilepsy, family = poisson()) # throws warning about some pareto k estimates being too high (loo1 <- loo(fit1)) # perform 10-fold cross validation (kfold1 <- kfold(fit1, chains = 1)) # use the future package for parallelization library(future) plan(multiprocess) kfold(fit1, chains = 1) ## End(Not run)