mcbdsc {triptych}

Evaluation of forecasts using score decompositions

Description

A score decomposition splits the mean score into the three components of miscalibration (MCB), discrimination (DSC), and uncertainty (UNC). Plotting the DSC component against the MCB component allows for a quick visual inspection of predictive performance for many forecasting methods.

Usage

mcbdsc(x, y_var = "y", ..., y = NULL, score = "Brier_score")

as_mcbdsc(x, r)

Arguments

x

A data frame, list, matrix, or other object that can be coerced to a tibble. Contains numeric forecasts, and observations (optional).

y_var

A variable in x that contains observations. Specified as the argument varin dplyr::pull().

...

Unused.

y

A numeric vector of observations. If supplied, overrides y_var. Otherwise, defaults to dplyr::pull(x, y_var).

score

A string specifying the score function. One of: "Brier_score" (default), "log_score", "MR_score".

r

A reference triptych_mcbdsc object whose attributes are used for casting.

Value

A triptych_mcbdsc object, that is a vctrs_vctr subclass, and has a length equal to number of forecasting methods supplied in x. Each entry is named according to the corresponding forecasting method, and contains a list of named objects:

• estimate: A data frame of the score decomposition.

• region: An empty list. Adding confidence regions is not yet supported.

• x: The numeric vector of original forecasts.

Access is most convenient through estimates(), regions(), and forecasts().

See Also

Accessors: estimates(), regions(), forecasts(), observations()

Visualization: plot.triptych_mcbdsc(), autoplot.triptych_mcbdsc()

Examples

data(ex_binary, package = "triptych")

md <- mcbdsc(ex_binary)
md

autoplot(md)
estimates(md)