BrierScore {s2dv}

Compute Brier score, its decomposition, and Brier skill score

Description

Compute the Brier score (BS) and the components of its standard decompostion with the two within-bin components described in Stephenson et al., (2008). It also returns the bias-corrected decomposition of the BS (Ferro and Fricker, 2012). BSS has the climatology as the reference forecast.

Usage

BrierScore(
  exp,
  obs,
  thresholds = seq(0.1, 0.9, 0.1),
  time_dim = "sdate",
  dat_dim = NULL,
  memb_dim = NULL,
  ncores = NULL
)

Arguments

exp

A vector or a numeric array with named dimensions. It should be the predicted probabilities which are within the range [0, 1] if memb_dim doesn't exist. If it has memb_dim, the value should be 0 or 1, and the predicted probabilities will be computed by ensemble mean. The dimensions must at least have 'time_dim'. range [0, 1].

obs

A numeric array with named dimensions of the binary observations (0 or 1). The dimension must be the same as 'exp' except memb_dim, which is optional. If it has 'memb_dim', then the length must be 1. The length of 'dat_dim' can be different from 'exp' if it has.

thresholds

A numeric vector used to bin the forecasts. The default value is seq(0.1, 0.9, 0.1), which means that the bins are [0, 0.1), [0.1, 0.2), ... [0.9, 1].

time_dim

A character string indicating the name of dimension along which Brier score is computed. The default value is 'sdate'.

dat_dim

A character string indicating the name of dataset dimension in 'exp' and 'obs'. The length of this dimension can be different between 'exp' and 'obs'. The default value is NULL.

memb_dim

A character string of the name of the member dimension in 'exp' (and 'obs', optional). The function will do the ensemble mean over this dimension. If there is no member dimension, set NULL. The default value is NULL.

ncores

An integer indicating the number of cores to use for parallel computation. The default value is NULL.

Value

A list that contains:

$rel

standard reliability

$res

standard resolution

$unc

standard uncertainty

$bs

Brier score

$bs_check_res

rel - res + unc

$bss_res

res - rel / unc

$gres

generalized resolution

$bs_check_gres

rel - gres + unc

$bss_gres

gres - rel / unc

$rel_bias_corrected

bias - corrected rel

$gres_bias_corrected

bias - corrected gres

$unc_bias_corrected

bias - corrected unc

$bss_bias_corrected

gres_bias_corrected - rel_bias_corrected / unc_bias_corrected

$nk

number of forecast in each bin

$fkbar

average probability of each bin

$okbar

relative frequency that the observed event occurred

The data type and dimensions of the items depend on if the input 'exp' and 'obs' are:
(a) Vectors
(b) Arrays with 'dat_dim' specified
(c) Arrays with no 'dat_dim' specified
Items 'rel', 'res', 'unc', 'bs', 'bs_check_res', 'bss_res', 'gres', 'bs_check_gres', 'bss_gres', 'rel_bias_corrected', 'gres_bias_corrected', 'unc_bias_corrected', and 'bss_bias_corrected' are (a) a number (b) an array with dimensions c(nexp, nobs, all the rest dimensions in 'exp' and 'obs' except 'time_dim' and 'memb_dim') (c) an array with dimensions of 'exp' and 'obs' except 'time_dim' and 'memb_dim'
Items 'nk', 'fkbar', and 'okbar' are (a) a vector of length of bin number determined by 'threshold' (b) an array with dimensions c(nexp, nobs, no. of bins, all the rest dimensions in 'exp' and 'obs' except 'time_dim' and 'memb_dim') (c) an array with dimensions c(no. of bin, all the rest dimensions in 'exp' and 'obs' except 'time_dim' and 'memb_dim')

References

Wilks (2006) Statistical Methods in the Atmospheric Sciences.
Stephenson et al. (2008). Two extra components in the Brier score decomposition. Weather and Forecasting, 23: 752-757.
Ferro and Fricker (2012). A bias-corrected decomposition of the BS. Quarterly Journal of the Royal Meteorological Society, DOI: 10.1002/qj.1924.

Examples

# Inputs are vectors
exp <- runif(10)
obs <- round(exp)
x <- BrierScore(exp, obs)

# Inputs are arrays
example(Load)
clim <- Clim(sampleData$mod, sampleData$obs)
ano_exp <- Ano(sampleData$mod, clim$clim_exp)
ano_obs <- Ano(sampleData$obs, clim$clim_obs)
bins_ano_exp <- ProbBins(ano_exp, thr = c(1/3, 2/3))
bins_ano_obs <- ProbBins(ano_obs, thr = c(1/3, 2/3))
res <- BrierScore(bins_ano_exp, MeanDims(bins_ano_obs, 'member'), memb_dim = 'member')