ReliabilityDiagram {SpecsVerification}

Reliability diagram for probability forecasts

Description

Reliability diagram for probability forecasts

Usage

ReliabilityDiagram(
  probs,
  obs,
  bins = 10,
  nboot = 500,
  plot = FALSE,
  plot.refin = TRUE,
  cons.probs = 0.95,
  attributes = FALSE,
  handle.na = c("na.fail", "use.pairwise.complete")
)

Arguments

probs

vector of N probability forecasts for the event obs=1

obs

vector of N binary observations, event/no event are coded as 0/1

bins

binning to estimate the calibration function (see Details), default: 10

nboot

number of bootstrap resamples to calculate the consistency bars, default: 500

plot

logical, whether to plot the reliability diagram, default: FALSE

plot.refin

Whether to add the frequency distribution of the forecasts to the reliability diagram. default: TRUE

cons.probs

The width of the consitency intervals. default: 0.95. Set to NA for no consistency bars.

attributes

locical, whether attributes lines are included in the diagram. default: FALSE

handle.na

how should missing values be handled; possible values are 'na.fail' and 'use.pairwise.complete'; default: 'na.fail'

Details

To estimate the reliability curve, the unit line is categorised into discrete bins, provided by the 'bins' argument. If 'bins' is a single number, it specifies the number of equidistant bins. If 'bins' is a vector of values between zero and one, these values are used as the bin-breaks.

Value

a data.frame with nrows equal to the number of bins (given by the 'bins' argument), with columns: average forecast probability per bin, conditional event frequency per bin, lower and upper limit of the consistency bar per bin, number of forecast probabilities per bin, lower and upper bin limit

References

Jolliffe IT, Stephenson DB, eds. (2012): Forecast verification: A practitioner's guide in atmospheric science. John Wiley & Sons, 2012. ISBN: 978-0-470-66071-3 Broecker J, Smith LA (2007): Increasing the Reliability of Reliability Diagrams. Wea. Forecasting, 22, 651–661 doi: 10.1175/WAF993.1

Examples

data(eurotempforecast)
p <- rowMeans(ens.bin)
ReliabilityDiagram(p, obs.bin, plot=TRUE)