measurement.error {verification}

Skill score with measurement error.

Description

Skill score that incorporates measurement error. This function allows the user to incorporate measurement error in an observation in a skill score.

Usage

measurement.error( obs, frcs = NULL, theta = 0.5, CI =
          FALSE, t = 1, u = 0, h = NULL, ...)
       

Arguments

obs

Information about a forecast and observation can be done in one of two ways. First, the results of a contingency table can be entered as a vector containing c(n11, n10, n01, n00), where n11 are the number of correctly predicted events and n01 is the number of incorrectly predicted non-events. Actual forecasts and observations can be used. In this case, obs is a vector of binary outcomes [0,1].

frcs

If obs is entered as a contingency table, this argument is null. If obs is a vector of outcomes, this column is a vector of probabilistic forecasts.

theta

Loss value (cost) of making a incorrect forecast by a non-event. Defaults to 0.5.

CI

Calculate confidence intervals for skill score.

t

Probability of forecasting an event, when an event occurs. A perfect value is 1.

u

Probability of forecasting that no event will occur, when and event occurs. A perfect value is 0.

h

Threshold for converting a probabilistic forecast into a binary forecast. By default, this value is NULL and the theta is used as this threshold.

...

Optional arguments.

Value

z

Error code

k

Skill score

G

Likelihood ratio statistic

p

p-value for the null hypothesis that the forecast contains skill.

theta

Loss value. Loss associated with an incorrect forecast of a non-event.

ciLO

Lower confidence interval

ciHI

Upper confidence interval

Author(s)

Matt Pocernich (R - code)

W.M Briggs <wib2004(at)med.cornell.edu> (Method questions)

References

W.M. Briggs, 2004. Incorporating Cost in the Skill Score Technical Report, wm-briggs.com/public/skillocst.pdf.

W.M. Briggs and D. Ruppert, 2004. Assessing the skill of yes/no forecasts. Submitting to Biometrics.

J.P. Finley, 1884. Tornado forecasts. Amer. Meteor. J. 85-88. (Tornado data used in example.)

Examples

DAT<- data.frame( obs = round(runif(50)), frcs = runif(50))

A<-   measurement.error(DAT$obs, DAT$frcs, CI = TRUE)
A
### Finley Data

measurement.error(c(28, 23, 72, 2680)) ## assuming perfect observation,