roc.area {verification}

Area under curve (AUC) calculation for Response Operating Characteristic curve.

Description

This function calculates the area underneath a ROC curve following the process outlined in Mason and Graham (2002). The p-value produced is related to the Mann-Whitney U statistics. The p-value is calculated using the wilcox.test function which automatically handles ties and makes approximations for large values.

The p-value addresses the null hypothesis $H_o:$ The area under the ROC curve is 0.5 i.e. the forecast has no skill.

Usage

    roc.area(obs, pred)
       

Arguments

obs

A binary observation (coded {0, 1 } ).

pred

A probability prediction on the interval [0,1].

Value

A

Area under ROC curve, adjusted for ties in forecasts, if present

n.total

Total number of records

n.events

Number of events

n.noevents

Number of non-events

p.value

Unadjusted p-value

Note

This function is used internally in the roc.plot command to calculate areas.

Author(s)

Matt Pocernich

References

Mason, S. J. and Graham, N. E. (2002) Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation, Q. J. R. Meteorol. Soc. 128, 2145–2166.

See Also

roc.plot, verify, wilcox.test

Examples

# Data used from Mason and Graham (2002).
a<- c(1981, 1982, 1983, 1984, 1985, 1986, 1987, 1988, 1989, 1990,
 1991, 1992, 1993, 1994, 1995)
b<- c(0,0,0,1,1,1,0,1,1,0,0,0,0,1,1)
c<- c(.8, .8, 0, 1,1,.6, .4, .8, 0, 0, .2, 0, 0, 1,1)
d<- c(.928,.576, .008, .944, .832, .816, .136, .584, .032, .016, .28, .024, 0, .984, .952)

A<- data.frame(a,b,c, d)
names(A)<- c("year", "event", "p1", "p2")

## for model with ties
roc.area(A$event, A$p1)

## for model without ties
roc.area(A$event, A$p2)