metrics {MRMCaov}

Performance Metrics

Description

Estimated performance metrics from ROC curves.

Usage

binary_sens(truth, rating)

binary_spec(truth, rating)

binormal_auc(
  truth,
  rating,
  partial = FALSE,
  min = 0,
  max = 1,
  normalize = FALSE
)

binormal_eu(truth, rating, slope = 1)

binormal_sens(truth, rating, spec)

binormal_spec(truth, rating, sens)

binormalLR_auc(
  truth,
  rating,
  partial = FALSE,
  min = 0,
  max = 1,
  normalize = FALSE
)

binormalLR_eu(truth, rating, slope = 1)

binormalLR_sens(truth, rating, spec)

binormalLR_spec(truth, rating, sens)

empirical_auc(
  truth,
  rating,
  partial = FALSE,
  min = 0,
  max = 1,
  normalize = FALSE
)

empirical_eu(truth, rating, slope = 1)

empirical_sens(truth, rating, spec)

empirical_spec(truth, rating, sens)

trapezoidal_auc(
  truth,
  rating,
  partial = FALSE,
  min = 0,
  max = 1,
  normalize = FALSE
)

trapezoidal_sens(truth, rating, spec)

trapezoidal_spec(truth, rating, sens)

Arguments

truth

vector of true binary statuses.

rating

vector of 0-1 binary ratings for the binary metrics and ranges of numeric ratings for the others.

partial

character string "sensitivity" or "specificity" for calculation of partial AUC, or FALSE for full AUC. Partial matching of the character strings is allowed. "specificity" results in area under the ROC curve between the given min and max specificity values, whereas "sensitivity" results in area to the right of the curve between the given sensitivity values.

min, max

minimum and maximum sensitivity or specificity values over which to calculate partial AUC.

normalize

logical indicating whether partial AUC is divided by the interval width (max - min) over which it is calculated.

slope

slope of the iso-utility line at which to compute expected utility of the ROC curve.

sens, spec

numeric sensitivity/specificity at which to calculate specificity/sensitivity.

Details

Performance metrics measure the degree to which higher case ratings are associated with positive case statuses, where positive status is taken to be the highest level of truth. Available metrics include area under the ROC curve (auc), expected utility of the ROC curve (eu) at a given iso-utility line (Abbey, 2013), sensitivity (sens) at a given specificity, and specificity (spec) at a given sensitivity.

Value

Returns a numeric value.

References

Abbey CK, Samuelson FW and Gallas BD (2013). Statistical power considerations for a utility endpoint in observer performance studies. Academic Radiology, 20: 798-806.

See Also

mrmc, srmc, stmc