accuracyAtEachCutoff {petersenlab}

Accuracy at Each Cutoff.

Description

Find the accuracy at each possible cutoff. Actuals should be binary, where 1 = present and 0 = absent.

Usage

accuracyAtEachCutoff(
  predicted,
  actual,
  UH = NULL,
  UM = NULL,
  UCR = NULL,
  UFA = NULL
)

Arguments

predicted

vector of continuous predicted values.

actual

vector of binary actual values (1 = present and 0 = absent).

UH

(optional) utility of hits (true positives), specified as a value from 0-1, where 1 is the most highly valued and 0 is the least valued.

UM

(optional) utility of misses (false negatives), specified as a value from 0-1, where 1 is the most highly valued and 0 is the least valued.

UCR

(optional) utility of correct rejections (true negatives), specified as a value from 0-1, where 1 is the most highly valued and 0 is the least valued.

UFA

(optional) utility of false positives (false positives), specified as a value from 0-1, where 1 is the most highly valued and 0 is the least valued.

Details

Compute accuracy indices of predicted values in relation to actual values at each possible cutoff by specifying the predicted values and actual values. The target condition is considered present at or above each cutoff value. Optionally, you can specify the utility of hits, misses, correct rejections, and false alarms to calculate the overall utility of each possible cutoff.

Value

• cutoff = the cutoff specified

• TP = true positives

• TN = true negatives

• FP = false positives

• FN = false negatives

• SR = selection ratio

• BR = base rate

• percentAccuracy = percent accuracy

• percentAccuracyByChance = percent accuracy by chance

• percentAccuracyPredictingFromBaseRate = percent accuracy from predicting from the base rate

• RIOC = relative improvement over chance

• relativeImprovementOverPredictingFromBaseRate = relative improvement over predicting from the base rate

• SN = sensitivty

• SP = specificity

• TPrate = true positive rate

• TNrate = true negative rate

• FNrate = false negative rate

• FPrate = false positive rate

• HR = hit rate

• FAR = false alarm rate

• PPV = positive predictive value

• NPV = negative predictive value

• FDR = false discovery rate

• FOR = false omission rate

• youdenJ = Youden's J statistic

• balancedAccuracy = balanced accuracy

• f1Score = F1-score

• mcc = Matthews correlation coefficient

• diagnosticOddsRatio = diagnostic odds ratio

• positiveLikelihoodRatio = positive likelihood ratio

• negativeLikelhoodRatio = negative likelihood ratio

• dPrimeSDT = d-Prime index from signal detection theory

• betaSDT = beta index from signal detection theory

• cSDT = c index from signal detection theory

• aSDT = a index from signal detection theory

• bSDT = b index from signal detection theory

• differenceBetweenPredictedAndObserved = difference between predicted and observed values

• informationGain = information gain

• overallUtility = overall utility (if utilities were specified)

See Also

Other accuracy: accuracyAtCutoff(), accuracyOverall(), nomogrammer(), optimalCutoff()

Examples

# Prepare Data
data("USArrests")
USArrests$highMurderState <- NA
USArrests$highMurderState[which(USArrests$Murder >= 10)] <- 1
USArrests$highMurderState[which(USArrests$Murder < 10)] <- 0

# Calculate Accuracy
accuracyAtEachCutoff(predicted = USArrests$Assault,
  actual = USArrests$highMurderState)
accuracyAtEachCutoff(predicted = USArrests$Assault,
  actual = USArrests$highMurderState,
  UH = 1, UM = 0, UCR = .9, UFA = 0)