optimalCutoff {petersenlab}

Optimal Cutoff.

Description

Find the optimal cutoff for different aspects of accuracy. Actuals should be binary, where 1 = present and 0 = absent.

Usage

optimalCutoff(predicted, actual, UH = NULL, UM = NULL, UCR = NULL, UFA = NULL)

Arguments

predicted

vector of continuous predicted values.

actual

vector of binary actual values (1 = present and 0 = absent).

UH

(optional) utility of hits (true positives), specified as a value from 0-1, where 1 is the most highly valued and 0 is the least valued.

UM

(optional) utility of misses (false negatives), specified as a value from 0-1, where 1 is the most highly valued and 0 is the least valued.

UCR

(optional) utility of correct rejections (true negatives), specified as a value from 0-1, where 1 is the most highly valued and 0 is the least valued.

UFA

(optional) utility of false positives (false positives), specified as a value from 0-1, where 1 is the most highly valued and 0 is the least valued.

Details

Identify the optimal cutoff for different aspects of accuracy of predicted values in relation to actual values by specifying the predicted values and actual values. Optionally, you can specify the utility of hits, misses, correct rejections, and false alarms to calculate the overall utility of each possible cutoff.

Value

The optimal cutoff and optimal accuracy index at that cutoff based on:

• percentAccuracy = percent accuracy

• percentAccuracyByChance = percent accuracy by chance

• RIOC = relative improvement over chance

• relativeImprovementOverPredictingFromBaseRate = relative improvement over predicting from the base rate

• PPV = positive predictive value

• NPV = negative predictive value

• youdenJ = Youden's J statistic

• balancedAccuracy = balanced accuracy

• f1Score = F1-score

• mcc = Matthews correlation coefficient

• diagnosticOddsRatio = diagnostic odds ratio

• positiveLikelihoodRatio = positive likelihood ratio

• negativeLikelhoodRatio = negative likelihood ratio

• dPrimeSDT = d-Prime index from signal detection theory

• betaSDT = beta index from signal detection theory

• cSDT = c index from signal detection theory

• aSDT = a index from signal detection theory

• bSDT = b index from signal detection theory

• differenceBetweenPredictedAndObserved = difference between predicted and observed values

• informationGain = information gain

• overallUtility = overall utility (if utilities were specified)

See Also

Other accuracy: accuracyAtCutoff(), accuracyAtEachCutoff(), accuracyOverall(), nomogrammer()

Examples

# Prepare Data
data("USArrests")
USArrests$highMurderState <- NA
USArrests$highMurderState[which(USArrests$Murder >= 10)] <- 1
USArrests$highMurderState[which(USArrests$Murder < 10)] <- 0

# Determine Optimal Cutoff
optimalCutoff(predicted = USArrests$Assault,
  actual = USArrests$highMurderState)
optimalCutoff(predicted = USArrests$Assault,
  actual = USArrests$highMurderState,
  UH = 1, UM = 0, UCR = .9, UFA = 0)