mle_recal {BRcal}

Recalibration via Maximum Likelihood Estimates (MLEs)

Description

MLE recalibrate (i.e. LLO-adjust via \hat{\delta}_{MLE} and \hat{\gamma}_{MLE} as specified in Guthrie and Franck (2024).

Usage

mle_recal(x, y, probs_only = FALSE, event = 1, optim_details = TRUE, ...)

Arguments

x

a numeric vector of predicted probabilities of an event. Must only contain values in [0,1].

y

a vector of outcomes corresponding to probabilities in x. Must only contain two unique values (one for "events" and one for "non-events"). By default, this function expects a vector of 0s (non-events) and 1s (events).

probs_only

Logical. If TRUE, mle_recal() returns only the vector of MLE recalibrated probabilities.

event

Value in y that represents an "event". Default value is 1.

optim_details

Logical. If TRUE, the list returned by optim when minimizing the negative log likelihood is also returned by this function.

...

Additional arguments to be passed to optim.

Details

Given a set of probability predictions x, the corresponding MLE recalibrated set is c(x; \hat{\delta}_{MLE}, \hat{\gamma}_{MLE}) (see LLO).

Value

If probs_only=TRUE, mle_recal()returns a vector of MLE recalibrated probabilities. Otherwise, mle_recal() returns a list with the following attributes:

probs

The vector of MLE recalibrated probabilities.

MLEs

Maximum likelihood estimates for \delta and \gamma.

optim_details

If optim_details = TRUE, the list returned by optim when minimizing the negative log likelihood, includes convergence information, number of iterations, and achieved negative log likelihood value and MLEs. This arguement is ignored when probs_only=TRUE.

References

Guthrie, A. P., and Franck, C. T. (2024) Boldness-Recalibration for Binary Event Predictions, The American Statistician 1-17.

Examples

# Simulate 100 predicted probabilities
x <- runif(100)
# Simulated 100 binary event outcomes using `x` 
y <- rbinom(100, 1, x)

# MLE recalibrate `x`
mle_recal(x, y, optim_details=FALSE)  

# Just return the vector of MLE recalibrated probabilities
x_mle <- mle_recal(x, y, optim_details=FALSE, probs_only=TRUE)
x_mle

plot(x, x_mle)

# Use optim_details = TRUE to see returned info from call to optim(),
# details useful for checking convergence
mle_recal(x, y, optim_details=TRUE)  # no convergence problems in this example

# Use different start value in `optim()` call, start at delta = 5, gamma = 5
mle_recal(x, y, optim_details=TRUE, par=c(5,5))

# Use `L-BFGS-B` instead of `Nelder-Mead` 
mle_recal(x, y, optim_details=TRUE, method = "L-BFGS-B")  # same result

# What if events are defined by text instead of 0 or 1? 
y2 <- ifelse(y==0, "Loss", "Win")
mle_recal(x, y2, event="Win", optim_details=FALSE)  # same result

# What if we're interested in the probability of loss instead of win?
x2 <- 1 - x
mle_recal(x2, y2, event="Loss", optim_details=FALSE)