givitiCalibrationBelt {givitiR}

Calibration Belt

Description

givitiCalibrationBelt implements the computations necessary to plot the calibration belt.

Usage

givitiCalibrationBelt(o, e, devel, subset = NULL, confLevels = c(0.8, 0.95),
  thres = 0.95, maxDeg = 4, nPoints = 200)

Arguments

o

A numeric vector representing the binary outcomes. The elements must assume only the values 0 or 1. The predictions in e must represent the probability of the event coded as 1.

e

A numeric vector containing the predictions of the model under evaluation. The elements must be numeric and between 0 and 1. The lenght of the vector must be equal to the length of the vector o.

devel

A character string specifying if the model has been fit on the same dataset under evaluation (internal) or if the model has been developed on an external sample (external). See also the 'Details' section.

subset

An optional boolean vector specifying the subset of observations to be considered.

confLevels

A numeric vector containing the confidence levels of the calibration belt. The default values are set to .80 and .95.

thres

A numeric scalar between 0 and 1 representing 1 - the significance level adopted in the forward selection. By default is set to 0.95.

maxDeg

The maximum degree considered in the forward selection. By default is set to 4.

nPoints

A numeric scalar indicating the number of points to be considered to plot the calibration belt. The default value is 200.

Details

The calibration belt and the associated test can be used both to evaluate the calibration of the model in external samples or in the development dataset. However, the two cases have different requirements. When a model is evaluated on independent samples, the calibration belt and the related test can be applied whatever is the method used to fit the model. Conversely, they can be used on the development set only if the model is fitted with logistic regression.

Value

An object of class givitiCalibrationBelt. After computing the calibration belt with the present function, the plot method can be used to plot the calibration belt. The object returned is a list that contains the following components:

n

The size of the sample evaluated in the analysis, after discarding missing values from the vectors o and e.

resultCheck

Result of the check on the data. If the data are compatible with the construction of the calibration belt, the value is the boolean TRUE. Otherwise, the element contain a character string describing the problem found.

m

The degree of the polynomial at the end of the forward selection.

statistic

The value of the test's statistic.

p.value

The p-value of the test.

seqP

The vector of the probabilities where the points of the calibration belt has been evaluated.

minMax

A list with two elements named min and max representing the minimum and maximum probabilities in the model under evaluation

confLevels

The vector containing the confidence levels of the calibration belt.

intersByConfLevel

A list whose elements report the intervals where the calibration belt is significantly over/under the bisector for each confidence level in confLevels.

See Also

plot.givitiCalibrationBelt to plot the calibaration belt and givitiCalibrationTest to perform the associated calibration test.

Examples

#Random by-construction well calibrated model
e <- runif(100)
o <- rbinom(100, size = 1, prob = e)
cb <- givitiCalibrationBelt(o, e, "external")
plot(cb)

#Random by-construction poorly calibrated model
e <- runif(100)
o <- rbinom(100, size = 1, prob = logistic(logit(e)+2))
cb <- givitiCalibrationBelt(o, e, "external")
plot(cb)