plot.calibration_simplex {CalSim} | R Documentation |
Plot Calibration Simplex
## S3 method for class 'calibration_simplex' plot( x, true_error = TRUE, error_scale = 0.3, min_bin_freq = 10, plot_error_scale = TRUE, scale_area = NULL, indicate_bins = TRUE, category_labels = c("1", "2", "3"), use_pvals = FALSE, alphas = c(0.1, 0.01), ... )
x |
Object of class |
true_error |
Logical, specifying whether to use true miscalibration errors or approximate miscalibration errors. |
error_scale |
A number specifying the magnitude of the miscalibration errors (greater 0, usually should be less than 1, cf. note below). |
min_bin_freq |
A number. Lower bound for (absolute) frequencies, i.e. how many observations have to lie in a bin for it to be plotted. |
plot_error_scale |
Logical, specifying whether to plot a scale showing the magnitude of miscalibration errors. |
scale_area |
Optional. A number by which the areas of the points are scaled. Use if points are to small or to big. |
indicate_bins |
Logical, specifying whether to connect points to their respective bin (center of hexagon). |
category_labels |
A vector of length 3 containing the category names, e.g. |
use_pvals |
Logical, determines whether multinomial p-values are used for uncertainty quantification, see details. |
alphas |
Vector of length 2 with values 1 > |
... |
Arguments concerning the title (e.g. |
If multinomial p-values are used (use_pvals = TRUE
), the dots are colored in the following way:
Blue: p-value greater alphas[1]
(0.1 by default).
Orange: p-value between alphas[1]
and alphas[2]
(0.1 and 0.01 by default)
Red: p-value less than alphas[2]
(0.01 by default)
Black: p-value is exactly 0. This only happens if a category which is assigned 0 probability realizes.
Many small p-values (orange and red dots) indicate miscalibrated predictions, whereas many blue dots indicate that the predictions
may in fact be calibrated. WARNING: The use of the multinomial p-values is more of an experimental feature and may not yield reliable
p-values, especially if n
is small.
For details regarding the calculation of the p-values see also calibration_simplex
.
For details on the meaning of the error scale, cf. Wilks, 2013, especially Fig. 2. Note that the miscalibration error in each category is in "probability units" (as it is the average difference in relative frequency and forecast probability in each bin).