statistics_calibratR {CalibratR} | R Documentation |
statistics_calibratR
Description
this method offers a variety of statistical evaluation methods for the output of the calibrate
method.
All returned error values represent mean error values over the n_seeds
times repeated 10-fold CV.
Usage
statistics_calibratR(calibrate_object, t.test_partitions = TRUE,
significance_models = TRUE)
Arguments
calibrate_object |
list that is returned from the |
t.test_partitions |
Performs a paired two sided t.test over the error values (ECE, CLE1, CLE0, MCE, AUC, sensitivity and specificity) from the
random partition splits comparing a possible significant difference in mean among the calibration models. All models and the original, scaled and transformed values are tested against each other.
The p_value and the effect size of the t.test are returned to the user. Can only be performed, if the |
significance_models |
returns important characteristics of the implemented calibration models, Default: TRUE |
Details
DETAILS
Value
An object of class list, with the following components:
mean_calibration |
mean of calibration error values (ECE_equal_width, MCE_equal_width, ECE_equal_freq, MCE_equal_freq, RMSE, Class 1 CLE, Class 0 CLE, Brier Score, Class 1 Brier Score, Class 0 Brier Score) over |
standard_deviation |
standard deviation of calibration error values over |
var_coeff_calibration |
variation coefficient of calibration error values over |
mean_discrimination |
mean of discrimination error (sensitivity, specificity, AUC, positive predictive value, negative predictive value, accuracy) values over |
sd_discrimination |
standard deviation of discrimination error values over |
var_coeff_discrimination |
variation coefficient of discrimination error values over |
t.test_calibration |
=list(p_value=t.test.calibration, effect_size=effect_size_calibration), only returned if t.test=TRUE |
t.test_discrimination |
=list(p_value=t.test.discrimination, effect_size=effect_size_discrimination), only returned if t.test=TRUE |
significance_models |
only returned if significance_models=TRUE |
n_seeds |
number of random data set partitions into training and test set for |
original_values |
list object that consists of the |
Author(s)
Johanna Schwarz
See Also
Examples
## Loading dataset in environment
data(example)
calibration_model <- example$calibration_model
statistics <- statistics_calibratR(calibration_model)