statistics_calibratR {CalibratR} | R Documentation |

this method offers a variety of statistical evaluation methods for the output of the `calibrate`

method.
All returned error values represent mean error values over the `n_seeds`

times repeated 10-fold CV.

```
statistics_calibratR(calibrate_object, t.test_partitions = TRUE,
significance_models = TRUE)
```

`calibrate_object` |
list that is returned from the |

`t.test_partitions` |
Performs a paired two sided t.test over the error values (ECE, CLE1, CLE0, MCE, AUC, sensitivity and specificity) from the
random partition splits comparing a possible significant difference in mean among the calibration models. All models and the original, scaled and transformed values are tested against each other.
The p_value and the effect size of the t.test are returned to the user. Can only be performed, if the |

`significance_models` |
returns important characteristics of the implemented calibration models, Default: TRUE |

DETAILS

An object of class list, with the following components:

`mean_calibration` |
mean of calibration error values (ECE_equal_width, MCE_equal_width, ECE_equal_freq, MCE_equal_freq, RMSE, Class 1 CLE, Class 0 CLE, Brier Score, Class 1 Brier Score, Class 0 Brier Score) over |

`standard_deviation` |
standard deviation of calibration error values over |

`var_coeff_calibration` |
variation coefficient of calibration error values over |

`mean_discrimination` |
mean of discrimination error (sensitivity, specificity, AUC, positive predictive value, negative predictive value, accuracy) values over |

`sd_discrimination` |
standard deviation of discrimination error values over |

`var_coeff_discrimination` |
variation coefficient of discrimination error values over |

`t.test_calibration` |
=list(p_value=t.test.calibration, effect_size=effect_size_calibration), only returned if t.test=TRUE |

`t.test_discrimination` |
=list(p_value=t.test.discrimination, effect_size=effect_size_discrimination), only returned if t.test=TRUE |

`significance_models` |
only returned if significance_models=TRUE |

`n_seeds` |
number of random data set partitions into training and test set for |

`original_values` |
list object that consists of the |

Johanna Schwarz

```
## Loading dataset in environment
data(example)
calibration_model <- example$calibration_model
statistics <- statistics_calibratR(calibration_model)
```

[Package *CalibratR* version 0.1.2 Index]