boot_test {cutpointr} | R Documentation |

## Test for equivalence of a metric

### Description

This function performs a significance test based on the bootstrap results
of cutpointr to test whether a chosen metric is equal between subgroups
or between two cutpointr objects. The test statistic is calculated as
the standardized difference of the metric between groups. If `x`

contains subgroups, the test is run on all possible pairings of subgroups.
An additional adjusted p-value is returned in that case.

### Usage

```
boot_test(x, y = NULL, variable = "AUC", in_bag = TRUE, correction = "holm")
```

### Arguments

`x` |
A cutpointr object with bootstrap results |

`y` |
If x does not contain subgroups another cutpointr object |

`variable` |
The variable for testing |

`in_bag` |
Whether the in-bag or out-of-bag results should be used for testing |

`correction` |
The type of correction for multiple testing. Possible values are as in p.adjust.methods |

### Details

The variable name is looked up in the columns of the bootstrap results
where the suffixes _b and _oob indicate in-bag and out-of-bag estimates,
respectively (controlled via the `in_bag`

argument).
Possible values are optimal_cutpoint, AUC,
acc, sensitivity, specificity, and the metric that was selected
in `cutpointr`

. Note that there is no "out-of-bag optimal cutpoint", so
when selecting `variable = optimal_cutpoint`

the test will be based on
the in-bag data.

The test statistic is calculated as z = (t1 - t2) / sd(t1 - t2) where t1 and t2 are the metric values on the full sample and sd(t1 - t2) is the standard deviation of the differences of the metric values per bootstrap repetition. The test is two-sided.

If two cutpointr objects are compared and the numbers of bootstrap repetitions differ, the smaller number will be used.

Since pairwise differences are calculated for this test, the test function does not support multiple optimal cutpoints, because it is unclear how the differences should be calculated in that case.

### Value

A data.frame (a tibble) with the columns test_var, p, d, sd_d, z and in_bag. If a grouped cutpointr object was tested, the additional columns subgroup1, subgroup2 and p_adj are returned.

### Source

Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & MÃ¼ller, M. (2011). pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics, 12(1), 77. https://doi.org/10.1186/1471-2105-12-77

### See Also

Other main cutpointr functions:
`add_metric()`

,
`boot_ci()`

,
`cutpointr()`

,
`multi_cutpointr()`

,
`predict.cutpointr()`

,
`roc()`

### Examples

```
## Not run:
library(cutpointr)
library(dplyr)
set.seed(734)
cp_f <- cutpointr(suicide %>% filter(gender == "female"), dsi, suicide,
boot_runs = 1000, boot_stratify = TRUE)
set.seed(928)
cp_m <- cutpointr(suicide %>% filter(gender == "male"), dsi, suicide,
boot_runs = 1000, boot_stratify = TRUE)
# No significant differences:
boot_test(cp_f, cp_m, AUC, in_bag = TRUE)
boot_test(cp_f, cp_m, sum_sens_spec, in_bag = FALSE)
set.seed(135)
cp <- cutpointr(suicide, dsi, suicide, gender, boot_runs = 1000,
boot_stratify = TRUE)
# Roughly same result as above:
boot_test(cp, variable = AUC, in_bag = TRUE)
boot_test(cp, variable = sum_sens_spec, in_bag = FALSE)
## End(Not run)
```

*cutpointr*version 1.1.2 Index]