textPredictTest {text} | R Documentation |
Significance testing correlations If only y1 is provided a t-test is computed, between the absolute error from yhat1-y1 and yhat2-y1.
Description
If y2 is provided a bootstrapped procedure is used to compare the correlations between y1 and yhat1 versus y2 and yhat2. This is achieved by creating two distributions of correlations using bootstrapping; and then finally compute the distributions overlap.
Usage
textPredictTest(
y1,
y2,
yhat1,
yhat2,
method = "t-test",
statistic = "correlation",
paired = TRUE,
event_level = "first",
bootstraps_times = 1000,
seed = 6134,
...
)
Arguments
y1 |
The observed scores (i.e., what was used to predict when training a model). |
y2 |
The second observed scores (default = NULL; i.e., for when comparing models that are predicting different outcomes. In this case a bootstrap procedure is used to create two distributions of correlations that are compared (see description above). |
yhat1 |
The predicted scores from model 1. |
yhat2 |
The predicted scores from model 2 that will be compared with model 1. |
method |
Set "t-test" if comparing predictions from models that predict the SAME outcome. Set "bootstrap" if comparing predictions from models that predict DIFFERENT outcomes or comparison from logistic regression computing AUC distributions. |
statistic |
Character ("correlation", "auc") describing statistic to be compared in bootstrapping. |
paired |
Paired test or not in stats::t.test (default TRUE). |
event_level |
Character "first" or "second" for computing the auc in the bootstrap. |
bootstraps_times |
Number of bootstraps (when providing y2). |
seed |
Set seed. |
... |
Settings from stats::t.test or overlapping::overlap (e.g., plot = TRUE). |
Value
Comparison of correlations either a t-test or the overlap of a bootstrapped procedure (see $OV).
See Also
see textTrain
textPredict
Examples
# Example random data
y1 <- runif(10)
yhat1 <- runif(10)
y2 <- runif(10)
yhat2 <- runif(10)
boot_test <- textPredictTest(y1, y2, yhat1, yhat2)