AutoScore_parsimony {AutoScore} | R Documentation |
AutoScore STEP(ii): Select the best model with parsimony plot (AutoScore Modules 2+3+4)
Description
AutoScore STEP(ii): Select the best model with parsimony plot (AutoScore Modules 2+3+4)
Usage
AutoScore_parsimony(
train_set,
validation_set,
rank,
max_score = 100,
n_min = 1,
n_max = 20,
cross_validation = FALSE,
fold = 10,
categorize = "quantile",
quantiles = c(0, 0.05, 0.2, 0.8, 0.95, 1),
max_cluster = 5,
do_trace = FALSE,
auc_lim_min = 0.5,
auc_lim_max = "adaptive"
)
Arguments
train_set |
A processed |
validation_set |
A processed |
rank |
the raking result generated from AutoScore STEP(i) |
max_score |
Maximum total score (Default: 100). |
n_min |
Minimum number of selected variables (Default: 1). |
n_max |
Maximum number of selected variables (Default: 20). |
cross_validation |
If set to |
fold |
The number of folds used in cross validation (Default: 10). Available if |
categorize |
Methods for categorize continuous variables. Options include "quantile" or "kmeans" (Default: "quantile"). |
quantiles |
Predefined quantiles to convert continuous variables to categorical ones. (Default: c(0, 0.05, 0.2, 0.8, 0.95, 1)) Available if |
max_cluster |
The max number of cluster (Default: 5). Available if |
do_trace |
If set to TRUE, all results based on each fold of cross-validation would be printed out and plotted (Default: FALSE). Available if |
auc_lim_min |
Min y_axis limit in the parsimony plot (Default: 0.5). |
auc_lim_max |
Max y_axis limit in the parsimony plot (Default: "adaptive"). |
Details
This is the second step of the general AutoScore workflow, to generate the parsimony plot to help select a parsimonious model.
In this step, it goes through AutoScore Module 2,3 and 4 multiple times and to evaluate the performance under different variable list.
The generated parsimony plot would give researcher an intuitive figure to choose the best models.
If data size is small (ie, <5000), an independent validation set may not be a wise choice. Then, we suggest using cross-validation
to maximize the utility of data. Set cross_validation=TRUE
. Run vignette("Guide_book", package = "AutoScore")
to see the guidebook or vignette.
Value
List of AUC value for different number of variables
References
Xie F, Chakraborty B, Ong MEH, Goldstein BA, Liu N, AutoScore: A Machine Learning-Based Automatic Clinical Score Generator and Its Application to Mortality Prediction Using Electronic Health Records, JMIR Med Inform 2020;8(10):e21798, doi: 10.2196/21798
See Also
AutoScore_rank
, AutoScore_weighting
, AutoScore_fine_tuning
, AutoScore_testing
, Run vignette("Guide_book", package = "AutoScore")
to see the guidebook or vignette.
Examples
# see AutoScore Guidebook for the whole 5-step workflow
data("sample_data")
names(sample_data)[names(sample_data) == "Mortality_inpatient"] <- "label"
out_split <- split_data(data = sample_data, ratio = c(0.7, 0.1, 0.2))
train_set <- out_split$train_set
validation_set <- out_split$validation_set
ranking <- AutoScore_rank(train_set, ntree=100)
AUC <- AutoScore_parsimony(
train_set,
validation_set,
rank = ranking,
max_score = 100,
n_min = 1,
n_max = 20,
categorize = "quantile",
quantiles = c(0, 0.05, 0.2, 0.8, 0.95, 1)
)