pool_compare_models {psfmi}

Compare the fit and performance of prediction models across Multipy Imputed data

Description

pool_compare_model Compares the fit and performance of prediction models in multiply imputed data sets by using clinical important performance measures

Usage

pool_compare_models(
  pobj,
  compare.predictors = NULL,
  compare.group = NULL,
  cutoff = 0.5,
  boot_auc = FALSE,
  nboot = 1000
)

Arguments

pobj

An object of class pmods (pooled models), produced by a previous call to psfmi_lr.

compare.predictors

Character vector with the names of the predictors that are compared. See details.

compare.group

Character vector with the names of the group of predictors that are compared. See details.

cutoff

A numerical scalar. Cutoff used for the categorical NRI value. More than one cutoff value can be used.

boot_auc

If TRUE the standard error of the AUC is calculated with stratified bootstrapping. If FALSE (is default), the standard error is calculated with De Long's method.

nboot

A numerical scalar. The number of bootstrap samples for the AUC standard error, used when boot_auc is TRUE. Default is 1000.

Details

The fit of the models are compared by using the D3 method for pooling Likelihood ratio statistics (method of Meng and Rubin). The pooled AIC difference is calculated according to the formula AIC = D - 2*p, where D is the pooled likelihood ratio tests of constrained models (numerator in D3 statistic) and p is the difference in number of parameters between the full and restricted models that are compared. The pooled AUC difference is calculated, after the standard error is obtained in each imputed data set by method DeLong or bootstrapping. The NRI categorical and continuous and IDI are calculated in each imputed data set and pooled.

Value

An object from which the following objects can be extracted:

• DR_stats p-value of the D3 statistic, the D3 statistic, LRT fixed is the likelihood Ratio test value of the constrained models.

• stats_compare Mean of LogLik0, LogLik1, AIC0, AIC1, AIC_diff values of the restricted (containing a 0) and full models (containing a 1).

• NRI pooled values for the categorical and continuous Net Reclassification improvement values and the Integrated Discrimination improvement.

• AUC_stats Pooled Area Under the Curve of restricted and full models.

• AUC_diff Pooled difference in AUC.

• formula_test regression formula of full model.

• cutoff Cutoff value used for reclassification values.

• formula_null regression formula of null model

• compare_predictors Predictors used in full model.

• compare_group group of predictors used in full model.

References

Eekhout I, van de Wiel MA, Heymans MW. Methods for significance testing of categorical covariates in logistic regression models after multiple imputation: power and applicability analysis. BMC Med Res Methodol. 2017;17(1):129.

Consentino F, Claeskens G. Order Selection tests with multiply imputed data Computational Statistics and Data Analysis.2010;54:2284-2295.

Examples

 pool_lr <- psfmi_lr(data=lbpmilr, p.crit = 1, direction="FW", nimp=10, impvar="Impnr", 
 Outcome="Chronic", predictors=c("Radiation"), cat.predictors = ("Satisfaction"),
 int.predictors = NULL, spline.predictors="Tampascale", nknots=3, method="D1")

 res_compare <- pool_compare_models(pool_lr, compare.predictors = c("Pain", "Duration", 
 "Function"), cutoff = 0.4)
 res_compare