performance_equiv {LogRegEquiv}

performance_equiv function

Description

This function takes two logistic regression models M_A, M_B, test data, significance level \alpha and acceptable score degradation \delta_B. It checks whether the models perform equivalently on the test set and returns various figures.

Usage

performance_equiv(
  model_a,
  model_b,
  test_data,
  dv_index,
  delta_B = 1.1,
  alpha = 0.05
)

Arguments

model_a

logistic regression model M_A

model_b

logistic regression model M_B

test_data

testing dataset

dv_index

column number of the dependent variable

delta_B

acceptable score degradation (defaults to 1.1)

alpha

significance level \alpha (defaults to 0.05)

Value

equivalence

Are models M_A,M_B producing equivalent Brier scores for the given test data? (boolean)

brier_score_ac

M_A Brier score on the testing data

brier_score_bc

M_B Brier score on the testing data

diff_sd_l

SD of the lower Brier difference BS^A-\delta_B^2BS^B

diff_sd_u

SD of the upper Brier difference BS^A-\delta_B^{-2}BS^B

test_stat_l

t_L equivalence boundary for the test

test_stat_u

t_U equivalence boundary for the test

crit_val

a level-\alpha critical value for the test

delta_B

Calculated equivalence parameter

p_value_l

P-value for t_L

p_value_u

P-value for t_U