| permuteMeasEq {semTools} | R Documentation |
Permutation Randomization Tests of Measurement Equivalence and Differential Item Functioning (DIF)
Description
The function permuteMeasEq provides tests of hypotheses involving
measurement equivalence, in one of two frameworks: multigroup CFA or MIMIC
models.
Usage
permuteMeasEq(nPermute, modelType = c("mgcfa", "mimic"), con, uncon = NULL,
null = NULL, param = NULL, freeParam = NULL, covariates = NULL,
AFIs = NULL, moreAFIs = NULL, maxSparse = 10, maxNonconv = 10,
showProgress = TRUE, warn = -1, datafun, extra,
parallelType = c("none", "multicore", "snow"), ncpus = NULL, cl = NULL,
iseed = 12345)
Arguments
nPermute |
An integer indicating the number of random permutations used to form empirical distributions under the null hypothesis. |
modelType |
A character string indicating type of model employed:
multiple-group CFA ( |
con |
The constrained |
uncon |
Optional. The unconstrained |
null |
Optional. A |
param |
An optional character vector or list of character vectors
indicating which parameters the user would test for DIF following a
rejection of the omnibus null hypothesis tested using
( |
freeParam |
An optional character vector, silently ignored when
|
covariates |
An optional character vector, only applicable when
|
AFIs |
A character vector indicating which alternative fit indices (or
chi-squared itself) are to be used to test the multiparameter omnibus null
hypothesis that the constraints specified in |
moreAFIs |
Optional. A character vector indicating which (if any)
alternative fit indices returned by |
maxSparse |
Only applicable when |
maxNonconv |
An integer indicating the maximum number of consecutive
times that a random permutation can yield a sample for which the model does
not converge on a solution. If such a sample occurs, permutation is
attempted repeatedly until a sample is obtained for which the model does
converge. If |
showProgress |
Logical. Indicating whether to display a progress bar
while permuting. Silently set to |
warn |
Sets the handling of warning messages when fitting model(s) to
permuted data sets. See |
datafun |
An optional function that can be applied to the data
(extracted from |
extra |
An optional function that can be applied to any (or all) of the
fitted lavaan objects ( |
parallelType |
The type of parallel operation to be used (if any). The
default is |
ncpus |
Integer: number of processes to be used in parallel operation.
If |
cl |
An optional parallel or snow cluster for use when
|
iseed |
Integer: Only used to set the states of the RNG when using
parallel options, in which case |
Details
The function permuteMeasEq provides tests of hypotheses involving
measurement equivalence, in one of two frameworks:
1 For multiple-group CFA models, provide a pair of nested lavaan objects, the less constrained of which (
uncon) freely estimates a set of measurement parameters (e.g., factor loadings, intercepts, or thresholds; specified inparam) in all groups, and the more constrained of which (con) constrains those measurement parameters to equality across groups. Group assignment is repeatedly permuted and the models are fit to each permutation, in order to produce an empirical distribution under the null hypothesis of no group differences, both for (a) changes in user-specified fit measures (seeAFIsandmoreAFIs) and for (b) the maximum modification index among the user-specified equality constraints. Configural invariance can also be tested by providing that fitted lavaan object toconand leavinguncon = NULL, in which caseparammust beNULLas well.2 In MIMIC models, one or a set of continuous and/or discrete
covariatescan be permuted, and a constrained model is fit to each permutation in order to provide a distribution of any fit measures (namely, the maximum modification index among fixed parameters inparam) under the null hypothesis of measurement equivalence across levels of those covariates.
In either framework, modification indices for equality constraints or fixed
parameters specified in param are calculated from the constrained
model (con) using the function lavTestScore.
For multiple-group CFA models, the multiparameter omnibus null hypothesis of
measurement equivalence/invariance is that there are no group differences in
any measurement parameters (of a particular type). This can be tested using
the anova method on nested lavaan objects, as seen in the
output of measurementInvariance, or by inspecting
the change in alternative fit indices (AFIs) such as the CFI. The
permutation randomization method employed by permuteMeasEq generates
an empirical distribution of any AFIs under the null hypothesis, so
the user is not restricted to using fixed cutoffs proposed by Cheung &
Rensvold (2002), Chen (2007), or Meade, Johnson, & Braddy (2008).
If the multiparameter omnibus null hypothesis is rejected, partial
invariance can still be established by freeing invalid equality constraints,
as long as equality constraints are valid for at least two indicators per
factor. Modification indices can be calculated from the constrained model
(con), but multiple testing leads to inflation of Type I error rates.
The permutation randomization method employed by permuteMeasEq
creates a distribution of the maximum modification index if the null
hypothesis is true, which allows the user to control the familywise Type I
error rate in a manner similar to Tukey's q (studentized range)
distribution for the Honestly Significant Difference (HSD) post hoc test.
For MIMIC models, DIF can be tested by comparing modification indices of
regression paths to the permutation distribution of the maximum modification
index, which controls the familywise Type I error rate. The MIMIC approach
could also be applied with multiple-group models, but the grouping variable
would not be permuted; rather, the covariates would be permuted separately
within each group to preserve between-group differences. So whether
parameters are constrained or unconstrained across groups, the MIMIC
approach is only for testing null hypotheses about the effects of
covariates on indicators, controlling for common factors.
In either framework, lavaan's group.label
argument is used to preserve the order of groups seen in con when
permuting the data.
Value
The permuteMeasEq object representing the results of
testing measurement equivalence (the multiparameter omnibus test) and DIF
(modification indices), as well as diagnostics and any extra output.
Author(s)
Terrence D. Jorgensen (University of Amsterdam; TJorgensen314@gmail.com)
References
Papers about permutation tests of measurement equivalence:
Jorgensen, T. D., Kite, B. A., Chen, P.-Y., & Short, S. D. (2018). Permutation randomization methods for testing measurement equivalence and detecting differential item functioning in multiple-group confirmatory factor analysis. Psychological Methods, 23(4), 708–728. doi:10.1037/met0000152
Kite, B. A., Jorgensen, T. D., & Chen, P.-Y. (2018). Random permutation testing applied to measurement invariance testing with ordered-categorical indicators. Structural Equation Modeling 25(4), 573–587. doi:10.1080/10705511.2017.1421467
Jorgensen, T. D. (2017). Applying permutation tests and multivariate modification indices to configurally invariant models that need respecification. Frontiers in Psychology, 8(1455). doi:10.3389/fpsyg.2017.01455
Additional reading:
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. doi:10.1080/10705510701301834
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. doi:10.1207/S15328007SEM0902_5
Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93(3), 568–592. doi:10.1037/0021-9010.93.3.568
Widamin, K. F., & Thompson, J. S. (2003). On specifying the null model for incremental fit indices in structural equation modeling. Psychological Methods, 8(1), 16–37. doi:10.1037/1082-989X.8.1.16
See Also
TukeyHSD, lavTestScore,
measurementInvariance,
measurementInvarianceCat
Examples
## Not run:
########################
## Multiple-Group CFA ##
########################
## create 3-group data in lavaan example(cfa) data
HS <- lavaan::HolzingerSwineford1939
HS$ageGroup <- ifelse(HS$ageyr < 13, "preteen",
ifelse(HS$ageyr > 13, "teen", "thirteen"))
## specify and fit an appropriate null model for incremental fit indices
mod.null <- c(paste0("x", 1:9, " ~ c(T", 1:9, ", T", 1:9, ", T", 1:9, ")*1"),
paste0("x", 1:9, " ~~ c(L", 1:9, ", L", 1:9, ", L", 1:9, ")*x", 1:9))
fit.null <- cfa(mod.null, data = HS, group = "ageGroup")
## fit target model with varying levels of measurement equivalence
mod.config <- '
visual =~ x1 + x2 + x3
textual =~ x4 + x5 + x6
speed =~ x7 + x8 + x9
'
fit.config <- cfa(mod.config, data = HS, std.lv = TRUE, group = "ageGroup")
fit.metric <- cfa(mod.config, data = HS, std.lv = TRUE, group = "ageGroup",
group.equal = "loadings")
fit.scalar <- cfa(mod.config, data = HS, std.lv = TRUE, group = "ageGroup",
group.equal = c("loadings","intercepts"))
####################### Permutation Method
## fit indices of interest for multiparameter omnibus test
myAFIs <- c("chisq","cfi","rmsea","mfi","aic")
moreAFIs <- c("gammaHat","adjGammaHat")
## Use only 20 permutations for a demo. In practice,
## use > 1000 to reduce sampling variability of estimated p values
## test configural invariance
set.seed(12345)
out.config <- permuteMeasEq(nPermute = 20, con = fit.config)
out.config
## test metric equivalence
set.seed(12345) # same permutations
out.metric <- permuteMeasEq(nPermute = 20, uncon = fit.config, con = fit.metric,
param = "loadings", AFIs = myAFIs,
moreAFIs = moreAFIs, null = fit.null)
summary(out.metric, nd = 4)
## test scalar equivalence
set.seed(12345) # same permutations
out.scalar <- permuteMeasEq(nPermute = 20, uncon = fit.metric, con = fit.scalar,
param = "intercepts", AFIs = myAFIs,
moreAFIs = moreAFIs, null = fit.null)
summary(out.scalar)
## Not much to see without significant DIF.
## Try using an absurdly high alpha level for illustration.
outsum <- summary(out.scalar, alpha = .50)
## notice that the returned object is the table of DIF tests
outsum
## visualize permutation distribution
hist(out.config, AFI = "chisq")
hist(out.metric, AFI = "chisq", nd = 2, alpha = .01,
legendArgs = list(x = "topright"))
hist(out.scalar, AFI = "cfi", printLegend = FALSE)
####################### Extra Output
## function to calculate expected change of Group-2 and -3 latent means if
## each intercept constraint were released
extra <- function(con) {
output <- list()
output["x1.vis2"] <- lavTestScore(con, release = 19:20, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[70]
output["x1.vis3"] <- lavTestScore(con, release = 19:20, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[106]
output["x2.vis2"] <- lavTestScore(con, release = 21:22, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[70]
output["x2.vis3"] <- lavTestScore(con, release = 21:22, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[106]
output["x3.vis2"] <- lavTestScore(con, release = 23:24, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[70]
output["x3.vis3"] <- lavTestScore(con, release = 23:24, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[106]
output["x4.txt2"] <- lavTestScore(con, release = 25:26, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[71]
output["x4.txt3"] <- lavTestScore(con, release = 25:26, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[107]
output["x5.txt2"] <- lavTestScore(con, release = 27:28, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[71]
output["x5.txt3"] <- lavTestScore(con, release = 27:28, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[107]
output["x6.txt2"] <- lavTestScore(con, release = 29:30, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[71]
output["x6.txt3"] <- lavTestScore(con, release = 29:30, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[107]
output["x7.spd2"] <- lavTestScore(con, release = 31:32, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[72]
output["x7.spd3"] <- lavTestScore(con, release = 31:32, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[108]
output["x8.spd2"] <- lavTestScore(con, release = 33:34, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[72]
output["x8.spd3"] <- lavTestScore(con, release = 33:34, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[108]
output["x9.spd2"] <- lavTestScore(con, release = 35:36, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[72]
output["x9.spd3"] <- lavTestScore(con, release = 35:36, univariate = FALSE,
epc = TRUE, warn = FALSE)$epc$epc[108]
output
}
## observed EPC
extra(fit.scalar)
## permutation results, including extra output
set.seed(12345) # same permutations
out.scalar <- permuteMeasEq(nPermute = 20, uncon = fit.metric, con = fit.scalar,
param = "intercepts", AFIs = myAFIs,
moreAFIs = moreAFIs, null = fit.null, extra = extra)
## summarize extra output
summary(out.scalar, extra = TRUE)
###########
## MIMIC ##
###########
## Specify Restricted Factor Analysis (RFA) model, equivalent to MIMIC, but
## the factor covaries with the covariate instead of being regressed on it.
## The covariate defines a single-indicator construct, and the
## double-mean-centered products of the indicators define a latent
## interaction between the factor and the covariate.
mod.mimic <- '
visual =~ x1 + x2 + x3
age =~ ageyr
age.by.vis =~ x1.ageyr + x2.ageyr + x3.ageyr
x1 ~~ x1.ageyr
x2 ~~ x2.ageyr
x3 ~~ x3.ageyr
'
HS.orth <- indProd(var1 = paste0("x", 1:3), var2 = "ageyr", match = FALSE,
data = HS[ , c("ageyr", paste0("x", 1:3))] )
fit.mimic <- cfa(mod.mimic, data = HS.orth, meanstructure = TRUE)
summary(fit.mimic, stand = TRUE)
## Whereas MIMIC models specify direct effects of the covariate on an indicator,
## DIF can be tested in RFA models by specifying free loadings of an indicator
## on the covariate's construct (uniform DIF, scalar invariance) and the
## interaction construct (nonuniform DIF, metric invariance).
param <- as.list(paste0("age + age.by.vis =~ x", 1:3))
names(param) <- paste0("x", 1:3)
# param <- as.list(paste0("x", 1:3, " ~ age + age.by.vis")) # equivalent
## test both parameters simultaneously for each indicator
do.call(rbind, lapply(param, function(x) lavTestScore(fit.mimic, add = x)$test))
## or test each parameter individually
lavTestScore(fit.mimic, add = as.character(param))
####################### Permutation Method
## function to recalculate interaction terms after permuting the covariate
datafun <- function(data) {
d <- data[, c(paste0("x", 1:3), "ageyr")]
indProd(var1 = paste0("x", 1:3), var2 = "ageyr", match = FALSE, data = d)
}
set.seed(12345)
perm.mimic <- permuteMeasEq(nPermute = 20, modelType = "mimic",
con = fit.mimic, param = param,
covariates = "ageyr", datafun = datafun)
summary(perm.mimic)
## End(Not run)