mxModelAverage {OpenMx}  R Documentation 
InformationTheoretic ModelAveraging and Multimodel Inference
Description
omxAkaikeWeights()
orders a list of MxModels (hereinafter, the "candidate set" of models) from best to worst AIC, reports their Akaike weights, and indicates which are in the confidence set for bestapproximating model. mxModelAverage()
calls omxAkaikeWeights()
and includes its output, and also reports modelaverage point estimates and (if requested) their standard errors.
Usage
mxModelAverage(reference=character(0), models=list(),
include=c("onlyFree","all"), SE=NULL, refAsBlock=FALSE, covariances=list(),
type=c("AIC","AICc"), conf.level=0.95)
omxAkaikeWeights(models=list(), type=c("AIC","AICc"), conf.level=0.95)
Arguments
reference 
Vector of character strings referring to parameters, MxMatrices, or MxAlgebras for which modelaverage estimates are to be computed. Defaults to 
models 
The candidate set of models: a list of at least two MxModel objects, each of which must be uniquely identified by the value of its 
include 
Character string, either 
SE 
Logical; should standard errors be reported for the modelaverage point estimates? Defaults to 
refAsBlock 
Logical. If 
covariances 
Optional list of repeatedsampling covariance matrices of free parameter estimates (possibly from bootstrapping or the sandwich estimator); defaults to an empty list. A nonempty list must either be of the same length as 
type 
Character string specifying which information criterion to use: either 
conf.level 
Numeric proportion specifying the desired coverage probability of the confidence set for bestapproximating model among the candidate set (Burnham & Anderson, 2002). Defaults to 0.95. 
Details
If statistical inferences (hypothesis tests and confidence intervals) are the motivation for calculating modelaverage point estimates and their standard errors, then include="onlyFree"
(the default) is recommended. Note that, if models in which a quantity is held fixed are included in calculating the quantity's modelaverage estimate, then that estimate cannot even asymptotically be normally distributed (Bartels, 1997).
If argument covariances
is nonempty, then either it must be of the same length as argument models
, or all of its elements must be named after an MxModel in models
(an MxModel's name is the character string in its name
slot). If covariances
is of the same length as models
but lacks element names, mxModelAverage()
will assume that they are ordered so that the first element of covariances
is to be used with the first MxModel, the second element is to be used with the second MxModel, and so on. Otherwise, mxModelAverage()
assigns the elements of covariances
to the MxModels by matching element names to MxModel names. If covariances
doesn't provide a covariance matrix for a given MxModel–perhaps because it is empty, or only provides matrices for a nonempty proper subset of the candidate set–mxModelAverage()
will fall back to its default behavior of calculating a covariance matrix from the Hessian matrix in the MxModel's output slot. If a covariance matrix cannot be thus calculated and SE=TRUE
, SE
is coerced to FALSE
, with a warning.
The matrices in covariances
must have complete row and column names, equal to the free parameter labels of the corresponding MxModel. These names indicate to which free parameter a given row or column corresponds.
Value
omxAkaikeWeights()
returns a dataframe, with one row for each element of models
. The rows are sorted by their MxModel's AIC (or AICc), from best to worst. The dataframe has five columns:

"model"
: Character string. The name of the MxModel. 
"AIC"
or"AICc"
: Numeric. The MxModel's AIC or AICc. 
"delta"
: Numeric. The MxModel's AIC (or AICc) minus the best (smallest) AIC (or AICc) in the candidate set. 
"AkaikeWeight"
: Numeric. The MxModel's Akaike weight. This column will sum to unity. 
"inConfidenceSet"
: Character. Will contain an asterisk if the MxModel is in the confidence set for bestapproximating model.
The dataframe also has an attribute, "unsortedModelNames"
, which contains the names of the MxModels in the same order as they appear in models
(i.e., without sorting them by their AIC).
If a zerolength value is provided for argument reference
, then mxModelAverage()
returns only the output of omxAkaikeWeights()
, with a warning. Otherwise, for the default values of its arguments, mxModelAverage()
returns a list with four elements:

"ModelAverage Estimates"
: A numeric matrix with one row for each distinct quantity specified byreference
, and as many as two columns. Its rows are named for the corresponding reference quantities. Its first column,"Estimate"
, contains the modelaverage point estimates. If standard errors are being calculated, then its second column,"SE"
, contains the "modelunconditional" standard errors of the modelaverage point estimates. Otherwise, there is no second column. 
"Modelwise Estimates"
: A numeric matrix with one row for each distinct quantity specified byreference
(indicated by row name), and one column for each MxModel (indicated by column name). Each element is an estimate of the given reference quantity, from the given MxModel. Quantities that cannot be evaluated for a given MxModel are reported asNA
. 
"Modelwise Sampling Variances"
: A numeric matrix just like the one in list element 2, except that its elements are the estimated sampling variances of the corresponding modelconditional point estimates in list element 2. Variances for fixed quantities are reported as 0 ifinclude="all"
, and asNA
ifinclude="onlyFree"
; however, if no covariance matrix is available for a model, all of that model's sampling variances will be reported asNA
. 
"AkaikeWeights Table"
: The output fromomxAkaikeWeights()
.
If refAsBlock=TRUE
, list element 3 will instead contain be named "Joint Covariance Matrix"
, and if SE=TRUE
, it will contain the joint sampling covariance matrix for the modelaverage point estimates.
Note
The "bestapproximating model" is defined as the model that truly ("in the population," so to speak) has the smallest KullbackLeibler divergence from full reality, among the models in the candidate set (Burnham & Anderson, 2002).
A model's Akaike weight is interpretable as the relative weightofevidence for that model being the bestapproximating model, given the observed data and the candidate set. It has a Bayesian interpretation as the posterior probability that the given model is the bestapproximating model in the candidate set, assuming a "savvy" prior probability that depends upon sample size and the number of free parameters in the model (Burnham & Anderson, 2002).
The confidence set for bestapproximating model serves to reflect sampling error in the AICs. When fitting the candidate set to data over repeated sampling, the confidence set is expected to contain the bestapproximating model with probability equal to its confidence level.
The sampling variances and covariances of the modelaverage point estimates are calculated from Equations (4) and (5) in Burnham & Anderson (2004). The standard errors reported by mxModelAverage()
are the square roots of those sampling variances.
For an example of modelaveraging and multimodel inference applied to structural equation modeling using OpenMx v1.3 (i.e., well before the functions documented here were implemented), see Kirkpatrick, McGue, & Iacono (2015).
References
Bartels, L. M. (1997). Specification uncertainty and model averaging. American Journal of Political Science, 41(2), 641674.
Burnham, K. P., & Anderson, D. R. (2002). Model Selection and Multimodel Inference: A Practical InformationTheoretic Approach (2nd ed.). New York: Springer.
Burnham, K. P., & Anderson, D. R. (2004). Multimodel inference: Understanding AIC and BIC in model selection. Sociological Methods & Research, 33(2), 261304. doi:10.1177/0049124104268644
Hurvich, C. M., & Tsai, CL. (1989). Regression and time series model selection in small samples. Biometrika, 76(2), 297307.
Kirkpatrick, R. M., McGue, M., & Iacono, W. G. (2015). Replication of a geneenvironment interaction via multimodel inference: Additivegenetic variance in adolescents' general cognitive ability increases with familyoforigin socioeconomic status. Behavior Genetics, 45, 200214.
See Also
Examples
require(OpenMx)
data(demoOneFactor)
factorModel1 < mxModel(
"OneFactor1",
mxMatrix(
"Full", 5, 1, values=0.8,
labels=paste("a",1:5,sep=""),
free=TRUE, name="A"),
mxMatrix(
"Full", 5, 1, values=1,
labels=paste("u",1:5,sep=""),
free=TRUE, name="Udiag"),
mxMatrix(
"Symm", 1, 1, values=1,
free=FALSE, name="L"),
mxAlgebra(vec2diag(Udiag),name="U"),
mxAlgebra(A %*% L %*% t(A) + U, name="R"),
mxExpectationNormal(
covariance = "R",
dimnames = names(demoOneFactor)),
mxFitFunctionML(),
mxData(cov(demoOneFactor), type="cov", numObs=500))
factorFit1 < mxRun(factorModel1)
#Constrain unique variances equal:
factorModel2 < omxSetParameters(
model=factorModel1,labels=paste("u",1:5,sep=""),
newlabels="u",name="OneFactor2")
factorFit2 < mxRun(factorModel2)
omxAkaikeWeights(models=list(factorFit1,factorFit2))
mxModelAverage(
reference=c("A","Udiag"), include="all",
models=list(factorFit1,factorFit2))