mc.d {mc.heterogeneity}

Standardized Mean Differences (d): Monte Carlo Based Heterogeneity Test for Between-study Heterogeneity in Random- or Mixed- Effects Model

Description

mc.d returns the Monte Carlo based tests of the residual heterogeneity in random- or mixed- effects model of standardized mean differences (d).

Usage

mc.d(
  n1,
  n2,
  est,
  model = "random",
  adjust = FALSE,
  mods = NULL,
  nrep = 10^4,
  p_cut = 0.05,
  mc.include = FALSE
)

Arguments

n1

a vector of sample sizes from group 1 in each of the included studies.

n2

a vector of sample sizes from group 2 in each of the included studies.

est

a vector of unbiased estimates of standardized mean differences.

model

choice of random- or mixed- effects models. Can only be set to "random", or "mixed".

adjust

if biased estimates (i.e., g values) are provided, adjust must be set to TRUE to compensate for small sample bias. By default, adjust is set to FALSE.

mods

optional argument to include one or more moderators in the model. mods is NULL for random-effects model and a dataframe for mixed-effects model. A single moderator can be given as a vector of length k specifying the values of the moderator. Multiple moderators are specified by giving a matrix with k rows and as many columns as there are moderator variables. See rma for more details.

nrep

number of replications used in Monte Carlo Simulations. Default to 10^4.

p_cut

cutoff for p-values, which is the alpha level. Default to 0.05.

mc.include

if true, Monte Carlo simulation results are included in the output (e.g., Monte Carlo critical values).

Details

For standardized mean difference, if the biased estimates (i.e., g values) are provided, adjust=TRUE can be specified to obtain the corresponding unbiased estimates.

This function returns the test statistics as well as their p-value and significances using (1) Q-test, (2) Monte Carlo Based Heterogeneity Test with Maximum Likelihood (ML), and (3) Monte Carlo Based Heterogeneity Test with Restricted Maximum Likelihood (REML).

The results of significances are classified as "sig" or "n.s" based on the cutoff p-value (i.e., alpha level). "sig" means that the between-study heterogeneity is significantly different from zero whereas "n.s" means the between-study heterogeneity is not significantly different from zero. The default alpha level is 0.05.

References

Hedges, L. V. (1981). Distribution theory for glass’s estimator of effect size and related estimators. Journal of Educational and Behavioral Statistics, 6(2), 107–128.

Hedges, L. V., Giaconia, R. M., & Gage, N. L. (1981). Meta-analysis of the effect of open and traditional instruction. Stanford, CA: Stanford University, Program on Teaching Effectiveness.

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1-48. URL: https://www.jstatsoft.org/v36/i03/

Examples

# A meta-analysis of 18 studies in which the effect of open versus traditional
# education on students' self-concept was studied (Hedges et al., 1981).
selfconcept <- mc.heterogeneity:::selfconcept
# n1 and n2 are lists of samples sizes in two groups
n1 <- selfconcept$n1
n2 <- selfconcept$n2
# g is a list of biased estimates of standardized mean differences in the meta-analytical study
g <- selfconcept$g
cm <- (1-3/(4*(n1+n2-2)-1)) #correct factor to compensate for small sample bias (Hedges, 1981)
d <- cm*g
## Not run: 
mc.run <- mc.d(n1, n2, est = d, model = 'random', p_cut = 0.05)
# is equivalent to:
mc.run2 <- mc.d(n1, n2, est = g, model = 'random', adjust = TRUE, p_cut = 0.05)

## End(Not run)

# A hypothetical meta-analysis of 15 studies with 3 moderators.
hypo_moder <- mc.heterogeneity:::hypo_moder
## Not run: 
mc.run3 <- mc.d(n1 = hypo_moder$n1, n2 = hypo_moder$n2, est = hypo_moder$d, model = 'mixed',
mods = cbind(hypo_moder$cov.z1, hypo_moder$cov.z2, hypo_moder$cov.z3), p_cut = 0.05)

## End(Not run)
## Note: this mc.d() function will soon be deprecated
## and replaced by \link[boot.heterogeneity]{boot.d} in
## package [boot.heterogeneity](https://CRAN.R-project.org/package=boot.heterogeneity).