boot.d {boot.heterogeneity}

Standardized Mean Differences (d): Bootstrap-based Heterogeneity Test for Between-study Heterogeneity in Random- or Mixed- Effects Model

Description

boot.d returns the bootstrap-based tests of the residual heterogeneity in random- or mixed- effects model of standardized mean differences (d).

Usage

boot.d(
  n1,
  n2,
  est,
  lambda = 0,
  model = "random",
  adjust = FALSE,
  mods = NULL,
  nrep = 10^4,
  p_cut = 0.05,
  boot.include = FALSE,
  parallel = FALSE,
  cores = 4,
  verbose = FALSE
)

Arguments

n1

A vector of sample sizes from group 1 in each of the included studies.

n2

A vector of sample sizes from group 2 in each of the included studies.

est

A vector of unbiased estimates of standardized mean differences.

lambda

Size of the magnitude to be tested in the alternative hypothesis of the heterogeneity magnitude test. Default to 0.

model

Choice of random- or mixed- effects models. Can only be set to "random", or "mixed".

adjust

If biased estimates (i.e., g values) are provided, adjust must be set to TRUE to compensate for small sample bias. By default, adjust is set to FALSE.

mods

Optional argument to include moderators in the model. mods is NULL for random-effects model and a dataframe of moderators for mixed-effects model. A single moderator can be given as a vector specifying the values of the moderator. Multiple moderators are specified by giving a matrix with as many columns as there are moderator variables. See rma for more details.

nrep

Number of replications used in bootstrap simulations. Default to 10^4.

p_cut

Cutoff for p-values, which is the alpha level. Default to 0.05.

boot.include

If true, bootstrap simulation results are included in the output (e.g., bootstrap critical values).

parallel

If true, parallel computing using 4 cores will be performed during bootstrapping stage. Otherwise, for loop is used.

cores

The number of cores used in the parallel computing. Default to 4.

verbose

If true, show the progress of bootstrapping.

Details

For standardized mean difference, if the biased estimates (i.e., g values) are provided, adjust=TRUE can be specified to obtain the corresponding unbiased estimates.

This function returns the test statistics as well as their p-value and significances using (1) Q-test and (2) Bootstrap-based Heterogeneity Test with Restricted Maximum Likelihood (REML).

The results of significances are classified as "sig" or "n.s" based on the cutoff p-value (i.e., alpha level). "sig" means that the between-study heterogeneity is significantly different from zero whereas "n.s" means the between-study heterogeneity is not significantly different from zero. The default alpha level is 0.05.

Value

A dataframe that contains the test statistics ('stat'), p-values ('p_value'), and significances of effect size heterogeneity ("Heterogeneity").

References

Hedges, L. V. (1981). Distribution theory for glass’s estimator of effect size and related estimators. Journal of Educational and Behavioral Statistics, 6(2), 107–128.

Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press.

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1-48. URL: http://www.jstatsoft.org/v36/i03/

Examples

# Demo 1: A meta-analysis of 18 studies in which the effect of open versus
# traditional education on students' self-concept was studied (Hedges & Olkin, 1985).

selfconcept <- boot.heterogeneity:::selfconcept

# n1 and n2 are lists of samples sizes in two groups
n1 <- selfconcept$n1
n2 <- selfconcept$n2

# g is a list of biased estimates of standardized mean differences in the meta-analytical study
g <- selfconcept$g
cm <- (1-3/(4*(n1+n2-2)-1)) #correct factor to compensate for small sample bias (Hedges, 1981)
d <- cm*g

## Not run: 
boot.run <- boot.d(n1, n2, est = d, model = 'random', p_cut = 0.05)
# is equivalent to:
boot.run2 <- boot.d(n1, n2, est = g, model = 'random', adjust = TRUE, p_cut = 0.05)

## End(Not run)

# Demo 2: A hypothetical meta-analysis of 15 studies with 3 moderators.
hypo_moder <- boot.heterogeneity:::hypo_moder
## Not run: 
boot.run3 <- boot.d(n1 = hypo_moder$n1, n2 = hypo_moder$n2, est = hypo_moder$d, model = 'mixed',
mods = cbind(hypo_moder$cov.z1, hypo_moder$cov.z2, hypo_moder$cov.z3), p_cut = 0.05)

## End(Not run)