EMMEANS {bruceR}R Documentation

Simple-effect analysis and post-hoc multiple comparison.

Description

Perform (1) simple-effect (and simple-simple-effect) analyses, including both simple main effects and simple interaction effects, and (2) post-hoc multiple comparisons (e.g., pairwise, sequential, polynomial), with p values adjusted for factors with >= 3 levels.

This function is based on and extends (1) emmeans::joint_tests(), (2) emmeans::emmeans(), and (3) emmeans::contrast(). You only need to specify the model object, to-be-tested effect(s), and moderator(s). Almost all results you need will be displayed together, including effect sizes (partial η^2 and Cohen's d) and their confidence intervals (CIs). 90% CIs for partial η^2 and 95% CIs for Cohen's d are reported.

By default, the root mean square error (RMSE) is used to compute the pooled SD for Cohen's d. Specifically, it uses:

  1. the square root of mean square error (MSE) for between-subjects designs;

  2. the square root of mean variance of all paired differences of the residuals of repeated measures for within-subjects and mixed designs.

In both situations, it extracts the lm object from the returned value of MANOVA(). Then, it mainly uses sigma() and residuals(), respectively, to do these estimates. For source code, please see the file bruceR_stats_03_manova.R on the GitHub Repository.

Disclaimer: There is substantial disagreement on the appropriate pooled SD to use in computing the effect size. For alternative methods, see emmeans::eff_size() and effectsize::t_to_d(). Users should not take the default output as the only right results and are completely responsible for specifying sd.pooled.

Usage

EMMEANS(
  model,
  effect = NULL,
  by = NULL,
  contrast = "pairwise",
  reverse = TRUE,
  p.adjust = "bonferroni",
  sd.pooled = NULL,
  model.type = "multivariate",
  digits = 2,
  nsmall = digits
)

Arguments

model

The model object returned by MANOVA.

effect

Effect(s) you want to test. If set to a character string (e.g., "A"), it reports the results of omnibus test or simple main effect. If set to a character vector (e.g., c("A", "B")), it also reports the results of simple interaction effect.

by

Moderator variable(s). Default is NULL.

contrast

Contrast method for multiple comparisons. Default is "pairwise".

Alternatives can be "pairwise" ("revpairwise"), "seq" ("consec"), "poly", "eff". For details, see ?emmeans::`contrast-methods`.

reverse

The order of levels to be contrasted. Default is TRUE (higher level vs. lower level).

p.adjust

Adjustment method of p values for multiple comparisons. Default is "bonferroni". For polynomial contrasts, default is "none".

Alternatives can be "none", "fdr", "hochberg", "hommel", "holm", "tukey", "mvt", "dunnettx", "sidak", "scheffe", "bonferroni". For details, see stats::p.adjust() and emmeans::summary().

sd.pooled

By default, it uses sqrt(MSE) (root mean square error, RMSE) as the pooled SD to compute Cohen's d. Users may specify this argument as the SD of a reference group, or use effectsize::sd_pooled() to obtain a pooled SD. For an issue about the computation method of Cohen's d, see Disclaimer above.

model.type

"multivariate" returns the results of pairwise comparisons identical to SPSS, which uses the lm (rather than aov) object of the model for emmeans::joint_tests() and emmeans::emmeans().

"univariate" requires also specifying aov.include=TRUE in MANOVA (not recommended by the afex package; for details, see afex::aov_ez()).

digits, nsmall

Number of decimal places of output. Default is 2.

Value

The same model object as returned by MANOVA (for recursive use), along with a list of EMMEANS tables: sim (simple effects), emm (estimated marginal means), con (contrasts). Each EMMEANS appends one list to the returned object.

Statistical Details

Some may confuse the statistical terms "simple effects", "post-hoc tests", and "multiple comparisons". Such a confusion is not uncommon. Here I explain what these terms actually refer to.

1. Simple Effect

When we speak of "simple effect", we are referring to ...

  • simple main effect

  • simple interaction effect (only for designs with 3 or more factors)

  • simple simple effect (only for designs with 3 or more factors)

When the interaction effect in ANOVA is significant, we should then perform a "simple-effect analysis". In regression, we call this "simple-slope analysis". They are identical in statistical principles.

In a two-factors design, we only test "simple main effect". That is, at different levels of a factor "B", the main effects of "A" would be different. However, in a three-factors (or more) design, we may also test "simple interaction effect" and "simple simple effect". That is, at different combinations of levels of factors "B" and "C", the main effects of "A" would be different.

To note, simple effects per se never require p-value adjustment, because what we test in simple-effect analyses are still "omnibus F-tests".

2. Post-Hoc Test

The term "post-hoc" means that the tests are performed after ANOVA. Given this, some may (wrongly) regard simple-effect analyses also as a kind of post-hoc tests. However, these two terms should be distinguished. In many situations, "post-hoc tests" only refer to "post-hoc comparisons" using t-tests and some p-value adjustment techniques. We need post-hoc comparisons only when there are factors with 3 or more levels.

Post-hoc tests are totally independent of whether there is a significant interaction effect. It only deals with factors with multiple levels. In most cases, we use pairwise comparisons to do post-hoc tests. See the next part for details.

3. Multiple Comparison

As mentioned above, multiple comparisons are indeed post-hoc tests but have no relationship with simple-effect analyses. Post-hoc multiple comparisons are independent of interaction effects and simple effects. Furthermore, if a simple main effect contains 3 or more levels, we also need to do multiple comparisons within the simple-effect analysis. In this situation, we also need p-value adjustment with methods such as Bonferroni, Tukey's HSD (honest significant difference), FDR (false discovery rate), and so forth.

Options for multiple comparison:

  • "pairwise" - Pairwise comparisons (default is "higher level - lower level")

  • "seq" or "consec" - Consecutive (sequential) comparisons

  • "poly" - Polynomial contrasts (linear, quadratic, cubic, quartic, ...)

  • "eff" - Effect contrasts (vs. the grand mean)

See Also

TTEST, MANOVA, bruceR-demodata

Examples

#### Between-Subjects Design ####

between.1
MANOVA(between.1, dv="SCORE", between="A") %>%
  EMMEANS("A")
MANOVA(between.1, dv="SCORE", between="A") %>%
  EMMEANS("A", p.adjust="tukey")
MANOVA(between.1, dv="SCORE", between="A") %>%
  EMMEANS("A", contrast="seq")
MANOVA(between.1, dv="SCORE", between="A") %>%
  EMMEANS("A", contrast="poly")

between.2
MANOVA(between.2, dv="SCORE", between=c("A", "B")) %>%
  EMMEANS("A") %>%
  EMMEANS("A", by="B") %>%
  EMMEANS("B") %>%
  EMMEANS("B", by="A")

between.3
MANOVA(between.3, dv="SCORE", between=c("A", "B", "C")) %>%
  EMMEANS("A", by="B") %>%
  EMMEANS(c("A", "B"), by="C") %>%
  EMMEANS("A", by=c("B", "C"))
## just to name a few here
## you may test other combinations


#### Within-Subjects Design ####

within.1
MANOVA(within.1, dvs="A1:A4", dvs.pattern="A(.)",
       within="A") %>%
  EMMEANS("A")

within.2
MANOVA(within.2, dvs="A1B1:A2B3", dvs.pattern="A(.)B(.)",
       within=c("A", "B")) %>%
  EMMEANS("A", by="B") %>%
  EMMEANS("B", by="A")  # singular error matrix
# :::::::::::::::::::::::::::::::::::::::
# This would produce a WARNING because of
# the linear dependence of A2B2 and A2B3.
# see: Corr(within.2[c("A2B2", "A2B3")])

within.3
MANOVA(within.3, dvs="A1B1C1:A2B2C2", dvs.pattern="A(.)B(.)C(.)",
       within=c("A", "B", "C")) %>%
  EMMEANS("A", by="B") %>%
  EMMEANS(c("A", "B"), by="C") %>%
  EMMEANS("A", by=c("B", "C"))


#### Mixed Design ####

mixed.2_1b1w
MANOVA(mixed.2_1b1w, dvs="B1:B3", dvs.pattern="B(.)",
       between="A", within="B", sph.correction="GG") %>%
  EMMEANS("A", by="B") %>%
  EMMEANS("B", by="A")

mixed.3_1b2w
MANOVA(mixed.3_1b2w, dvs="B1C1:B2C2", dvs.pattern="B(.)C(.)",
       between="A", within=c("B", "C")) %>%
  EMMEANS("A", by="B") %>%
  EMMEANS(c("A", "B"), by="C") %>%
  EMMEANS("A", by=c("B", "C"))

mixed.3_2b1w
MANOVA(mixed.3_2b1w, dvs="B1:B2", dvs.pattern="B(.)",
       between=c("A", "C"), within="B") %>%
  EMMEANS("A", by="B") %>%
  EMMEANS("A", by="C") %>%
  EMMEANS(c("A", "B"), by="C") %>%
  EMMEANS("B", by=c("A", "C"))


#### Other Examples ####
air=airquality
air$Day.1or2=ifelse(air$Day %% 2 == 1, 1, 2) %>%
  factor(levels=1:2, labels=c("odd", "even"))
MANOVA(air, dv="Temp", between=c("Month", "Day.1or2"),
       covariate=c("Solar.R", "Wind")) %>%
  EMMEANS("Month", contrast="seq") %>%
  EMMEANS("Month", by="Day.1or2", contrast="poly")


[Package bruceR version 0.8.0 Index]