EMMEANS {bruceR}  R Documentation 
Simpleeffect analysis and posthoc multiple comparison.
Description
Perform (1) simpleeffect (and simplesimpleeffect) analyses, including both simple main effects and simple interaction effects, and (2) posthoc multiple comparisons (e.g., pairwise, sequential, polynomial), with p values adjusted for factors with >= 3 levels.
This function is based on and extends
(1) emmeans::joint_tests()
,
(2) emmeans::emmeans()
, and
(3) emmeans::contrast()
.
You only need to specify the model object, tobetested effect(s), and moderator(s).
Almost all results you need will be displayed together,
including effect sizes (partial \eta^2
and Cohen's d) and their confidence intervals (CIs).
90% CIs for partial \eta^2
and 95% CIs for Cohen's d are reported.
By default, the root mean square error (RMSE) is used to compute the pooled SD for Cohen's d. Specifically, it uses:
the square root of mean square error (MSE) for betweensubjects designs;
the square root of mean variance of all paired differences of the residuals of repeated measures for withinsubjects and mixed designs.
Disclaimer:
There is substantial disagreement on the appropriate pooled SD to use in computing the effect size.
For alternative methods, see emmeans::eff_size()
and effectsize::t_to_d()
.
Users should not take the default output as the only right results and are completely responsible for specifying sd.pooled
.
Usage
EMMEANS(
model,
effect = NULL,
by = NULL,
contrast = "pairwise",
reverse = TRUE,
p.adjust = "bonferroni",
sd.pooled = NULL,
model.type = "multivariate",
digits = 3
)
Arguments
model 
The model object returned by 
effect 
Effect(s) you want to test.
If set to a character string (e.g., 
by 
Moderator variable(s). Defaults to 
contrast 
Contrast method for multiple comparisons.
Defaults to Alternatives can be 
reverse 
The order of levels to be contrasted.
Defaults to 
p.adjust 
Adjustment method of p values for multiple comparisons.
Defaults to Alternatives can be 
sd.pooled 
By default, it uses 
model.type 

digits 
Number of decimal places of output. Defaults to 
Value
The same model object as returned by
MANOVA
(for recursive use),
along with a list of tables:
sim
(simple effects),
emm
(estimated marginal means),
con
(contrasts).
Each EMMEANS(...)
appends one list to the returned object.
Interaction Plot (See Examples Below)
You can save the returned object and use the emmeans::emmip()
function
to create an interaction plot (based on the fitted model and a formula).
See examples below for the usage.
Note: emmeans::emmip()
returns a ggplot
object,
which can be modified and saved with ggplot2
syntax.
Statistical Details
Some may confuse the statistical terms "simple effects", "posthoc tests", and "multiple comparisons". Such a confusion is not uncommon. Here I explain what these terms actually refer to.
 1. Simple Effect

When we speak of "simple effect", we are referring to ...
simple main effect
simple interaction effect (only for designs with 3 or more factors)
simple simple effect (only for designs with 3 or more factors)
When the interaction effect in ANOVA is significant, we should then perform a "simpleeffect analysis". In regression, we call this "simpleslope analysis". They are identical in statistical principles.
In a twofactors design, we only test "simple main effect". That is, at different levels of a factor "B", the main effects of "A" would be different. However, in a threefactors (or more) design, we may also test "simple interaction effect" and "simple simple effect". That is, at different combinations of levels of factors "B" and "C", the main effects of "A" would be different.
To note, simple effects per se never require pvalue adjustment, because what we test in simpleeffect analyses are still "omnibus Ftests".
 2. PostHoc Test

The term "posthoc" means that the tests are performed after ANOVA. Given this, some may (wrongly) regard simpleeffect analyses also as a kind of posthoc tests. However, these two terms should be distinguished. In many situations, "posthoc tests" only refer to "posthoc comparisons" using ttests and some pvalue adjustment techniques. We need posthoc comparisons only when there are factors with 3 or more levels.
Posthoc tests are totally independent of whether there is a significant interaction effect. It only deals with factors with multiple levels. In most cases, we use pairwise comparisons to do posthoc tests. See the next part for details.
 3. Multiple Comparison

As mentioned above, multiple comparisons are indeed posthoc tests but have no relationship with simpleeffect analyses. Posthoc multiple comparisons are independent of interaction effects and simple effects. Furthermore, if a simple main effect contains 3 or more levels, we also need to do multiple comparisons within the simpleeffect analysis. In this situation, we also need pvalue adjustment with methods such as Bonferroni, Tukey's HSD (honest significant difference), FDR (false discovery rate), and so forth.
Options for multiple comparison:

"pairwise"
 Pairwise comparisons (default is "higher level  lower level") 
"seq"
or"consec"
 Consecutive (sequential) comparisons 
"poly"
 Polynomial contrasts (linear, quadratic, cubic, quartic, ...) 
"eff"
 Effect contrasts (vs. the grand mean)

See Also
TTEST
, MANOVA
, bruceRdemodata
Examples
#### BetweenSubjects Design ####
between.1
MANOVA(between.1, dv="SCORE", between="A") %>%
EMMEANS("A")
MANOVA(between.1, dv="SCORE", between="A") %>%
EMMEANS("A", p.adjust="tukey")
MANOVA(between.1, dv="SCORE", between="A") %>%
EMMEANS("A", contrast="seq")
MANOVA(between.1, dv="SCORE", between="A") %>%
EMMEANS("A", contrast="poly")
between.2
MANOVA(between.2, dv="SCORE", between=c("A", "B")) %>%
EMMEANS("A", by="B") %>%
EMMEANS("B", by="A")
## How to create an interaction plot using `emmeans::emmip()`?
## See help page: ?emmeans::emmip()
m = MANOVA(between.2, dv="SCORE", between=c("A", "B"))
emmip(m, ~ A  B, CIs=TRUE)
emmip(m, ~ B  A, CIs=TRUE)
emmip(m, B ~ A, CIs=TRUE)
emmip(m, A ~ B, CIs=TRUE)
between.3
MANOVA(between.3, dv="SCORE", between=c("A", "B", "C")) %>%
EMMEANS("A", by="B") %>%
EMMEANS(c("A", "B"), by="C") %>%
EMMEANS("A", by=c("B", "C"))
## Just to name a few...
## You may test other combinations...
#### WithinSubjects Design ####
within.1
MANOVA(within.1, dvs="A1:A4", dvs.pattern="A(.)",
within="A") %>%
EMMEANS("A")
within.2
MANOVA(within.2, dvs="A1B1:A2B3", dvs.pattern="A(.)B(.)",
within=c("A", "B")) %>%
EMMEANS("A", by="B") %>%
EMMEANS("B", by="A") # singular error matrix
# :::::::::::::::::::::::::::::::::::::::
# This would produce a WARNING because of
# the linear dependence of A2B2 and A2B3.
# See: Corr(within.2[c("A2B2", "A2B3")])
within.3
MANOVA(within.3, dvs="A1B1C1:A2B2C2", dvs.pattern="A(.)B(.)C(.)",
within=c("A", "B", "C")) %>%
EMMEANS("A", by="B") %>%
EMMEANS(c("A", "B"), by="C") %>%
EMMEANS("A", by=c("B", "C"))
## Just to name a few...
## You may test other combinations...
#### Mixed Design ####
mixed.2_1b1w
MANOVA(mixed.2_1b1w, dvs="B1:B3", dvs.pattern="B(.)",
between="A", within="B", sph.correction="GG") %>%
EMMEANS("A", by="B") %>%
EMMEANS("B", by="A")
mixed.3_1b2w
MANOVA(mixed.3_1b2w, dvs="B1C1:B2C2", dvs.pattern="B(.)C(.)",
between="A", within=c("B", "C")) %>%
EMMEANS("A", by="B") %>%
EMMEANS(c("A", "B"), by="C") %>%
EMMEANS("A", by=c("B", "C"))
## Just to name a few...
## You may test other combinations...
mixed.3_2b1w
MANOVA(mixed.3_2b1w, dvs="B1:B2", dvs.pattern="B(.)",
between=c("A", "C"), within="B") %>%
EMMEANS("A", by="B") %>%
EMMEANS("A", by="C") %>%
EMMEANS(c("A", "B"), by="C") %>%
EMMEANS("B", by=c("A", "C"))
## Just to name a few...
## You may test other combinations...
#### Other Examples ####
air = airquality
air$Day.1or2 = ifelse(air$Day %% 2 == 1, 1, 2) %>%
factor(levels=1:2, labels=c("odd", "even"))
MANOVA(air, dv="Temp", between=c("Month", "Day.1or2"),
covariate=c("Solar.R", "Wind")) %>%
EMMEANS("Month", contrast="seq") %>%
EMMEANS("Month", by="Day.1or2", contrast="poly")