In the particular data set we are working with here, the only higher
order effect that was significant was the Wordtype by Grade interaction.
This means that we would typically only follow up that effect by looking
at simple main effects of one of those factors at levels of the other,
as well as their interaction contrasts. We would also follow up with
contrasts on the Feedback factor since its main effect was significant
and no interactions with Feedback were significant.
However, the approach taken here is to demonstrate how the whole
suite of effects can be obtained. Initially this document uses the
phia package for these evaluations and makes extensive
use of the testInteractions function from that package. The
methods by “holm”, and “Hochberg” are possible for p value adjustments
with contrasts.
Simple two way
interactions
Simple two way interactions can also be obtained (three sets). We
don’t expect to be interested in any of these since the three way
interaction was not significant, but we might have had an
a-priori hypothesis about one or more of them, so illustration
is included for completeness.
Each of the rows of the tables represent tests of the simple two-way
at that level of the factor at which the effects are examined
(e.g.,feedback by grade at levels of grade in the first
illustration).
The general strategy in the testInteractions function is
to use an argument called “fixed” to specify the variable at which
effects of other factors are to be examined in simple effects. The
“across” argument specifies a variable for which the effect is being
requested.
Note that the combined effects of two variables are requested by
putting both of them in the “across” argument - the variable “AT” which
these effects are examined is designated by the “fixed” argument.
Reading the table requires some understanding of the idiosyncracies
of testInteractions formatting. The columns labeled (for
example) feedback1:wordtype1 are not interactions as the colon implies.
The numeric appendage (e.g., feedback1) means a contrast. The numeric
values (e.g., 6, 3.2, etc) are “estimates” of the underlying contrast.
But since we are not examining contrasts here we can ignore these
columns. The F tests are tests of the two different simple two way
interactions at the two levels of grade. At least the question of level
of grade is clear in the table - rows.
# next, feedback*wordtype at levels of grade
testInteractions(fit.1aov,fixed=c("grade"), across=c("feedback", "wordtype"),adjust="none")
| fifth |
6 |
3.2 |
-3.2 |
-2.8 |
4 |
23.333333 |
3.0882353 |
0.0209781 |
| twelfth |
3 |
0.2 |
-0.2 |
0.2 |
4 |
1.333333 |
0.1764706 |
0.9498252 |
| Residuals |
NA |
NA |
NA |
NA |
72 |
136.000000 |
NA |
NA |
A second possible set of simple two way interactions is found here,
wordtype by grade at levels of feedback.
# first wordtype*grade at levels of feedback
testInteractions(fit.1aov,fixed=c("feedback"), across=c("wordtype","grade"),adjust="none")
| None |
-0.2 |
0.2 |
2 |
0.0666667 |
0.0176471 |
0.9825120 |
| Positive |
2.8 |
-2.8 |
2 |
13.0666667 |
3.4588235 |
0.0367876 |
| Negative |
2.8 |
-2.8 |
2 |
13.0666667 |
3.4588235 |
0.0367876 |
| Residuals |
NA |
NA |
72 |
136.0000000 |
NA |
NA |
And now the third set:
# next, feedback*grade at levels of wordtype
testInteractions(fit.1aov,fixed=c("wordtype"), across=c("feedback","grade"),adjust="none")
| LF_LE |
-0.6 |
-0.2 |
2 |
0.2 |
0.0529412 |
0.9484727 |
| HF_LE |
-0.6 |
-0.2 |
2 |
0.2 |
0.0529412 |
0.9484727 |
| HF_HE |
-3.6 |
-3.2 |
2 |
18.2 |
4.8176471 |
0.0108733 |
| Residuals |
NA |
NA |
72 |
136.0 |
NA |
NA |
The analyst would likely never do all three sets. One one should
suffice and it would probably be the one where the two primary IVs were
interacted. In our example, the first one fits the way the ggplot bar
graph is drawn so that is probably what the choice would be.
Simple-simple main
effects
These are effects of one factor at combined levels of two other
factors.
First examine the so-called simple, simple main effects (three
different sets). These effects would be assessed after finding
significant simple two way interactions. Notice that the
testInteractions function accepts an argument for p value
adjustment (e.g., bonferroni/holm/fdr), but that I’ve set it to “none”
in these illustrations.
The general strategy in the testInteractions function is
to use an argument called “fixed” to specify the variable at which
effects of other factors are to be examined in simple effects. The
“across” argument specifies a variable for which the effect is being
requested.
The first set here examines effects of wordtype at combinations of
the other two factor levels and is one of the sets of simple simple main
effects most visible from the ggplot bar graph above.
Here, and for later tables, “values” of effects are differentiated by
the contrasts associated with the factors although those are aggregated
to obtain the simple effects - seen since many of them have multiple df.
We are ignoring those “effect” values here since we have not addressed
the contrasts in place at the time the aov fit was done
(default is dummy coding). The exact choice of contrast set doesn’t
influence the 2 df simple simple main effects here.
Notice that each of these have 2 df (three means compared, so 2 df).
Later we will break the effects down into contrasts. Compare the
significance test results to the ggplot bar graph. The only significant
simple simple main effects in this table are the wordtype comparisons in
the upper panel (fifth grade) for the negative and positive feedback
conditions. This pattern of significance fits what the eye sees in the
graph.
# 1, effects of wordtype @ combinations of feedback and grade
testInteractions(fit.1aov,fixed=c("feedback","grade"), across="wordtype",adjust="none")
| None : fifth |
0.0 |
-0.8 |
2 |
1.600000 |
0.4235294 |
0.6563524 |
| Positive : fifth |
3.2 |
-3.6 |
2 |
40.933333 |
10.8352941 |
0.0000770 |
| Negative : fifth |
4.6 |
-3.8 |
2 |
53.733333 |
14.2235294 |
0.0000062 |
| None : twelfth |
0.2 |
-1.0 |
2 |
2.533333 |
0.6705882 |
0.5145724 |
| Positive : twelfth |
0.4 |
-0.8 |
2 |
1.733333 |
0.4588235 |
0.6338618 |
| Negative : twelfth |
1.8 |
-1.0 |
2 |
5.200000 |
1.3764706 |
0.2590309 |
| Residuals |
NA |
NA |
72 |
136.000000 |
NA |
NA |
We could also look at other sets of simple simple main effects but
would probably only look at one (such as the immediately preceding one),
followed up by contrasts (later in this doc).
# 2, effects of grade @ combinations of feedback and wordtype
testInteractions(fit.1aov,fixed=c("feedback","wordtype"), across="grade",adjust="none")
| None : LF_LE |
0.0 |
1 |
0.0 |
0.0000000 |
1.0000000 |
| Positive : LF_LE |
-0.2 |
1 |
0.1 |
0.0529412 |
0.8186747 |
| Negative : LF_LE |
-0.4 |
1 |
0.4 |
0.2117647 |
0.6467745 |
| None : HF_LE |
0.0 |
1 |
0.0 |
0.0000000 |
1.0000000 |
| Positive : HF_LE |
-0.2 |
1 |
0.1 |
0.0529412 |
0.8186747 |
| Negative : HF_LE |
-0.4 |
1 |
0.4 |
0.2117647 |
0.6467745 |
| None : HF_HE |
0.2 |
1 |
0.1 |
0.0529412 |
0.8186747 |
| Positive : HF_HE |
-3.0 |
1 |
22.5 |
11.9117647 |
0.0009371 |
| Negative : HF_HE |
-3.2 |
1 |
25.6 |
13.5529412 |
0.0004450 |
| Residuals |
NA |
72 |
136.0 |
NA |
NA |
And the third possible set of simple simple main effects:
# 3, effects of feedback @ combinations of grade and wordtype
testInteractions(fit.1aov,fixed=c("grade","wordtype"), across="feedback",adjust="none")
| fifth : LF_LE |
-0.2 |
-0.6 |
2 |
0.9333333 |
0.2470588 |
0.7817542 |
| twelfth : LF_LE |
0.4 |
-0.4 |
2 |
0.5333333 |
0.1411765 |
0.8685758 |
| fifth : HF_LE |
-1.6 |
-0.8 |
2 |
3.7333333 |
0.9882353 |
0.3772246 |
| twelfth : HF_LE |
-1.0 |
-0.6 |
2 |
1.7333333 |
0.4588235 |
0.6338618 |
| fifth : HF_HE |
-4.8 |
-3.6 |
2 |
51.6000000 |
13.6588235 |
0.0000094 |
| twelfth : HF_HE |
-1.2 |
-0.4 |
2 |
1.6000000 |
0.4235294 |
0.6563524 |
| Residuals |
NA |
NA |
72 |
136.0000000 |
NA |
NA |
Simple main
effects
In studies where the 3way interaction is not significant, the analyst
would progress to examining the omnibus 2way interactions. If any are
significant then they would partly be characterized by examining simple
main effects (there are six possible sets here, two each for each of the
three possible 2way interactions).
It is not possible to interpret such simple main effects without
reference to the tables of collapsed/marginal means such as those
produce above following the initial aov fit OR be examining
the redrawn graph that depicts 2way layouts, collapsed on one of the
three factors.
Such a redrawn graph is presented here, to depict the two way layout
of wordtype and feedback (a 3x3 structure), since the first of the six
sets of SME shown here examines the effect of wordtype at levels of
feedback. If this were actual data analysis of a true data set (rather
than a textbook/artifical one), then we would only look to follow up the
wordtype by grade two way interaction since it was the only one of the
three that reached traditional significance levels. But this first
graphical illustration permits seeing the more complex 3x3 arrangement,
even though the wordtype by feedback interaction was NS. The reader
should recognize that the effects of wordtype (seen in the plot below)
appear to depend on level of feedback.
From this depiction, we might expect to find an effect of wordtype
only in the negative and positive feedback conditions. In reality the
2way interaction tests whether the wordtype effect is different in the
three levels of feedback. It was the general impression that the HF_HE
group is lower only in the positive and negative feedback levels.
Nontheless while this impression is clear, the differnece was not
significant with the wordtype by feedback interaction was tested.
# first, summarize the data set to produce a new data frame that ggplot can use
myData2.wg <- Rmisc::summarySE(data=bg.3way,measurevar="numrecall", groupvars=c("feedback", "wordtype"))
# look at the new data frame that contains the summary statistics
myData2.wg
| None |
LF_LE |
10 |
8.4 |
1.173788 |
0.3711843 |
0.8396772 |
| None |
HF_LE |
10 |
8.8 |
1.316561 |
0.4163332 |
0.9418111 |
| None |
HF_HE |
10 |
7.9 |
1.286684 |
0.4068852 |
0.9204382 |
| Positive |
LF_LE |
10 |
7.9 |
1.370320 |
0.4333333 |
0.9802681 |
| Positive |
HF_LE |
10 |
8.1 |
1.370320 |
0.4333333 |
0.9802681 |
| Positive |
HF_HE |
10 |
5.9 |
2.024846 |
0.6403124 |
1.4484873 |
| Negative |
LF_LE |
10 |
8.2 |
1.316561 |
0.4163332 |
0.9418111 |
| Negative |
HF_LE |
10 |
7.8 |
1.229273 |
0.3887301 |
0.8793686 |
| Negative |
HF_HE |
10 |
5.4 |
2.170509 |
0.6863753 |
1.5526889 |
#library(ggplot2)
#library(ggthemes)
# Now create the Default bar plot
p3 <- ggplot(myData2.wg, aes(x=feedback, y=numrecall, fill=wordtype)) +
geom_bar(stat="identity", color="black",
position=position_dodge()) +
geom_errorbar(aes(ymin=numrecall-se, ymax=numrecall+se), width=.2,
position=position_dodge(.9))
p4 <- p3 +labs(title="Words Recalled +/- SEM", x="Feedback", y = "Mean Number Recalled")+
theme_bw() +
theme(panel.grid.major.x = element_blank(),
panel.grid.major.y = element_blank(),
panel.grid.minor.x = element_blank(),
panel.grid.minor.y = element_blank(),
panel.background = element_blank(),
axis.line.y = element_line(colour="black", size=.7),
axis.line.x = element_line(colour="black", size=.7),
plot.title = element_text(hjust=.5)
) +
scale_fill_manual(values=c('honeydew3','honeydew2', 'honeydew1'))
p4
The following sequence of tests of six sets of simple main effects is
presented only to demonstrate how to generate each of them. We would
probably only examine number 3 or 4.
In the first analysis we find that in the both the negative feedback
condition the effect of feedback significant, as the graph seems to
suggest. But note that most of the difference is due to the position of
the HF_HE group. We would pursue that pattern with examination of simple
main effect contrasts.
# 1-2 effects of wordtype at levels of feedback, and then feedback at levels of wordtype
# note these are collapsed on grade
testInteractions(fit.1aov,fixed="feedback", across="wordtype",adjust="none")
| None |
0.1 |
-0.9 |
2 |
4.066667 |
1.076471 |
0.3462168 |
| Positive |
1.8 |
-2.2 |
2 |
29.600000 |
7.835294 |
0.0008341 |
| Negative |
3.2 |
-2.4 |
2 |
45.866667 |
12.141177 |
0.0000286 |
| Residuals |
NA |
NA |
72 |
136.000000 |
NA |
NA |
testInteractions(fit.1aov,fixed="wordtype", across="feedback",adjust="none")
| LF_LE |
0.1 |
-0.5 |
2 |
1.266667 |
0.3352941 |
0.7162383 |
| HF_LE |
-1.3 |
-0.7 |
2 |
5.266667 |
1.3941176 |
0.2546664 |
| HF_HE |
-3.0 |
-2.0 |
2 |
35.000000 |
9.2647059 |
0.0002627 |
| Residuals |
NA |
NA |
72 |
136.000000 |
NA |
NA |
Next we will begin to examine the origins of omnibus wordtype by
grade interaction that was significant. In order to interpret the sets
of simples possible in this 3x2 grid of means, we should visualize the
outcome.
# first, summarize the data set to produce a new data frame that ggplot can use
myData3.wg <- Rmisc::summarySE(data=bg.3way,measurevar="numrecall", groupvars=c("wordtype", "grade"))
# look at the new data frame that contains the summary statistics
myData3.wg
| LF_LE |
fifth |
15 |
8.066667 |
1.279881 |
0.3304638 |
0.7087744 |
| LF_LE |
twelfth |
15 |
8.266667 |
1.279881 |
0.3304638 |
0.7087744 |
| HF_LE |
fifth |
15 |
8.133333 |
1.302013 |
0.3361783 |
0.7210308 |
| HF_LE |
twelfth |
15 |
8.333333 |
1.397276 |
0.3607752 |
0.7737858 |
| HF_HE |
fifth |
15 |
5.400000 |
2.292846 |
0.5920103 |
1.2697358 |
| HF_HE |
twelfth |
15 |
7.400000 |
1.352247 |
0.3491486 |
0.7488493 |
#library(ggplot2)
#library(ggthemes)
# Now create the Default bar plot
p5 <- ggplot(myData3.wg, aes(x=wordtype, y=numrecall, fill=grade)) +
geom_bar(stat="identity", color="black",
position=position_dodge()) +
geom_errorbar(aes(ymin=numrecall-se, ymax=numrecall+se), width=.2,
position=position_dodge(.9))
p6 <- p5 +labs(title="Words Recalled +/- SEM", x="WordType", y = "Mean Number Recalled")+
theme_bw() +
theme(panel.grid.major.x = element_blank(),
panel.grid.major.y = element_blank(),
panel.grid.minor.x = element_blank(),
panel.grid.minor.y = element_blank(),
panel.background = element_blank(),
axis.line.y = element_line(colour="black", size=.7),
axis.line.x = element_line(colour="black", size=.7),
plot.title = element_text(hjust=.5)
) +
scale_fill_manual(values=c('honeydew3','honeydew2', 'honeydew1'))
p6
In the first of the testInteractions analyses in this next
code chunk (#3), the set of simple main effects of wordtype at levels of
grade is produced. From the graph we may expect wordtype to be
significant in the 12th graders, but perhaps not in the 5th graders.
This is exactly what happened, as seen in the first table. In the second
testInteractions output here (#4) the effect of grade is
significant only in the HF_HE condition, as the visual inspection would
have suggested.
# 3-4 effects of wordtype at levels of grade, and then grade at levels of feedback
# note these are collapsed on feedback, respectively
testInteractions(fit.1aov,fixed="grade", across="wordtype",adjust="none")
| fifth |
2.6 |
-2.7333333 |
2 |
72.933333 |
19.305882 |
0.0000002 |
| twelfth |
0.8 |
-0.9333333 |
2 |
8.133333 |
2.152941 |
0.1235609 |
| Residuals |
NA |
NA |
72 |
136.000000 |
NA |
NA |
testInteractions(fit.1aov,fixed="wordtype", across="grade",adjust="none")
| LF_LE |
-0.2 |
1 |
0.3 |
0.1588235 |
0.6914214 |
| HF_LE |
-0.2 |
1 |
0.3 |
0.1588235 |
0.6914214 |
| HF_HE |
-2.0 |
1 |
30.0 |
15.8823529 |
0.0001597 |
| Residuals |
NA |
72 |
136.0 |
NA |
NA |
The third omnibus 2way can also be followed up with either of its set
of simple main effects.
# 5-6 effects of feedback at levels of grade, and then grade at levels of feedback
# note these are collapsed on wordtype
testInteractions(fit.1aov,fixed="grade", across="feedback",adjust="none")
| fifth |
-2.2 |
-1.6666667 |
2 |
32.933333 |
8.7176471 |
0.0004071 |
| twelfth |
-0.6 |
-0.4666667 |
2 |
2.533333 |
0.6705882 |
0.5145724 |
| Residuals |
NA |
NA |
72 |
136.000000 |
NA |
NA |
testInteractions(fit.1aov,fixed="feedback", across="grade",adjust="none")
| None |
0.0666667 |
1 |
0.0333333 |
0.0176471 |
0.8946887 |
| Positive |
-1.1333333 |
1 |
9.6333333 |
5.1000000 |
0.0269569 |
| Negative |
-1.3333333 |
1 |
13.3333333 |
7.0588235 |
0.0097084 |
| Residuals |
NA |
72 |
136.0000000 |
NA |
NA |
Contrast Analyses in
factorial designs using phia
Initially, we recreate the custom contrasts for the Wordtype and
Feedback factors since they each have more than two levels. Note that
these are the same contrasts employed above in the omnibus analyses, but
phia requires them to be created in a slightly different manner for
usage in the testInteraction function. The exact orthogonal
sets employed here were arbitrarily chosen
# first for the feedback factor:
fbc1 <- list(feedback=c(-.5, -.5, 1)) # same as defined above, except using fractions
fbc2 <- list(feedback=c(-1, 1, 0)) # same as defined above, except using fractions
fbc1
## $feedback
## [1] -0.5 -0.5 1.0
## $feedback
## [1] -1 1 0
# now for the wordtype factor
wtypec1 <- list(wordtype=c(1, -.5, -.5)) # same as defined above, except using fractions
wtypec2 <- list(wordtype=c(0, -1, 1)) # same as defined above, except using fractions
wtypec1
## $wordtype
## [1] 1.0 -0.5 -0.5
## $wordtype
## [1] 0 -1 1
Simple Effect
Contrasts and Simple Interaction Contrasts
We already obtained 2 and 3-way interaction contrasts above with the
summary/split approach and by using summary.lm to obtain
Type III SS equivalents. Here, we obtain simple interaction contrasts
and simple effect contrasts.
In this code, notice that testInteractions takes a
“custom” argument and that is where we can specify the contrast to be
used.
Simple 2 way
interaction contrasts
First we examine the simple two-way two way interactions (even though
the 3way is NS) to serve as a template. NEEDS REWORDING Note that the
labeling in the table is odd. The twotestInteraction
functions produce the two tables. The first examines the effect
involving the first contrast on feedback (called fbc1 in the custom
argument), and the second examines the second contrast on feedback
(called fbc1). Examining the first of the two tables, each row
represents evaluation of a simple two way interaction contrast of the
Feedback (first contrast) and Grade at each of the three levels of
Wordtype.
The use of the colon symbol does NOT imply an interaction between
those two effects. In addition, the labeling of the Feedback effect as
“feedback1” doesn’t refer to the exact contrast that we called fbc1. It
is just a generic label for the fact that a contrast on feedback was
specified. So in the first table, the first F value of .0794 represents
a test of fbc1 by grade AT LF_LE. The second, is the same effect, but AT
HF_LE and so forth. In the second table, the first row evaluates the
fbc2 by grade interaction AT LF_LE, with the F value of .0265 EVEN
THOUGH the table still uses the “feedback1” label. The confusion can
arise because the tables do not employ the fbc1 and fbc2 labels that we
created. In each table, there are three tests because there are three
levels of Wordtype and the effect is either fbc1 by grade (table 1) or
fbc2 by grade (table 2). This pattern plays out in subsequent
implementations as well.
# first, an example of simple 2-way interaction contrasts of feedback with grade at levels of wordtype
testInteractions(fit.1aov, fixed="wordtype", custom=fbc1, adjustment="none", across="grade")
| LF_LE : feedback1 |
-0.3 |
1 |
0.15 |
0.0794118 |
0.7789045 |
| HF_LE : feedback1 |
-0.3 |
1 |
0.15 |
0.0794118 |
0.7789045 |
| HF_HE : feedback1 |
-1.8 |
1 |
5.40 |
2.8588235 |
0.0951989 |
| Residuals |
NA |
72 |
136.00 |
NA |
NA |
testInteractions(fit.1aov, fixed="wordtype", custom=fbc2, adjustment="none", across="grade")
| LF_LE : feedback1 |
-0.2 |
1 |
0.05 |
0.0264706 |
0.8712121 |
| HF_LE : feedback1 |
-0.2 |
1 |
0.05 |
0.0264706 |
0.8712121 |
| HF_HE : feedback1 |
-3.2 |
1 |
12.80 |
6.7764706 |
0.0112095 |
| Residuals |
NA |
72 |
136.00 |
NA |
NA |
When both factors in a simple 2way interaction have contrasts then
the code specification is slightly more complex. With the interaction of
wordtype and feedback at levels of grade, grade is specified as “fixed”.
The “across” argument is not used. Rather, both contrasts are specified
simultaneously in the “custom” argument and `testInteractions
understands that the interaction of those two contrasts is to be
examined.
Note that the output uses the colon symbol again, but it doesn’t mean
interaction. The interaction here is between fbc1 and wtypec1 (thus an
interaction contrast) at fifth grade.
# simple 2-2way interaction contrast of wordtype and grade at levels of feedback
# one requested at a time here, both of the first contrasts (fbc1 and wtypec1) are requested.
testInteractions(fit.1aov, fixed="grade", custom=c(fbc1,wtypec1), adjustment="none")
| fifth : feedback1 : wordtype1 |
1.50 |
1 |
5.00 |
2.6470588 |
0.1081106 |
| twelfth : feedback1 : wordtype1 |
0.75 |
1 |
1.25 |
0.6617647 |
0.4186197 |
| Residuals |
NA |
72 |
136.00 |
NA |
NA |
Code for the other six contrasts available in this set is shown here,
but results suppressed to save space. If one were to run them the same
“feedback1” and “wordtype1” labels would appear, but meaning either
fbc1/fbc2 or wtypec1/wtypec2, respectively.
testInteractions(fit.1aov,fixed="grade", custom=c(fbc1,wtypec2), adjustment="none")
testInteractions(fit.1aov,fixed="grade", custom=c(fbc2,wtypec1), adjustment="none")
testInteractions(fit.1aov,fixed="grade", custom=c(fbc2,wtypec2), adjustment="none")
Sometimes we prefer to examine simple interaction comparisons rather
than simple interaction contrasts. That kind of effect is exemplified
with this type of code where a contrast is specified for one factor
(wordtype here) and the other factor is not broken into contrasts
(feedback here, specified in the “across” argument). These will be a 2
df interaction contrasts: either wordtype contrast 1 by feedback at
levels of grade or wordtype contrast2 by feedback at levels of
grade.
Once again the fifth: and twelfth: notation just specify the levels
of grade AT which the simple interaction comparisons are located.
# now simple 2-way interaction comparisons of wordtype contrasts and feedback at levels of grade
# note that these effects have 2 df
testInteractions(fit.1aov, fixed="grade", custom=wtypec1, adjustment="none", across="feedback")
| fifth : wordtype1 |
3.0 |
1.6 |
2 |
9.266667 |
2.4529412 |
0.0932021 |
| twelfth : wordtype1 |
1.5 |
0.1 |
2 |
1.266667 |
0.3352941 |
0.7162383 |
| Residuals |
NA |
NA |
72 |
136.000000 |
NA |
NA |
testInteractions(fit.1aov, fixed="grade", custom=wtypec2, adjustment="none", across="feedback")
| fifth : wordtype1 |
-3.2 |
-2.8 |
2 |
14.0666667 |
3.7235294 |
0.028918 |
| twelfth : wordtype1 |
-0.2 |
0.2 |
2 |
0.0666667 |
0.0176471 |
0.982512 |
| Residuals |
NA |
NA |
72 |
136.0000000 |
NA |
NA |
Simple and
simple-simple main effect contrasts
Contrasts breaking down the simple-simple or simple main effects are
also available. Once again, care is required in reading the table, which
does not label the different contrasts. The first table is for the fbc1
contrast and the second table for the fbc2 contrast, each at the six
combined levels of Grade and Wordtype in this first illustration.
Once again, the “feedback1” label in the tables is misleading and not
necessary. We have to discern which contrast on feedback is being tested
by knowing which code line produced which table.
# Now, how about simple SME contrasts for the feedback factor at
# combined levels of grade and wordtype
testInteractions(fit.1aov, fixed=c("wordtype", "grade"), custom=fbc1, adjustment="none")
| LF_LE : fifth : feedback1 |
-0.1 |
1 |
0.0333333 |
0.0176471 |
0.8946887 |
| HF_LE : fifth : feedback1 |
-0.8 |
1 |
2.1333333 |
1.1294118 |
0.2914524 |
| HF_HE : fifth : feedback1 |
-2.4 |
1 |
19.2000000 |
10.1647059 |
0.0021193 |
| LF_LE : twelfth : feedback1 |
0.2 |
1 |
0.1333333 |
0.0705882 |
0.7912415 |
| HF_LE : twelfth : feedback1 |
-0.5 |
1 |
0.8333333 |
0.4411765 |
0.5086768 |
| HF_HE : twelfth : feedback1 |
-0.6 |
1 |
1.2000000 |
0.6352941 |
0.4280406 |
| Residuals |
NA |
72 |
136.0000000 |
NA |
NA |
testInteractions(fit.1aov, fixed=c("wordtype", "grade"), custom=fbc2, adjustment="none")
| LF_LE : fifth : feedback1 |
-0.6 |
1 |
0.9 |
0.4764706 |
0.4922445 |
| HF_LE : fifth : feedback1 |
-0.8 |
1 |
1.6 |
0.8470588 |
0.3604591 |
| HF_HE : fifth : feedback1 |
-3.6 |
1 |
32.4 |
17.1529412 |
0.0000926 |
| LF_LE : twelfth : feedback1 |
-0.4 |
1 |
0.4 |
0.2117647 |
0.6467745 |
| HF_LE : twelfth : feedback1 |
-0.6 |
1 |
0.9 |
0.4764706 |
0.4922445 |
| HF_HE : twelfth : feedback1 |
-0.4 |
1 |
0.4 |
0.2117647 |
0.6467745 |
| Residuals |
NA |
72 |
136.0 |
NA |
NA |
Next, we evaluate Simple Main effect contrasts of Feedback at levels
of Wordtype, collapsed on Grade. These next three code chunks also
produce the two contrast tests with and without a “holm” adjustment.
The first code chunk is thus the test of feedback contrast 1 at
levels of wordtype, with no p value adjustment. Once again, the
“feedback1” label can be ignored. As an interpretation example, consider
the third F test with a value of 7.94. This effect is feedback contrast
1 in the HF_HE group (collapsed on grade)
# or SME contrasts at levels of wordtype (collapsed on grade)
testInteractions(fit.1aov, fixed=c("wordtype"), custom=fbc1, adjustment="none")
| LF_LE : feedback1 |
0.05 |
1 |
0.0166667 |
0.0088235 |
0.9254228 |
| HF_LE : feedback1 |
-0.65 |
1 |
2.8166667 |
1.4911765 |
0.2260177 |
| HF_HE : feedback1 |
-1.50 |
1 |
15.0000000 |
7.9411765 |
0.0062342 |
| Residuals |
NA |
72 |
136.0000000 |
NA |
NA |
The next chunk does the same analysis with a “holm” adjustment.
# note that adjustment for Type I error inflation can be done. The "adjustment"
# argument uses the methods available in p.adjust
testInteractions(fit.1aov, fixed=c("wordtype"), custom=fbc1, adjustment="holm")
| LF_LE : feedback1 |
0.05 |
1 |
0.0166667 |
0.0088235 |
0.9254228 |
| HF_LE : feedback1 |
-0.65 |
1 |
2.8166667 |
1.4911765 |
0.4520354 |
| HF_HE : feedback1 |
-1.50 |
1 |
15.0000000 |
7.9411765 |
0.0187027 |
| Residuals |
NA |
72 |
136.0000000 |
NA |
NA |
And the same approach is used for the second feedback contrast.
testInteractions(fit.1aov, fixed=c("wordtype"), custom=fbc2, adjustment="none")
| LF_LE : feedback1 |
-0.5 |
1 |
1.25 |
0.6617647 |
0.4186197 |
| HF_LE : feedback1 |
-0.7 |
1 |
2.45 |
1.2970588 |
0.2585263 |
| HF_HE : feedback1 |
-2.0 |
1 |
20.00 |
10.5882353 |
0.0017348 |
| Residuals |
NA |
72 |
136.00 |
NA |
NA |
testInteractions(fit.1aov, fixed=c("wordtype"), custom=fbc2, adjustment="holm")
| LF_LE : feedback1 |
-0.5 |
1 |
1.25 |
0.6617647 |
0.5170526 |
| HF_LE : feedback1 |
-0.7 |
1 |
2.45 |
1.2970588 |
0.5170526 |
| HF_HE : feedback1 |
-2.0 |
1 |
20.00 |
10.5882353 |
0.0052044 |
| Residuals |
NA |
72 |
136.00 |
NA |
NA |
There are many other simple main effect contrasts that could be
exmained (following up on the six sets of possible simple main effects
described above). The important to examine in the context of this data
set would be contrasts on wordtype at levels of grade, since the omnibus
wordtype by grade interaction was significant. In order to shorten this
doc, only one of those SME contrasts is shown here, wordtype contrast2 @
fifth grade.
testInteractions(fit.1aov, fixed="grade", custom=fbc2, adjustment="none")
| fifth : feedback1 |
-1.6666667 |
1 |
20.833333 |
11.0294118 |
0.0014106 |
| twelfth : feedback1 |
-0.4666667 |
1 |
1.633333 |
0.8647059 |
0.3555324 |
| Residuals |
NA |
72 |
136.000000 |
NA |
NA |
3way interaction
contrasts with phia
Next, we will examine 3-way interaction contrasts. This shows that
phia can recreate what we did above with the
summary/split approach and with the summary.lm function.
Those two match because of the equal sample sizes in this example data
set. If the data set were unbalanced, the tests here, from
testInteractions would match the squared t-values from
tests obtained in the summary.lm output.
Here, two of the three factors have contrasts available and those
specifications are put in the “custom” argument. The third factor is
only a 2-level factor (already a contrasts), and can just be specified
in the “across” argument.
testInteractions(fit.1aov, custom=c(fbc1,wtypec1), adjustment="none",across="grade")
| feedback1 : wordtype1 |
0.75 |
1 |
0.625 |
0.3308824 |
0.5669333 |
| Residuals |
NA |
72 |
136.000 |
NA |
NA |
testInteractions(fit.1aov, custom=c(fbc1,wtypec2), adjustment="none",across="grade")
| feedback1 : wordtype1 |
-1.5 |
1 |
1.875 |
0.9926471 |
0.3224329 |
| Residuals |
NA |
72 |
136.000 |
NA |
NA |
testInteractions(fit.1aov, custom=c(fbc2,wtypec1), adjustment="none",across="grade")
| feedback1 : wordtype1 |
1.5 |
1 |
1.875 |
0.9926471 |
0.3224329 |
| Residuals |
NA |
72 |
136.000 |
NA |
NA |
testInteractions(fit.1aov, custom=c(fbc2,wtypec2), adjustment="none",across="grade")
| feedback1 : wordtype1 |
-3 |
1 |
5.625 |
2.977941 |
0.0886962 |
| Residuals |
NA |
72 |
136.000 |
NA |
NA |
Two-way interaction
contrasts
By leaving the “across=grade” argument out of the above code chunk,
we can obtain the two-way interaction contrasts of Feedback an Wordtype
(collapsed on Grade).
testInteractions(fit.1aov, custom=c(fbc1,wtypec1), adjustment="none")
| feedback1 : wordtype1 |
1.125 |
1 |
5.625 |
2.977941 |
0.0886962 |
| Residuals |
NA |
72 |
136.000 |
NA |
NA |
testInteractions(fit.1aov, custom=c(fbc1,wtypec2), adjustment="none")
| feedback1 : wordtype1 |
-0.85 |
1 |
2.408333 |
1.275 |
0.2625789 |
| Residuals |
NA |
72 |
136.000000 |
NA |
NA |
testInteractions(fit.1aov, custom=c(fbc2,wtypec1), adjustment="none")
| feedback1 : wordtype1 |
0.85 |
1 |
2.408333 |
1.275 |
0.2625789 |
| Residuals |
NA |
72 |
136.000000 |
NA |
NA |
testInteractions(fit.1aov, custom=c(fbc2,wtypec2), adjustment="none")
| feedback1 : wordtype1 |
-1.3 |
1 |
4.225 |
2.236765 |
0.1391332 |
| Residuals |
NA |
72 |
136.000 |
NA |
NA |
Finally, in this data set, the most interesting interaction effect to
follow up would have been the Wordtype By Grade interaction which was
the only significant one. Simple main effects of Wordtype at Levels of
Grade (and the contrasts of those SME) were obtained above. Here, the
interaction contrasts were obtained to decompose this two way
interaction.
Notice that the sum of the SS for the two effects equals the SS for
the Wordtype by Grade interaction seen in the initial ANOVA summary
table, as it should for an orthogonal partitioning.
In this pair of analyses, only the second of the two contrasts is
significant and this should not be surprising. Most of what was varying
in the wordtype by grade array of means was the position of the HF_HE
group. So, it makes sense that the second contrast which compares HF_LE
to HF_HE interacts with grade since the difference is larger in the 12th
graders.
testInteractions(fit.1aov, custom=wtypec1, across="grade", adjustment="none")
| wordtype1 |
0.9 |
1 |
4.05 |
2.144118 |
0.14747 |
| Residuals |
NA |
72 |
136.00 |
NA |
NA |
testInteractions(fit.1aov, custom=wtypec2, across="grade", adjustment="none")
| wordtype1 |
-1.8 |
1 |
12.15 |
6.432353 |
0.0133756 |
| Residuals |
NA |
72 |
136.00 |
NA |
NA |
The 
