PSY254
Statistical Applications
in Behavioral Sciences
ANOVA: One way ANOVA
Dr. Burak Emre Gürsoy
ANOVA: One way ANOVA
ANOVA
• This week introduces one of the most widely used tools
in psychological statistics, known as “the analysis of
variance”, but usually referred to as ANOVA.
• The basic technique was developed by Sir Ronald Fisher
in the early 20th century.
Important topics: effect sizes, post hoc tests and
corrections
ANOVA: One way ANOVA
ANOVA
• Suppose you’ve become involved in a clinical trial in which
you are testing a new antidepressant drug called Joyzepam.
• In order to construct a fair test of the drug’s effectiveness,
the study involves three separate drugs to be administered.
• One is a placebo, and the other is an existing
antidepressant / anti-anxiety drug called Anxifree.
• A collection of 18 participants with moderate to severe
depression are recruited for your initial testing.
ANOVA: One way ANOVA
ANOVA
• Because the drugs are sometimes administered in conjunction
with psychological therapy, your study includes 9 people
undergoing cognitive behavioral therapy (CBT) and 9 who are
not.
• Participants are randomly assigned (doubly blinded, of course) a
treatment, such that there are 3 CBT people and 3 no-therapy
people assigned to each of the 3 drugs.
• Psychologist assesses the mood of each person after a 3-month
run with each drug, and the overall improvement in each
person’s mood is assessed on a scale ranging from −5 to +5.
ANOVA: One way ANOVA
ANOVA
Main question: the effect of drug on [Link]
Data: [Link]
ANOVA: One way ANOVA
ANOVA: One way ANOVA
How ANOVA works
• We’re interested in comparing the average mood change for the
• In that sense, we’re talking about an analysis similar to the 𝑡-
three different drugs.
test
ANOVA: One way ANOVA
A reminder:
ANOVA: One way ANOVA
A reminder:
So much work for one short sentence.
“One-way ANOVA showed a significant effect of drug on mood gain ( 𝐹 (2,
15) = 18.6, 𝑝 < .001).”
ANOVA: One way ANOVA
Using jamovi for ANOVA
• Go to the ‘ANOVA’ – ‘ANOVA’ analysis
• The [Link] ‘Dependent Variable’
• The drug variable ‘Fixed Factors’
ANOVA: One way ANOVA
Using jamovi for ANOVA
• The jamovi results table shows you the sums of squares values,
the degrees of freedom, and a couple of other quantities
• However, that jamovi doesn’t use the names “between group” and
“within group”. Instead, it tries to assign more meaningful names.
In our example,
• The between groups variance corresponds to the effect that the
drug has on the outcome variable,
• The within groups variance corresponds to the “leftover” variability
so it calls that the residuals.
ANOVA: One way ANOVA
ANOVA: One way ANOVA
Effect size
ANOVA, but the most commonly used measures are 𝜂2 (eta squared)
• There’s a few different ways you could measure the effect size in an
and partial 𝜂2.
• The interpretation of 𝜂2 is equally straightforward. It refers to the
• For a one-way analysis of variance, they’re identical to each other.
proportion of the variability in the outcome variable ([Link]) that can
• A value of 𝜂2 = 0 means that there is no relationship at all between the
be explained in terms of the predictor (drug).
two, whereas a value of 𝜂2 = 1 means that the relationship is perfect.
• Better yet, the 𝜂2 value is very closely related to 𝑅 square,
ANOVA: One way ANOVA
• Although many statistics textbooks suggest 𝜂2 as the default
Note:
effect size measure in ANOVA, there’s an interesting blog post
by Daniel Lakens suggesting that eta-squared is perhaps not
the best measure of effect size in real-world data analysis,
because it can be a biased estimator.
squared (𝜔2), which is less biased, alongside etasquared.
• Usefully, there is also an option in jamovi to specify omega-
[Link]
[Link]
ANOVA: One way ANOVA
Multiple comparisons and post hoc tests
• Which groups are actually different from one another?
• In our example, our null hypothesis was that all three drugs
(placebo, Anxifree and Joyzepam) have the exact same effect
on mood.
ANOVA: One way ANOVA
Multiple comparisons and post hoc tests
• How might we go about solving our problem?
• Given that we’ve got three separate pairs of means (placebo
versus Anxifree, placebo versus Joyzepam, and Anxifree versus
separate 𝑡-tests and see what happens.
Joyzepam) to compare, what we could do is run three
• Is it ok?
ANOVA: One way ANOVA
Multiple comparisons and post hoc tests
• How might we go about solving our problem?
• Given that we’ve got three separate pairs of means (placebo
versus Anxifree, placebo versus Joyzepam, and Anxifree versus
separate 𝑡-tests and see what happens.
Joyzepam) to compare, what we could do is run three
• Is it ok?
• NO! - We’re running lots and lots of tests without much
theoretical guidance in the hope that some of them come up
significant - “fishing expedition”
ANOVA: One way ANOVA
Corrections for multiple testing
Post hoc analysis (“post hoc” being Latin for “after this”)
Bonferroni corrections: Although the Bonferroni correction is the simplest
- “multiply all your raw 𝑝-values by 𝑚”
adjustment out there, it’s not usually the best one to use.
you’re doing the tests sequentially, starting with the smallest (raw) 𝑝-value
Holm corrections: The idea behind the Holm correction is to pretend that
-“ the biggest 𝑝-value remains unchanged, the second biggest 𝑝-value is
and moving onto the largest one.
doubled, the third biggest 𝑝-value is tripled, and so on ”
Default
Games Howell – Unequal variances
Tukey – Equal variances
ANOVA: One way ANOVA
Report
effect of drug on mood gain (𝐹 (2, 15)
One-way ANOVA showed a significant
= 18.6, 𝑝 < .001, η2 = .71 ). Holm Post
hoc tests indicated that Joyzepam
change than both Anxifree (𝑝 = .001)
produced a significantly larger mood
and the placebo (𝑝 < .001). There is
better than the placebo (𝑝 = .15).
no evidence that Anxifree performed
ANOVA: One way ANOVA
The assumptions of one-way ANOVA
• Homogeneity of variance
• Normality
• Independence
ANOVA: One way ANOVA
The assumptions of one-way ANOVA
• Levene test is non-significant (𝐹2,15 = 1.45, 𝑝 =.266), so it looks
Homogeneity of variance:
like the homogeneity of variance assumption is fine
• However, looks can be deceptive!
• If your sample size is pretty big, then the Levene test could show
up a significant effect (i.e. p < .05)
• What this means is that, alongside any statistical test of the
assumption being met, you should always plot the standard
deviation around the means for each group / category in the
analysis…just to see if they look fairly similar
• How do we save our ANOVA when the homogeneity of variance
assumption is violated?
Welch one-way test!
ANOVA: One way ANOVA
The assumptions of one-way ANOVA
• If the Shapiro-Wilk test is not significant (i.e. 𝑝 >
Normality
.05) then this indicates that the assumption of
normality is not violated.
• However, as with Levene’s test, if the sample
size is large then a significant Shapiro-Wilk test
may in fact be a false positive, where the
assumption of normality is not violated in any
substantive problematic sense for the analysis.
• How do we save our ANOVA when the
homogeneity of variance assumption is violated?
Kruskal-Wallis test
ANOVA: One way ANOVA
The assumptions of one-way ANOVA
Independence
• There’s not an obvious or simple way to test for this, but there
are some situations that are clear violations of this.
• For instance, if you have a repeated measures design, where
each participant in your study appears in more than one
condition, then independence doesn’t hold.
• There’s a special relationship between some observations,
namely those that correspond to the same person! When that
happens, you need to use something like a Repeated measures
one-way ANOVA.
