Experimental Design

Basic Concepts

Experimental Unit
An experimental unit is defined as any entity that receives a
treatment. This could be plant, person, plot of ground, petri dish
e.t.c

Treatment and control groups

The terminologies treatment group and control group is common in
experimental research. When a group of subjects are administered
an experimental stimulus that group is referred to as treatment
group and the stimulus is called treatment while other subjects who
are not given such a stimulus are referred to as the control group.
Individuals in control groups are usually given placebo that is a
substance that looks like the treatment but has no capacity of
initiating an response. For instance in a drug trial if the treatment is
in form of liquid, the placebo could be just distilled water.
Generally treatment is considered successful if subjects in the
treatment group respond more favourably on outcome variables
than control group subjects. E.g Consider a treatment group that is
administerd chloroquinne and the control group, distilled water. If
the individuals get cured at the end of the experiment and malaria
persists in the control group, the treatment is said to be successful.

Multiple levels of experimental stimulus or treatment may be

administered, in which case, there may be more than one treatment
group. For example, in order to test the effects of a plant extract on
the treatment of malaria in mice induced with plasmodium berghei,
the mice are randomly divided into perhaps five groups, the four
groups receiving different doses of the extract (e.g 800ppm,
600ppm, 400ppm, 200ppm) and the fifth group receives a placebo
such as a sugar solution (control group). The first four groups are
experimental groups and the fifth group is a control group. After
administering the extract for a period of time, if the condition of the
experimental group subjects improved significantly more than the
control group subjects, we can say that the extract is effective. The
researcher can also compare the conditions of the high and low
dosage experimental groups to determine if the high dose is more
effective than the low dose.

Treatment manipulation
Treatment manipulation helps control for the “cause” in cause-effect
relationships. Therefore validity of experimental research depends
on how well the treatment was manipulated. Treatment
manipulation are usually checked using pre-tests and pilot tests
prior to the experimental study. Any measurements conducted

1X
before the treatment is administered are called pre-test measures,
while those conducted after the treatment are post-test measures.

Random selection and assignment

Random selection is the process of randomly drawing a sample from
a population or a sampling frame. This approach assures that each
unit in the population has a positive chance of being selected into
the sample. Random assignment is however a process of randomly
assigning subjects to experimental or control groups. This is a
standard practice in true experimental research to ensure that
treatment groups are similar to each other and to the control group,
prior to administration of treatment. It is possible to have both
random selection and random assignment in well-designed
experimental research.

True Experimental Research Design

A true experimental research design is one of the most accurate
forms of research design because it provides specific scientific
evidence. It requires:
I. manipulating an independent variable,
II. random assignment of participants to different groups, and
III. measuring the dependent variable.
Furthermore, only a true experimental design can truly establish a
cause-effect relationship within a group. The unique strength of
experimental design is its ability to link cause and effect through
treatment manipulation, while controlling for the spurious effect of
extraneous variables [variables in the experimental environment
that could influence outcomes].

Conditions for true experimental design

1. There is a control group that is not subjected to

changes and an experimental group that will
experience the changed variables
True experiments always include a control group and at least
one experimental group. The control group consists of
subjects that are similar to the experimental group, but they
do not receive the experimental treatment. This allows the
researcher to establish baseline data to compare against the
experimental results. The experimental group is the group of
subjects who receive the experimental treatment. True
experimental design may include more than one experimental
group.

2. A variable that can be manipulated by the researcher

Remember for every research, there are hypotheses to be

tested. The same rule applies here, so, the researcher

2X
 manipulates the factor that is the basis for the hypothesis.
Manipulating that specific variable will allow the researcher to
affect the outcome and will provide the best evidence of a
cause-and-effect relationship.
There are two types of variables in an experiment: an
independent variable and a dependent variable.
The independent variable, also called the predictor
variable, is the factor that the researcher controls.
The dependent variable, also called the outcome
variable, is the outcome the researcher is attempting to
study in a measurable way. The outcome variable is not
manipulated, but rather it is measured as a way to judge the
researcher's hypothesis.

3. Random distribution of the variables

It is critical that the participants in a true experiment have an

equal chance of being assigned to either the control group or
one of the experimental groups. This random assignment
ensures that any differences in the groups are due to chance.
True experiments are often conducted by putting the subjects
into groups in such a way that neither the subjects nor the
researcher know which group a particular subject is a member
of. This is called a double blind study. If neither the study
participant nor the researcher knows if a subject is a member
of the control or the experimental group, the study will not be
tainted by preconceived theories or beliefs.

Types of Experimental Design

1. Completely Randomised Design (CRD)
This is the simplest of all experimental designs. It is one where the
treatments are assigned completely at random so that each
experimental unit has the same chance of receiving any one
treatment. The design assumes that the experimental units are
homogenous. For instance, applying this design method to the
cholesterol-level study, the three types of exercise program
(treatment) would be randomly assigned to the experimental units
(patients). CRD is mostly useful in laboratory, green house and pot
culture experiments in agricultural studies, animal feeding
experiments, environmental experiments. The CRD is best suited
for experiments with a small number of treatments.

3X
The first step is to identify the experimental units and the number of
replications that will be used, then assign the treatments to the
experimental units in a random manner. This is typically done by
listing the treatments and assigning a random number to each.
Example
Imagine a researcher in the field of tissue culture wants to study the
effect the growth regulator, BAP, on micro-propagation of papaya
[Carica papaya L] in the laboratory. He works with the following
hypotheses in mind
Null hypothesis
There are no significant differences in root length of papaya in
response to different concentrations of BAP

Alternative hypothesis
There are significant differences in root length of papaya in
response to different concentrations of BAP

The CRD would be the most appropriate design for the experiment
because it satisfies the required conditions: he would be working
with different concentrations of BAP (treatment), experimental unit
could be homogenous because he would work with shoot tips of
same length. The randomisation and layout of the experiment would
be as follows:

Number of treatments (T): 5 concentrations of BAP

Number of replicates (R): 4 times

Total number of experimental units (N): N = T* R = 5*4 = 20

4X
 1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

17 18 19 20

Experimental units in 20 test tubes

1:T2 2:T3 3:T2 4:T4

5:T4 6:T5 7:T1 8:T5

Treatments were randomly
distributed to 9:T4 10:T2 11: T3 12 T3
experimental units
13:T5 14:T5 15:T1 16:T1
Any difference among
17:T3 18:T2 19:T4 20:T1
experimental units
receiving the same
treatment is considered as experimental error. Hence, CRD is
appropriate only for experiments with homogeneous experimental
units, such as laboratory experiments, where environmental effects
are relatively easy to control. For field experiments, where there is
generally large variation among experimental plots in such
environmental factors as soil, the CRD is rarely used because it
would be difficult to control extraneous variables like environmental
factors.
Advantages
1. Simple to use
2. Flexible
3. Statistical analysis is easy and straight forward
4. Lost experimental unit does not complicate statistical
analysis.
E
Disadvantages
1. Experimental units are expected to be homogenous which in
reality is difficult

2. The size of experimental error is relatively large compared to

other designs

Randomised Block Design

In this type of design the researcher divides experimental units
(subjects, plants, patients) into sub groups called blocks such that
the variability within the blocks is less than the variability between

5X
the blocks. The treatments are randomly allocated to
the experimental units within each block. This kind of design is used
to minimize the effects of systematic error such as variability and
potential confounding factors.
Example 1
A field of corn is affected by a plant disease and the experimenter
wants to test the efficacy of different fungicides in controlling it.
Using randomised block design, the field is split into blocks and each
block is randomly treated with the various fungicides to be tested.
By splitting the field into blocks, the farm may be able to account for
certain variations and confounding variables that might exist in the
field.
Block Treatment
F1 F2 F3 F4 Control
A 10 10 10 10 10
B 10 10 10 10 10
C 10 10 10 10 10
D 10 10 10 10 10

Example 2
Consider an experiment testing the efficacy of Artemisinin
combination therapy (ACT) in the treatment of malaria among 2000
patients. Using randomised block design, participants are assigned
to blocks based on gender (table ). Then within each block,
participants are randomly assigned to treatment.

Gender Treatment

ACT Placebo

Male 500 500

Female 500 500

Each block has equal number of participants thereby removing

gender as a potential source of variability and as a confounding
variable.

6X
Factorial Designs
Experiments that require the study of the effects of several factors
(the independent variable that is manipulated in the experiment by
the researcher or through selection) in a single experiment fall into
factorial design. Factorial designs enable the researcher to examine
not only the individual effect of each treatment on the dependent
variables (called main effects), but also their joint effect (called
interaction effects).

Types of Factorial Designs

2 X 2 Design
This is the most basic factorial design. It consists of two treatments,
each with two levels (such as high/low or present/absent) in other
words the effect of two independent variables (each with two levels)
on a single dependent variable. For instance, let’s say that you want
to understand the effects of sunlight (low or high) and watering
frequency (daily or weekly) on the growth of certain plant species.
The two independent variables are Sunlight and watering frequency.
Notice that that each has two levels: for sunlight, it is either high or
low while the watering frequency is daily and weekly.

Watering frequency
Daily Weekly

Plant
Plant
Low Growth
Growth
Sunlight

Plant Plant
High Growth Growth

Figure 2 x 2 factorial design

The dependent variable is just one and that is plant growth. A 2 x 2

factorial design allows the experimenter to analyse the following
effects:

Main effects:
These are the effects that just one independent variable has on the
dependent variable. For example, using the scenario above the
main effects of sunlight and water frequency on plant growth could

7X
be determined by finding the mean plant growth of all the plants
that

1. received low sunlight.

2. received high sunlight.
3. were watered daily
4. were watered weekly

Interaction Effects
These occur when the effect that one independent variable has on
the dependable variable depends on the level of the other
independent variable. For instance, using the scenario above again
we could analyse the following interaction effects:

1. Does the effect of sunlight on plant growth depend on

watering frequency?
2. Does the effect of watering frequency on plant growth depend
on the amount of sunshine?
3. When we use 2 x 2 factorial designs, we often plot graph of
the means to gain a better understanding of the effects that
the independent variables have on the dependent variable.

Here’s how to interpret the values in the plot:

● The mean growth for plants that received high sunlight and
daily watering was about 8.2 inches.
● The mean growth for plants that received high sunlight and
weekly watering was about 9.6 inches.
● The mean growth for plants that received low sunlight and
daily watering was about 5.3 inches.
● The mean growth for plants that received low sunlight and
weekly watering was 5.8 inches.
Now the interpretation of this is that there is no interaction
effect (between watering frequency and sunlight exposure)

8X
 since the two lines are roughly parallel. However,
peradventure the lines are not parallel then, there is an
interaction effect.

In the second scenario, the two lines are not parallel at all (in fact,
they cross!), which indicates that there is likely an interaction effect
between them. The two lines are not parallel at all (in fact, they
cross!), which indicates that there is likely an interaction effect
between them. For example, this means the effect that sunlight has
on plant growth depends on the watering frequency. In other words,
sunlight and watering frequency do not affect plant growth
independently. Rather, there is an interaction effect between the
two independent variables.

Plotting the means is a visual way of examining the effects the

independent variables have on the dependent variable. However
you can also analyse the effects by performing a Two-way ANOVA to
formally test whether or not the independent variables have a
statistically significant relationship with the dependent variable.
ANOVA is treated in depth in chapter . There are other factorial
designs and these are the 2 x 3 and 2x 2 x 2 factorial designs

2 x 3 Factorial Design
If you wish to add a third independent variable to the dependent
variable then one of the independent variable would have three
levels and the other one would have two levels. This would be a 2 x
3 factorial design. Let’s go back to our previous hypothetical
example. Suppose the experimenter wants to understand the
effects of sunlight (low vs. medium vs. high) and watering frequency
(daily vs. weekly) on the growth of a certain species of pant.

9X
 Watering frequency

Daily Weekly

Plant Plant
Low
Growth Growth

Sunlight

Plant Plant
Medium
Growth Growth

Plant Plant
High
Growth Growth

This is an example of a 2×3 factorial design because there are two

independent variables, one having two levels and the other having
three levels:

⮚ Independent variable 1: Sunlight

● Levels: Low, Medium, High
⮚ Independent variable 2: Watering Frequency
● Levels: Daily, Weekly

⮚ Dependent variable: Plant growth.

Just as in 2 x 2 designs, a 2×3 factorial design also allows you to

analyze:

Main Effects of sunlight on plant growth:

● Mean growth of all plants that received low sunlight.

● Mean growth of all plants that received medium
sunlight.
● Mean growth of all plants that received high sunlight.

Main effect of watering frequency on plant growth:

10X
 ● Mean growth of all plants that were watered daily.
● Mean growth of all plants that were watered weekly.

Interaction Effects: From our example, we can analyze the following

interaction effects:

● Does the effect of sunlight on plant growth depend on watering

frequency?
● Does the effect of watering frequency on plant growth depend on
the amount of sunlight?

On the other hand, if you wish to add a third factor such that the
two independent variables have three levels each, then you have a
3 x3 factorial design while a 2 x 2 x 2 design has three independent
variables, each with two levels.
Notice that the 2 x 2 factorial design has four treatment groups,
corresponding to the four combinations of the two levels of each
factor. Correspondingly, the 2 x 3 design will have six treatment
groups, and the 2 x 2 x 2 design will have eight treatment groups.
As a rule of thumb, each cell in a factorial design should have a
minimum sample size of 20 (this estimate is derived from Cohen’s
power calculations based on medium effect sizes). So a 2 x 2 x 2
factorial design requires a minimum total sample size of 160
subjects, with at least 20 subjects in each cell. As you can see, the
cost of data collection can increase substantially with more levels or
factors in your factorial design. Sometimes, due to resource
constraints, some cells in such factorial designs may not receive any
treatment at all, which are called incomplete factorial designs. Such
incomplete designs hurt our ability to draw inferences about the
incomplete factors.

Quasi-Experimental Designs

Quasi-experimental designs are almost identical to true

experimental designs, but lack one feature and that is random
assignment. For instance, one entire organization is used as the
treatment group, while a different organization in the same industry
is used as the control group. This lack of random assignment
potentially results in groups that are non-equivalent, such as one
group possessing greater mastery of a certain content than the
other group, say by virtue of having a better management which
introduces the possibility of selection bias .

Quasi-experimental designs are therefore inferior to true

experimental designs in interval validity due to the presence of a
variety of selection related threats such as

11X
 i. selection-maturation threat (the treatment and control groups
maturing at different rates)
ii. history threat (the treatment and control groups being
differentially impact by extraneous or historical events)
iii. selection-regression threat (the treatment and control groups
regressing toward the mean between pre-test and post-test at
different rates)
iv. selection-instrumentation threat (the treatment and control
groups responding differently to the measurement)
v. selection-testing (the treatment and control groups
responding differently to the pre-test)
vi. selection-mortality (the treatment and control groups
demonstrating differential dropout rates).

Given these selection threats, it is generally preferable to avoid

quasi-experimental designs to the greatest extent possible.

Example:
To run a true experiment, you randomly assign half the patients in a
mental health clinic to receive the new treatment. The other half—
the control group—receives the standard course of treatment for
depression. Every few months, patients fill out a sheet describing
their symptoms to see if the new treatment produces significantly
better (or worse) effects than the standard one. However, for ethical
reasons, the manager of the mental health clinic may not give you
permission to randomly assign their patients to treatments. In this
case, you cannot run a true experiment. Instead, you can use a
quasi-experimental design.
If for instance, you discover that a few of the psychotherapists in the
clinic have decided to try out the new therapy, while others who
treat similar patients have chosen to stick with the normal protocol.
You can use these pre-existing groups to study the symptom
progression of the patients treated with the new therapy versus
those receiving the standard course of treatment.
Although the groups were not randomly assigned, if you properly
account for any systematic differences between them, you can be
reasonably confident any differences must arise from the treatment
and not other confounding variables.

Here are several common differences between true and quasi-

experimental designs.

12X
 True experimental design Quasi-experimental
design

Assignment to The researcher randomly Non-random method is

treatment assigns subjects to control used
and treatment groups. to assign subjects to
groups.

Control over The researcher The researcher

treatment usually designs the often does not have
treatment. control over the
treatment, but instead
studies pre-existing
groups that received
different treatments.

Use of control groups Requires the use of control Control groups are not
and treatment groups. required (although they
are commonly used).

13X