Chapter 7:
EXPERIMENTAL
DESIGN
Research Methods for Business
Objectives
Distinguish between causal and correlational analysis.
Explain the difference between lab and field
experiments.
Explain the following terms: nuisance variables,
manipulation, experimental and control groups,
treatment effect, matching, and randomization.
Discuss internal and external validity in experimental
design.
Discuss the seven possible threats to internal validity in
experimental designs.
Describe the different types of experimental designs.
Discuss the solomon four-group design and its
implications for internal validity.
Apply what has been learned to class assignments and
exams.
To establish that variable X causes
variable Y, all three of the following
conditions should be met:
Both X and Y should covary.
X should precede Y.
No other factor should possible cause the
change in the dependent variable Y.
The Lab Experiment
When a cause-and-effect relationship
between an independent and a dependent
variable of interest is to be clearly
established, then all other variables that
might contaminate or confound the
relationship have to be tightly controlled.
Control
When we postulate cause-and-effect
relationships between two variables X and
Y, it is possible that some other factor, say
A, might also influence the dependent
variable Y. in such a case, it will not be
possible to determine the extent to which
Y occurred only because of X, since we do
not know how much of the total variation
of Y was caused by the presence of the
other factor A.
Manipulation of the
Independent Variable
In order to examine the causal effects of
an independent variable on a dependent
variable, certain manipulations need to be
tried. Manipulation simply means that we
create different levels of the independent
variable to assess the impact on the
dependent variable.
Example: how causal relationships are
established by manipulating the
independent variable. We want to test the
effects of lighting on worker production
levels among sewing machine operators.
Controlling the Contaminating
Exogenous OR “Nuisance”
Variables
Matching Groups:
one way of controlling the contaminating or
“nuisance” variables is to match the various
groups by picking the confounding
characteristics and deliberately spreading
them across groups.
Randomization:
another way of controlling the contaminating
variables is to assign members randomly to
the groups.
Different Between Matching &
Randomization
The difference between matching and
randomizations that in the former case
individuals are deliberately and consciously
matched to control the differences among
group members, whereas the latter case we
expect that the process of randomization
would distribute the inequalities among the
groups, based on the laws of normal
distribution. Thus, we need not be
particularly concerned about any known or
unknown confounding factor.
Internal Validity
Internal validity refers to the confidence
we place in the cause-and-effect
relationship. On other words, it addresses
the question, “To what extent does the
research design permit us to say that the
independent variable A causes a change in
the dependent variable B?”
In lab experiments where cause-and-effect
relationships are substantiated, internal
validity can be said to be high.
External Validity OR
Generalizability of Lab
Experiment
To what extent would the results found
in the lab setting be transferable or
generalizable to the actual
organizational or field setting? In the
other words, if we do find a cause-and-
effect relationship after conducting a lab
experiment, can we then confidently say
that the same cause-and-effect
relationship will also hold true in the
organizational setting?
The Field Experiment
A field experiment, as the name implies, is
an experiment done in the natural
environment in which work goes on as
usual, but treatment are given to one or
more groups. Thus in the field experiment,
even though it may not be possible to
control all the nuisance variable because
members cannot be either randomly
assigned to groups, or matched, the
treatment can still be manipulated.
External Validity
External Validity: refers to the extent of
generalizability of the results of a causal
study to other setting, people, or events,
and internal validity refers to the degree
of our confidence in the causal effects
that variable X causes variable Y.
Trade-off Between Internal &
External Validity
There is thus a trade-off between internal
validity. If we want high internal validity,
we should be willing to settle for lower
external validity and vice versa. To ensure
both types of validity, researchers usually
try first to test the causal relationships in a
tightly controlled artificial or lab setting,
and once the relationship in a field
experiment.
Factors Affecting Internal
Validity
Some confounding factors might still be
present that could offer rival explanations as
to what is causing the dependent variable.
These possible confounding factors pose:
History Effects: Certain events or factors that
would have an impact on the independent
variable-dependent variable relationship
might unexpectedly occur while the
experiment is in progress, and this history of
events would confound the cause-and-effect
relationship between the two variables, thus
affecting the internal validity.
Factors Affecting Internal
Validity
Maturation Effects: cause-and-effect
inferences can also be contaminated by the
effects of the passage of time—another
uncontrollable variable. Such contamination
is called maturation effects. The maturation
effects are a function of the process—both
passage of time.
Testing Effects: Frequently, to test the
effects of a treatment, subjects are given
what is called a pretest (say, a short
questionnaire eliciting their felling and
attitudes).
Factors Affecting Internal
Validity
Instrumentation Effects: instrumentation
effects are yet another source of threat to
internal validity. These might arise because
of a change in the measuring instrument
between pretest and posttest, and not
because of the treatment’s differential
impact at the end.
Selection Bias Effects: The threat to internal
validity could also come from improper or
unmatched selection of subjects for the
experimental and control groups.
Factors Affecting Internal
Validity
Statistical Regression: The effects of statistical
regression are brought about when the
members chosen for the experimental group
have extreme scores on the dependent
variable to begin with.
Mortality: Another confounding factor on the
cause-and-effect relationship is the mortality
or attrition of the members in the experimental
or control group or both, as the experiment
progresses. When the group composition
changes over time across the groups,
comparison between the groups becomes
difficult, because those who dropped out of the
experiment may confound the results.
Internal Validity in Case
Studies
If there are several threats to internal
validity even in a tightly controlled lab
experiment, it should become quite clear
why we cannot draw conclusions about
causal relationships from case studies
that describe the events that occurred
during a particular time.
Factors Affecting External
Validity
Whereas internal validity raises
questions about whether it is the
treatment alone or some additional
extraneous factor that causes the
effects, external validity raises issues
about the generalizability of the findings
to other setting.
Type of Experimental Designs
& Internal Validity
• Quasi-Experimental Designs
Some studies expose an experimental group to a
treatment and measure its effects. Such an experimental
design is the weakest of all design, and it does not
measure the true cause-and-effect relationship.
• Pretest & Posttest Experimental Group Design
An experimental group (without a control group) my be given
a pretest, exposed to a treatment, and then given a
posttest to measure the effects of the treatment.
• Posttests Only with Experimental & Control Groups
Some experimental design are set up with an experimental
and a control group, the former alone being exposed to a
treatment and not the latter. The effects of the treatment
are studied by assessing the difference in the outcomes—
that is, the posttest scores of the experimental and control
groups.
Type of Experimental Designs
& Internal Validity
• The Experimental Designs
Experimental designs, which include both the
treatment and control groups and record
information both before and after the experimental
group is exposed to the treatment.
– Pretest & Posttest Experimental & Control
Group Designs:
Two groups–one experimental and the other control—are
both exposed to the pretest and the posttest.
– Solomon Four-Group Design
To gain more confidence in internal validity in
experimental design, it is advisable to set up two
experimental groups and two control groups for the
experiment.
Type of Experimental Designs
& Internal Validity
• Solomon Four-Group Design & Threats to Internal
Validity:
• Lets us examine how the threats t internal validity are
taken care of in the Solomon four-group design. It is
important to note that subjects have been randomly
selected and randomly assigned to groups. This removes
the statistical regression and selection bases.
• Double-Blind Studies:
When extreme care and rigor are needed in experimental
design as in the case of discovery of new medicines that
could impact on human lives, blind studies are
conducted to avoid any bias that might creep in.
• Ex Post Facto Designs:
Cause-and-effect relationships are sometimes established
what is called the ex post facto design. Here, there is no
manipulation of the independent variable in the lab or
field setting, but subjects who have already been
exposed to a stimulus and those not so exposed are
Simulation
An alternative to lab and field
experimentation currently being used in
business research is simulation.
Simulation uses a model-building
technique to determine the effects of
changes, and computer-based
simulations are becoming popular in
business research
Experimental Design
Experimental Design
Experimental Design
Experimental Design
.
Experimental Design
Experimental Design
Experimental Design
Experimental Design
