ZeePedia

EXPERIMENTAL RESEARCH (Cont.):True Experimental Designs

<< EXPERIMENTAL RESEARCH:The Language of Experiments
EXPERIMENTAL RESEARCH (Cont.):Validity in Experiments >>
img
Research Methods ­STA630
VU
Lesson 34
EXPERIMENTAL RESEARCH (Cont.)
Steps in Conducting an Experiment
Following the basic steps of the research process, experimenters decide on a topic, narrow it into a
testable research problem or question, and then develop a hypothesis with variables. Once a researcher
has the hypothesis, the steps of experimental research are clear. Broadly there are about 12 steps in
conducting an experiment, which are as below:
1. Begin with a straightforward hypothesis that is appropriate for experimental research.
2. Decide on an experimental design that will test the hypothesis within practical limitations. The
researcher decides the number of groups to use, how and when to create treatment conditions,
the number of times to measure the dependent variable, and what the groups of subjects will
experience from beginning till end.
3. Decide how to introduce the treatment or create a situation that induces the independent
variable.
4. Develop a valid and reliable measure of the dependent variable.
5. Set up an experimental setting and conduct a pilot test of the treatment and dependent variable
measures.
6. Locate appropriate subjects or cases.
7. Randomly assign subjects to groups (if random assignment is used in the chosen research
design) and give careful instructions.
8. Gather data for the pretest measure of the dependent variable for all groups (if pretest is used in
thee chosen design).
9. Introduce the treatment to the experimental group only (or to the relevant groups if there are
multiple experimental groups) and monitor all groups.
10. Gather data for posttest measure of the dependent variable.
11. Debrief the subjects by informing them of the true purpose and reasons for the experiment. Ask
subjects what they thought was occurring. Debriefing is crucial when subjects have been
deceived about some aspect of the treatment.
12. Examine data collected and make comparisons between different groups. Where appropriate,
use statistics and graphs to determine whether or not the hypothesis is supported.
Types of Designs
Researchers combine parts of experiment (e.g. pretests, control groups, etc.) together into an
experimental design. For example some designs lack pretests, some do not have control groups, and
others have many experimental groups. Certain widely used standard designs have names.
Classical Experimental Design: All designs are variations of the classical experimental design, which
has random assignment of subjects, a pretest and a posttest, an experimental group, and a control group.
Quasi-Experimental Designs:
One-shot Case Study Design: Also called the one-group posttest-only design, the one-shot case study
design has only one group, a treatment, and a posttest. Because it is only one group, there is no random
assignment. For example, a researcher shows a group of students a horror film, then measures their
attitude with a questionnaire. A weakness of this design is that it is difficult to say for sure that the
treatment caused the dependent variable. If subjects were the same before and after the treatment, the
researcher would not know it.
One Group Pretest-posttest Design: This design has one group, a pretest, a treatment, and a posttest. It
lacks a control group and random assignment. Continuing with the previous example, the researcher
gives a group of students an attitude questionnaire to complete, shows a horror film, then has them
117
img
Research Methods ­STA630
VU
complete the same questionnaire second time. This is an improvement over the one-shot case study
because the researcher measures the dependent variable both before and after the treatment. But it lacks
the control group for comparison. The researcher cannot know whether something other than the
treatment occurred between the pretest and the posttest to cause the outcome.
Two Groups Posttest-only Design: It has two groups, a random assignment of subjects, a posttest, and
a treatment. It has all parts of the classical design except a pretest. Continuing with our previous
example, the researcher forms two groups through randomization process. He shows group a horror film
to one group i.e. the experimental group. The other group is not shown any film. Both groups then
complete the questionnaire. The random assignment reduces the chance that the groups differed before
the treatment, but without a pretest, a researcher cannot be as certain that the groups began the same on
the dependent variable.
True Experimental Designs
Experimental designs, which have at least two groups, a random assignment of subjects to experimental
and control groups, only experimental group is exposed to treatment, both groups record information
before and after the treatment, are known as ex-post facto experimental designs.
Pretest and Posttest Experimental and Control Group Design: Two groups, one control group and the
other experimental group, are formed randomly. Both the groups are exposed to pretest and posttest.
The experimental group is exposed to treatment while the control group is not.  Measuring the
difference between the differences in the post- and pretests of the two groups would give the net effects
of the treatment.
Experimental Group: Pretest (O1)  X
Posttest (O2)
Control Group: Pretest (O3)  -
Posttest (O4)
Randomization used for setting up the group.
[(O2 ­ O1) ­ (O4 ­ O3)] = Treatment effect (could be anywhere between 0 to -1 or +1).
Solomon's Four Group Design: To gain more confidence in internal validity in experimental designs,
it is advisable to set up two experimental groups and two control groups. One experimental group and
one control group can be given the both pretest and the posttest. The other two groups will be given
only the posttest. Here the effects of treatment can be calculated in several different ways as shown in
figure 1:
Figure 1: Solomon's four group design
Group
Pretest
Treatment
Posttest
1. Experimental
O1
X
O2
2. Control
O3
-
O4
3. Experimental
-
X
O5
4. Control
-
-
O6
(O2 ­ O1) = E
(O4 - O3) = E
(O5 ­ O6) = E
(O5 - O3) = E
[(O2 ­O1) ­ (O4 ­ O3)] = E
E = Effect
118
img
Research Methods ­STA630
VU
If all Es are similar, the cause and effect relationship is highly valid.
Interaction Effect
The effect of two variables together is likely to be greater than the individual effect of each put together.
The idea of an interaction effect is familiar, especially in the area of medicine or illness. As an example,
imagine that for a given population of 100 persons, all of the same age and sex, it was found that if all
100 smoked cigarettes the effect would be a lung cancer rate of 20 percent. Assume that for an identical
group of 100 persons who did not smoke but lived in a smoggy environment, 10 percent would get lung
cancer. Now consider a third identical group of 100 persons all of whom smoke and also live in a
smoggy environment. The additive effect of both smoking and smog would be 20 percent plus 10
percent, or a total of 30 percent (30 people) having cancer. However, imagine that an actual medical
survey of the population shows a cancer rate of 37 percent among persons experiencing both smoking
and smog. This extra 7 percent can be computed residually as:
Interaction Effect = Total effect ­ (smoking effect + smog effect) = 37 percent
= 37 percent - (20 percent + 10 percent)
= 37 percent - 30 percent
= 7 percent
In experiments we have the pretests and posttests, in which case we use the same instrument for
measuring the dependent variable, for example racial prejudice as an effect of a movie. In pretest is a
questionnaire in which items forming the prejudice scale are dispersed at random among other items so
that the subject does not know that his or her level of racial prejudice is being measured. Nevertheless,
the measurement of this variable (prejudice) itself, by presenting questions about race relations may
stimulate the subject's thinking and actually cause a change in his or her level of racial prejudice. Any
pretest effect that occurs will be visible as part of extraneous change (change caused by the test
stimulus) in the control group, as the pretest is also presented to the control group. Any change between
the pretest and posttest for measuring the dependent variable in the control group may be attributed to
the sensitization of the subjects with the instrument. In the experimental group of course a movie (an X
variable) was shown due to which we expect a change in the racial prejudice of the subjects. But that is
not all. The subjects in the experimental group were also exposed to the instrument for measuring the
racial prejudice, hence they were also sensitized. Their posttest results include the combined effect of
exposure to a movie and that of sensitization to the instrument. In other words the racial prejudice of
the subjects in the experimental group exhibits the interaction effect of the treatment plus that of
sensitization of the instrument.
In order to calculate the interaction effect in the experiment we shall have two experimental groups and
one control group created by using the randomization process. It may look like this:
Experimental group 1: Pretest (O1)
X
Posttest (O2)
Control group:
Pretest (O3)
-
Posttest (O4)
Why O4 be different from O3? The difference may be due to sensitization. So let us figure it out. Let us
take another experimental group and we do not pretest i.e. no sensitization with the instrument.
Experimental group 2: No pretest
X
Posttest (O5)
Let us work out the results:
(O2- O1) = D
(O4- O3) = D/
(O5 ­ O3)= D// (Since all groups are identical, so we can use the pretest of any of the
Other two groups)
Interaction effect = D ­ [D/ + D//]. Substituting it with our example of lung cancer
37 - [10 + 20] = 37 ­ 30 = 7
There are many other experimental designs like the randomized block design, Latin square design,
natural group design, and factorial design.
119
Table of Contents:
  1. INTRODUCTION, DEFINITION & VALUE OF RESEARCH
  2. SCIENTIFIC METHOD OF RESEARCH & ITS SPECIAL FEATURES
  3. CLASSIFICATION OF RESEARCH:Goals of Exploratory Research
  4. THEORY AND RESEARCH:Concepts, Propositions, Role of Theory
  5. CONCEPTS:Concepts are an Abstraction of Reality, Sources of Concepts
  6. VARIABLES AND TYPES OF VARIABLES:Moderating Variables
  7. HYPOTHESIS TESTING & CHARACTERISTICS:Correlational hypotheses
  8. REVIEW OF LITERATURE:Where to find the Research Literature
  9. CONDUCTING A SYSTEMATIC LITERATURE REVIEW:Write the Review
  10. THEORETICAL FRAMEWORK:Make an inventory of variables
  11. PROBLEM DEFINITION AND RESEARCH PROPOSAL:Problem Definition
  12. THE RESEARCH PROCESS:Broad Problem Area, Theoretical Framework
  13. ETHICAL ISSUES IN RESEARCH:Ethical Treatment of Participants
  14. ETHICAL ISSUES IN RESEARCH (Cont):Debriefing, Rights to Privacy
  15. MEASUREMENT OF CONCEPTS:Conceptualization
  16. MEASUREMENT OF CONCEPTS (CONTINUED):Operationalization
  17. MEASUREMENT OF CONCEPTS (CONTINUED):Scales and Indexes
  18. CRITERIA FOR GOOD MEASUREMENT:Convergent Validity
  19. RESEARCH DESIGN:Purpose of the Study, Steps in Conducting a Survey
  20. SURVEY RESEARCH:CHOOSING A COMMUNICATION MEDIA
  21. INTERCEPT INTERVIEWS IN MALLS AND OTHER HIGH-TRAFFIC AREAS
  22. SELF ADMINISTERED QUESTIONNAIRES (CONTINUED):Interesting Questions
  23. TOOLS FOR DATA COLLECTION:Guidelines for Questionnaire Design
  24. PILOT TESTING OF THE QUESTIONNAIRE:Discovering errors in the instrument
  25. INTERVIEWING:The Role of the Interviewer, Terminating the Interview
  26. SAMPLE AND SAMPLING TERMINOLOGY:Saves Cost, Labor, and Time
  27. PROBABILITY AND NON-PROBABILITY SAMPLING:Convenience Sampling
  28. TYPES OF PROBABILITY SAMPLING:Systematic Random Sample
  29. DATA ANALYSIS:Information, Editing, Editing for Consistency
  30. DATA TRANSFROMATION:Indexes and Scales, Scoring and Score Index
  31. DATA PRESENTATION:Bivariate Tables, Constructing Percentage Tables
  32. THE PARTS OF THE TABLE:Reading a percentage Table
  33. EXPERIMENTAL RESEARCH:The Language of Experiments
  34. EXPERIMENTAL RESEARCH (Cont.):True Experimental Designs
  35. EXPERIMENTAL RESEARCH (Cont.):Validity in Experiments
  36. NON-REACTIVE RESEARCH:Recording and Documentation
  37. USE OF SECONDARY DATA:Advantages, Disadvantages, Secondary Survey Data
  38. OBSERVATION STUDIES/FIELD RESEARCH:Logic of Field Research
  39. OBSERVATION STUDIES (Contd.):Ethical Dilemmas of Field research
  40. HISTORICAL COMPARATIVE RESEARCH:Similarities to Field Research
  41. HISTORICAL-COMPARATIVE RESEARCH (Contd.):Locating Evidence
  42. FOCUS GROUP DISCUSSION:The Purpose of FGD, Formal Focus Groups
  43. FOCUS GROUP DISCUSSION (Contd.):Uses of Focus Group Discussions
  44. REPORT WRITING:Conclusions and recommendations, Appended Parts
  45. REFERENCING:Book by a single author, Edited book, Doctoral Dissertation