THE ROLE OF RESEARCH IN CLINICAL PSYCHOLOGY:LIMITATION

<< ETHICAL STANDARDS FOR CLINICAL PSYCHOLOGISTS:PREAMBLE

THE RESEARCH PROCESS:GENERATING HYPOTHESES, RESEARCH METHODS >>

Clinical Psychology (PSY401)

LESSON 10

THE ROLE OF RESEARCH IN CLINICAL PSYCHOLOGY

Research lays a foundation of knowledge for understanding the phenomena of interest to clinical

psychologists, including psychopathology, mental health, and the relationship between psychological

factors and physical disease. Research also provides a body of evidence to guide clinical practice,

including empirically validated methods to assess people and their problems and empirically supported

methods of prevention and treatment. Psychological tests and other assessment methods used in clinical

practice should be based on research that has established their reliability and validity. Research findings

should also identify those interventions that have been shown to be more effective than no treatment or

alternative forms of treatment.

Just as research informs clinical practice, clinical experiences provide a source of ideas and hypotheses

for research. Research also provides ideas for new directions and applications for the field of clinical

psychology, including links between clinical psychology and research in other behavioral, biological,

and social sciences.

Because of the wide range of questions that confront researchers in clinical psychology, a variety of

methods are used in research in this field. Research designs used by clinical psychologists range from

single-case designs that study one individual at a time to large -scale, multisided studies involving

hundreds or even thousands of participants. Clinical psychologists conduct research in many different

settings including experimentally controlled laboratories as well as naturalistic settings such as

hospitals, clinics, schools, and the community. Clinical researchers utilize various methods of data

analysis, ranging from complex multivariate statistics used with large samples to non-statistical methods

in single-case studies. The methods that are chosen by researchers shape the types of questions that are

asked; reflect the hypotheses that are being tested; and influence the interpretation of findings.

RESEARCH DESIGNS

There are four basic types of research designs from which to choose: descriptive designs, co-relational

designs, experimental designs, and single-case designs.

DESCRIPTIVE RESEARCH DESIGNS

Descriptive research designs are used in clinical psychology to report on the prevalence or incidence of

a human characteristic or problem in the population. The goal of this type of research is to describe a

particular phenomenon without trying to predict or explain when or why it occurs. Descriptive studies

are often an important first step in research on a particular problem or disorder, because they allow the

researchers to define the scope of a problem in the population.

Researchers involved in descriptive research are primarily concerned with accurate measurement of the

problem and with the representativeness of the sample that they include in their study. If participation in

a study is biased toward a particular segment of the population, the results of the study could

misrepresent the prevalence of a problem as higher as or lower than it actually is in the population as a

whole. This type of research does not attempt to predict or understand the causes of a problem, however,

because other variables that might be hypothesized to be causes or correlates of the problem typically

are not measured.

A descriptive approach is used most frequently in epidemiological studies in which researchers try to

identify the prevalence of different forms of psychopathology. Epidemiological research is designed to

establish the number, or prevalence, of disorders in a population at a particular point in time as well as

the onset, or incidence, of new cases during a specified period of time (e.g., the past year).

Epidemiology has a long history in the field of public health, where studies have been conducted to

Clinical Psychology (PSY401)

understand the prevalence and incidence of physical disease. Epidemiological methods have been used

more recently to estimate the extent of psychiatric disorders within populations or countries.

CORRELATIONAL RESEARCH DESIGNS

Correlational research designs are used to determine the degree to which there is an association between

two or more variables. In these studies, the researcher wants to determine whether, and to what extent,

different variables are related to each other. This involves measuring each variable and then using

statistics to determine how changes in one variable are related to changes in another.

THE MODEL UNDERLYING CORRELATIONAL RESEARCH METHODS

Correlational research designs are founded on the assumption that reality is best described as a network

of interacting and mutually-causal relationships. Everything affects--and is affected by--everything else.

This web of relationships is not linear, as in experimental research. Thus, the dynamics of a system--

how each part of the whole system affects each other part--is more important than causality. As a rule,

correlational designs do not indicate causality.

A simple (or bivariate) correlation represents the relationship that is observed between two variables in

a sample of individuals. The same two variables are assessed for each person in the sample and a

correlation coefficient is calculated to provide a numerical representation of the magnitude and direction

of this association. In other words, the relationship has a "degree" and a "direction".

The degree of relationship (how closely they are related) is usually expressed as a number between -1

and +1, the so-called correlation coefficient. This coefficient can range from positive 1.00 (one variable

increases in value at exactly the same rate as the other variable increases in value), to zero (no

association or relationship between the variables), to negative 1.00 (one variable decreases in value at

exactly the same rate as the other variable increases in value). As the correlation coefficient moves

toward either -1 or +1, the relationship gets stronger until there is a "perfect correlation" at either

extreme.

The direction of the relationship is indicated by the "-" and "+" signs. A negative correlation means

that as scores on one variable rise, scores on the other decrease. A positive correlation indicates that the

scores move together, both increasing or both decreasing.

A student's grade and the amount of studying done, for example, are generally positively correlated,

meaning that the more study done, the higher the student's grade will be. Stress and health, on the other

hand, are generally negatively correlated, meaning that the more stresses a person will experience, and

the lower his /her health status will be.

LIMITATION

The researcher cannot make conclusions about cause and effect; even a strong correlation does not mean

that changes in one variable cause changes in another (correlation can be due to a third variable).

EXPERIMENTAL RESEARCH DESIGNS

Experimental research designs involve the control or manipulation of one or more variables (the

independent variables) to determine their effect on a second variable or set of variables (the dependent

variables). Because the independent variable is under the control of the researcher, it is possible to

determine if changes in this factor cause changes to occur in the dependent variable.

Experimental designs are used in two primary ways in clinical psychology research. First, researchers

conduct controlled experiments to study the possible causal relationship between two (or more)

Clinical Psychology (PSY401)

variables. Experimental studies of psychopathology are important to an understanding of the possible

causes of psychological disorders. However, ethical concerns obviously prohibit any research that

actually causes a psychological disorder. Rather, experimental studies are conducted on analogues

(representations) of psychopathology, or they are conducted with patients already suffering from a type

of psychopathology to learn about factors that are relevant to an understanding of the disorder.

Experimental research conducted with animals can have important implications for an understanding of

psychopathology in humans, because research ethics allow, for some what different procedures to be

used with animals.

The second major area in which experimental designs are used in clinical psychology is in studies that

are designed to evaluate the effectiveness of an intervention to prevent or treat a problem and in which

participants are randomly assigned to a group that receives the intervention or to an alternative condition

(a control group).

THE MODEL UNDERLYING EXPERIMENTAL RESEARCH METHODS

Experimental research designs are founded on the assumption that the world works according to causal

laws. These laws are essentially linear, though complicated and interactive. The goal of experimental

research is to establish these cause-and-effect laws by isolating causal variables.

A softer view of the philosophical assumptions behind experimental designs is that SOMETIMES and

IN SOME WAYS, the world works according to causal laws. Such cause-and-effect relationships may

not be a final view of reality, but demonstrating cause and effect is useful in some circumstances.

Both of these views agree that some (if not all) important psychological questions are questions about

what causes what. Experimental research designs are the tools to use for these questions.

ESSENTIAL CHARACTERISTICS OF AN EXPERIMENT

To be "experimental", a study must meet two conditions: having an experimental independent variable

with experimental control and having random assignment. These are described below:

In an experimental study, there is at least one experimental independent variable, and there is

experimental control.

(a) EXPERIMENTAL/INDEPENDENT VARIABLE

The researcher systematically alters/manipulates one variable (independent variable, or IV) to see if the

manipulation causes a change in some aspect of behavior (dependent variable, or DV). There must be at

least one manipulated variable for a study to be an experiment. Some examples include:

The effect of training program type (IV) on cashiers' job performance (DV)

The effect of servers' appearance (IV) on size of tip (DV)

(b) EXPERIMENTAL CONTROL

All factors other than the IV that could affect the DV must be held constant. This means that you avoid

confounding variables, such as when the experimenter affects the subjects' behavior unintentionally. If

there is no such control, the study is not an experiment.

2. In an experimental study, there is random assignment of subjects to groups (conditions). In an

experiment, subjects must be randomly assigned to experimental conditions, meaning that all subjects

have an equal chance of being exposed to each condition. In the examples given above:

Clinical Psychology (PSY401)

Newly hired cashiers are randomly assigned to one of 3 training programs

Servers in a restaurant are randomly assigned to one of 2 groups--the first group dresses in new

uniforms and the second group dresses in dirty uniforms

Random assignment to groups in treatment studies makes it more likely that the groups are equivalent

on all the important variables that relate to the possible effects of treatment. If the groups are identical

except for their exposure or lack of exposure to the treatment, any differences between the groups after

the completion of the treatment are inferred to have been caused directly by the treatment.

The GOAL OF EXPERIMENTAL RESEARCH METHODS is to establish cause-and-effect

relationships between variables.

We hypothesize that the Independent Variable caused the changes in the Dependent Variable. However,

these changes or effects may have been caused by many other factors or Alternative Hypotheses.

The PURPOSE, therefore, of experimental designs is to eliminate alternative hypotheses. If we can

successfully eliminate all alternative hypotheses, we can argue--by a process of elimination--that the

Independent Variable is the cause.

Good experimental designs are those which eliminate more alternative hypotheses.

FOR EXAMPLE: Say I am testing whether a new form of psychotherapy is successful at improving

mental health. I hypothesize that this psychotherapy is the cause of improved mental health in the

research participants.

I will use an experimental design to eliminate all (or as many as possible) alternative hypotheses. If I

can eliminate alternative explanations, I will be able to make the case that the psychotherapy was the

cause of the improvements in the research participants.

TYPES OF VARIABLES

1. INDEPENDENT VARIABLE (IV): IV has levels, conditions, or treatments. Experimenter may

manipulate conditions or measure and assign subjects to conditions; supposed to be the cause. In the

example, it is the psychotherapy.

2. DEPENDENT VARIABLE (DV): measured by the experimenter; the Effect or result. In the

example, it is the mental health of the participants.

3. CONTROL VARIABLES: held constant by the experimenter to eliminate them as potential causes.

For instance, if I use only research participants who have been problems with anxiety or depression, this

diagnosis would be a control variable.

4. RANDOM VARIABLES: allowed to vary freely to eliminate them as potential causes. Many other

characteristics of the research participants, as long as they really do vary freely, are also random

variables. Examples might include age, personality type, or career goals.

5. CONFOUNDING VARIABLES: vary systematically with the independent variable; may also be a

cause. Good experimental designs eliminate them.

Say I divide the research participants into two groups, one of which gets the new psychotherapy (the

experimental group) and one of which does not (the control group). If there is some systematic

difference between these two groups, it will not be a fair test.

Clinical Psychology (PSY401)

If those in the psychotherapy group know they are getting a new treatment and therefore expect to get

better while those in the control group know they are not getting any treatment and expect to get worse,

the expectations will be a confounding variable. If the experimental group does improve, we will not

know whether it was because of the psychotherapy itself (the Independent Variable) or because of the

participants' expectations (a Confounding Variable).

CONCEPTS IN EXPERIMENTAL RESEARCH

1. RELIABILITY

Are the results of the experiment repeatable? If the experiment were done the same way again, would it

produce the same results?

Reliability is a requirement before the validity of the experiment can be established. It refers to the

consistency of the results of an experiment i.e. if we get the same results again and again by repeating

an experiment, we can say that the results of this experiment are reliable.

2. INTERNAL VALIDITY

Internal validity refers to the accuracy or truth-value of an experiment (how accurately the experiment

measures the variables that it was designed to measure). Internal validity also indicates the extent to

which the experimenter is sure about the results i.e. did the independent variable cause the effects in the

dependent variable?

In experimental research, this usually means eliminating alternative hypotheses.

In the example evaluating a new psychotherapy, the issue of internal validity is whether the

psychotherapy really was the causal factor in improving participants' mental health.

3. EXTERNAL VALIDITY

External validity of an experiment refers to its generalizability i.e. to what extent can the results be

applied in another setting or to another population of research participants.

HYPOTHESES

Experimental research methods revolve around hypotheses, educated guesses. We typically start with a

hypothesis about how the results will turn out, i.e., that there is an effect and it is due to the independent

variable. This first hypothesis is the research hypothesis.

Then we hold the possibility that there is no effect of the independent variable on the dependent variable

or that the differences observed are due to chance only. This second hypothesis is the null hypothesis.

The first step in experimental research, then, is ruling out chance. Put another way, we set up an

experimental design that will allow us to reject the null hypothesis. If we can confidently reject the null

hypothesis, then we gain confidence in the research hypothesis.

At this point, another group of hypotheses comes into play, the alternative hypotheses. If there is an

effect beyond chance, it may be due to the independent variable or it may be due to a number of other

factors, so-called extraneous variables or confounding variables. Again, we use experimental designs to

allow us to eliminate alternative hypotheses.

TYPES OF HYPOTHESES

1. Research hypothesis states that results are due to the IV.

Clinical Psychology (PSY401)

In the example of a new form of psychotherapy, the research hypothesis is that the new form of

psychotherapy is better than either no therapy or conventional therapies.

2. Null hypothesis states that differences are due to chance or that there are no differences between

treatments (used in statistical analysis).

In the example, the null hypothesis is that the new form of psychotherapy is no better than either no

therapy or conventional therapies.

3. Alternative hypotheses suggest that results are due to factors other than IV. These factors, rather

than the independent variable, may cause the improvements.

Next is a list of alternative hypotheses.

ALTERNATIVE HYPOTHESES

1. Subject effect or selection effect: results are due to systematic differences in research participants

("subjects") assigned to different conditions or treatments.

Example: If the research participants who receive the new form of psychotherapy are different from

those in a control group, a selection effect would occur. One group could be healthier, more motivated,

or more experienced with psychotherapy.

A problem in some research is letting people choose to be part of a program or treatment and using

others who did not choose to be part of the program as a control group. Such "self-selected" groups are

usually different from groups made up of people who do not choose to be in a treatment group.

Common solution: Matching or random assignment to groups

2. History effect: results are due to events outside the experiment.

Example: This could occur if there is one group of research participants who are being measured at

several points in time. Some event that is not part of the research, say something traumatic like a natural

disaster, which occurs at the same time as the treatment could affect the results.

Common solution: A control group which will be exposed to the same history but not the new form of

psychotherapy.

3.Maturation effect: results are due to changes within subjects over time, e.g., growth, warm-up,

fatigue, learning to learn. This is a problem in research that measures a dependent variable over a period

of time and especially in research with repeated exposures to the independent variable.

Example: If there is one group of research participants, their mental health may improve over time

without the new form of psychotherapy.

Common solution: A control group which is measured over the same period of time but does not receive

the new psychotherapy.

4. Experimenter expectancy effect or Experimenter bias: results are due to the experimenter's actions

or expectations. A number of studies have shown that researchers tend to find the results they are

looking for, a kind of self-fulfilling prophecy. The causes for this result range from overt cheating to

very subtle influences on data collection and interactions with research participants. Experimenters are

not always aware of the extent of these influences.

Clinical Psychology (PSY401)

Example: If the researcher is the one to assess the research participants' mental health (the dependent

variable), he or she may distort the assessments in the direction of the research hypothesis. Other, more

subtle forms of influence may also occur.

Common solution: Use independent judges or more objective measurements of the dependent variable.

5. Demand characteristics or Hawthorne effect: results are due to subjects' expectations of desired

behavior in the research setting or the social psychology of the experiment.

This is called "demand" because participants may perceive a demand to behave or report on themselves

in a certain way. It is called the Hawthorne Effect after a famous series of experiments at a

manufacturing plant in Hawthorne, Ohio. In those studies, researchers selected a group of factory

workers and changed various conditions such as lighting to see what would increase performance. They

found that any change increased performance, suggesting that research participants were responding to

the general expectation that they would perform better and to the social dynamics of being observed

closely.

Example: The researcher communicates his or her expectations to the research participants which in turn

influences their responses. If the researcher is measuring depression, research participants may report

less depression regardless of their feelings because they think that is what is expected of them.

Common solution: Blind and double-blind designs help avoid these problems. Also, using a control

group which is measured the same way (thus getting some of the same influences) without the

treatment.

6. Testing effect or reactivity: results are due to the data gathering procedures, e.g., being influenced

by the test or learning from one test administration to the next.

Example: Measuring the participants' mental health could get them thinking about their lives, thus

improving them. Improvements would then be due to the data gathering, not the therapy itself.

Common solution: Use a control group which is also measured, but without the therapy or with an

alternative form of therapy.

7. Regression artifact or regression-to-the-mean: results are due to extreme scores moving toward the

mean over time.

Example: If a group is made up of those with the worst mental health scores (say, the most anxious or

the most depressed), over time they are likely to improve without therapy. This may be mistakenly

attributed to the therapy.

Common solution: Use a control group which has similar characteristics (mental health scores) but

which does not receive the new therapy.

8. Instrumentation: results are due to an aberration in measuring tools, either mechanical instrument or

test.

Example: The dependent variable (participants' mental health) may be measured by a poor test.

Common solution: Select or develop a better measure.

9. Halo effect: the researcher's expectations about certain subjects based on some subject

characteristics. E.g., an outgoing, sociable subject is rated as being more intelligent or having higher

values.

Clinical Psychology (PSY401)

Example: Judges rating the mental health of the participants (the dependent variable) may ascribe better

mental health based on other characteristics.

Common solutions: Random assignment, blind judges, more objective measures.

10. Attrition or mortality effect: When subjects drop out of an experiment, it can bias the results. This

is especially true when more subjects drop out of one treatment condition than another. The study is no

longer a fair test. This leads to a kind of subject effect because the subjects in the different groups are no

longer equivalent.

Example: Say the study consists of 3 groups: the new psychotherapy group, a conventional therapy

group, and a no-therapy group. If more research participants in the new therapy control group drop out

of the study, it may be because the new therapy was not appropriate for them. This leaves only those

who benefited most, making the therapy look better than it really is.

Common solution: There is no way to force research participants to stay in the study, but if attrition

looks like a problem, find out why participants dropped out. This can sometimes give important clues

about the study.

11. Other non-specific factors and alternative hypotheses that may arise in a particular experiment.

For instance, in psychotherapy research, the specific intervention itself may not cause the benefits.

Rather, the therapeutic relationship may lead to benefits. A therapy that allows for more and better

contact between therapist and client will look better, but the benefits are not because of the therapy

itself. The independent variable, the new therapy, is not causing the benefits. Instead, the relationship

factor which is confounded with the independent variable is causing the effects.

Solutions are specific to the research study and the particular alternative hypothesis.

A COMMENT

With so many ways to go wrong, it may seem from this list that all research is hopelessly flawed. In a

sense, this is accurate. There is no such thing as perfection in an experimental design. However,

perfection is not the best standard to use.

It is suggested that we look for studies that are good enough. Even though there are always ways to

refine and extend any study, there are many experiments that are good enough to base strong

conclusions on.

TYPES OF EXPERIMENTAL DESIGNS

TRUE EXPERIMENTAL DESIGNS

These designs attempt to eliminate most alternative hypotheses, especially those related to time (history,

maturation, and regression) and those related to make-up of the groups (selection effects). Such control

may be at the expense of making the situation too artificial.

A. RANDOMIZED GROUPS DESIGN OR BETWEEN-GROUPS DESIGN

Each research participant is randomly assigned to one group and gets only one level of the independent

variable. There may be pre-tests and post-tests or only post-tests. This design can eliminate selection,

history, and maturation effects.

B. REPEATED MEASURE DESIGN or WITHIN-SUBJECT DESIGN

Clinical Psychology (PSY401)

Each research participant gets all levels of the IV. Treatment orders must be counterbalanced to

eliminate order effects.

C. MIXED MODEL DESIGNS OR COMPLEX DESIGNS

These designs combine randomized groups and repeated measures designs. For instance, there may be

two IVs, one measured between groups and one measured within groups.

SINGLE-SUBJECT DESIGNS

Single-subject designs, or so-called "N=1" designs, are used most often in clinical psychology situations

with behavior modification. They have also been used in basic research on experimental analysis of

behavior using behaviorist model. Note that this is experimental research in a controlled setting with a

single independent variable; it is not case study research.

In many clinical situations, it is not possible or desirable to gather large groups of subjects. Here you

may choose a single-subject design. It can provide strong internal validity, but typically suffers from

low external validity.

In each design, a series of regular and planned observation is taken over a period of time. Observations

are divided into sessions of baseline and treatment conditions.

I. ABA OR REVERSAL DESIGN

A number of observations with no treatment (the A or baseline sessions) are followed by a number of

observations with treatment (B). If the treatment is successful, there should be improvement on the DV

in the B sessions. To show that the improvement is the effect of the IV and not maturation or history,

another no-treatment or A session is given. If the improvements reverse, the research hypothesis is

supported.

In the example, we could observe a client each day or a week. Then we would introduce the new

therapy for two weeks and see if there is improvement. If there is, we could take away the therapy and

see if the improvement goes away. If they do, we can be confident that the therapy works.

II. ABAB DESIGN

This is just like the ABA Design, only another series of B or treatment sessions is added. For ethical

reasons, it is often desirable to leave subjects with the advantage of a successful treatment. The ABAB

design does this. It also provides a replication of the AB comparison.

Although single-case designs were originally used typically to study a small number of very discrete

behaviors, they are now used to study increasingly complex patterns of behavior.

For example, in 1997, a group of researchers used a single-case design to evaluate the effects of family-

based behavioral treatment for a child with severe disabilities and severe behavior problems. This study

focused on a program to change self-injurious, aggressive, and destructive behaviors in a I 4-year-old

girl. The researchers used a multiple baseline approach in which they implemented several different

interventions through the parents' behavior (e.g., changing the ways that the parents responded to the

child's self-injuries) with their daughter in different settings (e.g., dinner at home, in restaurants, in the

grocery store).

The frequency of the girl's problem behaviors was assessed during the baseline condition, during the

training period in which the parents were taught to respond differently to her behavior, and during

Clinical Psychology (PSY401)

follow-up. The rates of the girl's problem behaviors decreased in each of the different settings following

the program to change the ways that her parents responded to these behaviors.

CONCLUSION TO EXPERIMENTAL DESIGNS

Which design is best? There is sentiment among some researchers that experimental research designs

are superior to descriptive or correlational approaches because only experimental designs can be used to

determine true causal, relationships. This view is a misrepresentation of the broad scope of the research

process, however, because each type of research design is useful for addressing some questions and

hypotheses and not others.

Clinical psychologists are often interested in observing things as they occur in the natural environment

and descriptive and correlational designs are best suited for this purpose. In other instances, clinical

psychologists are interested in determining cause-and-effect relations among variables or in determining

the effects of a specific form of treatment, goals that are addressed only with experimental designs.

Furthermore, ethical constraints often limit the types of research designs that can be used. Researchers

cannot ethically cause significant distress or psychopathology to occur in participants in human

research. The first priority of any researcher is the welfare of the individuals who participate in the

research, and any risks that are involved must be within reasonable limits and must be justified by the

potential benefits of the research. As a result, much of the research on the causes and course of

psychopathology must rely on descriptive and correlational designs combined with analogue or animal

research that uses experimental designs to test similar hypotheses.

Table of Contents: