THE ASSESSMENT OF INTELLIGENCE:RELIABILTY AND VALIDITY, CATTELL’S THEORY

<< THE CLINICAL INTERVIEW:The intake / admission interview, Structured interview

INTELLIGENCE TESTS:PURPOSE, COMMON PROCEDURES, PURPOSE >>

Clinical Psychology (PSY401)

LESSON 17

THE ASSESSMENT OF INTELLIGENCE

The history of clinical psychology is inextricably tied to the assessment of intelligence. Without the

success in this and related assessment enterprises, there might not have been a field of clinical

psychology. As the years passed, however; clinicians became increasingly interested in the more

"glamorous" aspects of the profession, such as therapy. Assessment began to take a back seat, and

technicians started to become the assessors, as they had been prior to World War II. However; all this is

beginning to change. Not only is the value of assessment being rediscovered, but intelligence tests in

particular remain prominent in the clinician's arsenal of assessment devices.

THE CONCEPT OF INTELLIGENCE

The concept of intelligence has presented clinical psychology with one of its greatest dilemmas. On the

one hand, psychologists have been pressured for almost 100 years to quantify individual differences in

intellectual functioning. On the other hand, intelligence has remained one of the most difficult and

controversial psychological construct to define and measure. The impetus to define and quantify

intelligence comes from both practical and scientific forces. The practical significance lies in the

potential use of measures of intelligence in predicting academic and work potential and achievement.

From a scientific perspective, precise understanding of how to measure intelligence can contribute to

our comprehension of an important aspect of human behavior and functioning.

Two issues that have plagued psychologists from the beginning are still not resolved. First, exactly what

is meant by the term intelligence? Second, how do we develop valid instruments for measuring it? We

will address both questions. But first we need to review the psychometric concepts of reliability and

validity.

RELIABILTY AND VALIDITY

RELIABILITY

With regard to psychological tests, reliability refers to the consistency with which individuals respond to

test stimuli. There are several ways of evaluating reliability.

First, there is test-retest reliability-the extent to which an individual makes similar responses to the same

test stimuli on repeated occasions. If each time we test a person we get different responses, the test data

may not be very useful. In some instances, clients may remember on the second occasion their responses

from the first time. Or they may develop a kind of "test-wiseness" from the first test that influences

their scores the second time around. In still other cases, clients may rehearse between testing occasions

or show practice effects. For all these reasons, another gauge of reliability is sometimes used-

equivalent-forms reliability. Here, equivalent or parallel forms of a test are developed to avoid the

preceding problems.

Sometimes it is too expensive (in time or money) to develop an equivalent form or it is difficult or

impossible to be sure the forms are really equivalent. Under such circumstances, or when retesting is not

practical, assessing split-half reliability is a possibility. This means that a test is divided into halves

(usually odd-numbered items versus even-numbered items), and participants' scores on the two halves

are compared. Split-half reliability also serves as one possible index of a test's internal consistency: Do

the items on the test appear to be measuring the same thing? That is, are the items highly correlated with

each other? The preferred method of assessing internal consistency reliability involves computing the

aver-age of all possible split-half correlations for a given test.

134

Clinical Psychology (PSY401)

Another aspect of reliability, inter-rater or inter-judge reliability is the index of the degree of agreement

between two or more raters or judgers as to the level of a trait that is present or the presence/absence of

a features or diagnosis.

VALIDITY

In general, validity refers to the extent to which as assessment technique measures what it is supposed to

measure. There are several forms of validity.

Content Validity indicates the degree to which a group of test items actually covers the various aspects

of the variable under study.

Predictive Validity is demonstrated when test scores accurately predict some behavior or event in the

future.

Concurrent Validity involves relating today's test scores to a concurrent criterion.

Finally, Construct Validity is shown when test scores relate to other measures or behaviors in a logical,

theoretically expected fashion.

DEFINITION OF INTELLIGENCE

Intelligence is a hypothetical construct; that is, intelligence is a concept that exists only in the way that

psychologists and the public chose to define it. You cannot touch intelligence nor can you directly

observe it. You can only observe the consequences of intelligence as they are reflected in the behavior

and performance of individual.

There is no universally accepted definition of intelligence. However, over the years, most have fallen

into one of three classes:

Definitions that emphasize adjustment or adaptation to the environment-adaptability to new

situations, the capacity to deal with a range of situations.

Definitions that focus on the ability to learn-on educability in the broad sense of the terms.

Definitions that emphasize abstract thinking the ability to use a wide range of symbols and

concepts, the ability to use both verbal and numerical symbols.

To illustrate a little of the long-standing diversity of definitions, consider the following examples:

Intelligence is the aggregate or global capacity of the individual to act purpose-fully, to think rationally,

and to deal effectively with his environment. (Wechsler, 1939)

As a concept, intelligence refers to the whole class of cognitive behaviors which reflects an individual's

capacity to solve problems with insight, to adapt himself to new situations, to think abstractly, and to

profit from his experience. (Robinson & Robinson, 1965)

Intelligence is expressed in terms of adaptive, goal-directed behavior the subset of such behavior that is

labeled "intelligent" seems to be determined in large part by cultural or societal norms. (Sternberg &

Salter, 1982)

Intelligence is a very general mental capability that, among other things, involves the ability to reason,

plan, solve problems, and think has very few traits in common-they resemble the prototype along

135

Clinical Psychology (PSY401)

different dimensions. Thus, there is no such thing as chariness - resemblance is an external fact and not

an internal essence. There can be no process-based definition of intelligence, because it is not a unitary

quality. It is a resemblance between two individuals, one real and the other prototypical. (Spearman)

THEORIES OF INTELLIGENCE

There have been many theoretical approaches to the understanding of intelligence. These include

psychometric theories, developmental theories, neuropsychological theories, and information-processing

theories. We present only a brief overview of several leading theories here.

FACTOR ANALYTICAL APPROACHES

Spearman (1927), the father of factor analysis, posited the existence of a g factor (general intelligence)

and S factors (specific intelligence). The elements that tests have in common are represented by g,

whereas the elements unique to a given test are S factors. Basically, however, Spearman's message,

buttressed by factor analytic evidence, was that intelligence is a broad, generalized entity.

A number of individuals took issue with Spearman's contentions, including E. L. Thorndike and L. L.

Thurstone. For example, Thurstone (1938) presented evidence (based on a factor analysis of 57 separate

tests that had been administered to 240 participants) for a series of "group" factors rather than the

almighty g factor. Ultimately, Thurstone described seven group factors which he labeled number, word

fluency, verbal meaning, perceptual speed, space, reasoning, and memory.

CATTELL'S THEORY

The work of R. B. Cattell (1987) emphasizes the centrality of g. At the same time, Cattell has offered a

tentative list of 17 primary ability concepts. He has described two important second-order factors that

seem to represent a partitioning of Spearman's & into two components: fluid ability (the person's geneti-

cally based intellectual capacity) and crystallized ability (the capacities, tapped by the usual

standardized intelligence test that can be attributed to culture-based learning). Essentially, Cattell's

approach might be described as a hierarchical model of intelligence.

GUILFORD'S THEORY: The views of Guilford (1967) were quite different from those of Cattell,

Spearman, Thurstone, and most other psychometricians. Guilford proposed a Structure of the Intellect (S

01) model and then used a variety of statistical and factor analytic techniques to test it. Whereas other

psychometric approaches generally attempted to infer a model, Guilford used the model as a guide in

generating data.

Guilford reasoned that the components of intelligence could be organized into three dimensions:

operations, contents, and products. The operations are cognition, memory, divergent production

(constructing logical alternatives), Convergent production (constructing logic-tight arguments), and

evaluation. The content dimension involves the areas of information in which the operations are

performed: figural, symbolic, semantic, and behavioral. Finally, when a particular mental operation is

applied to a specific type of content, there are six possible products: units, classes, systems, relations,

transformations, and implications. If we contemplate all possible combinations, we arrive at 120

separate intellectual abilities. Perhaps the most widely held reservation about Guilford's approach is that

it is taxonomy or classification rather than a theory.

RECENT DEVELOPMENTS:Traditionally, intelligence tests have been constructed to assess what

we know or can do. Recent approaches, however, have begun to take on a highly cognitive or in-

formation-processing look. For example, some researchers try to describe a person's moment-by-

moment attempts to solve a problem-from the moment a stimulus registers to the person's verbal or

motor response.. This is a more dynamic view of intelligence than the older theories of mental

components.

136

Clinical Psychology (PSY401)

Some of these researchers have focused on speed of information processing and others on strategies of

processing. A number of levels of processing have been studied, including speed of processing, speed in

making choices in response to stimuli, and speed with which individuals can extract various aspects of

language from their long-term memory. But many problems and questions remain (Gardner, 1983). Is

there a central processing mechanism for information? How do the processing elements change as the

person develops? Are there general problem-solving skills or merely skills specific to certain ability

areas? Perhaps time will tell.

Gardner (1983) has described a theory of multiple intelligences. Human intellectual competence

involves a set of problem-solving skills that enable the person to resolve problems or difficulties.

Sometimes this results in the potential for acquiring new information. Gardner suggests that there is a

family of six intelligences: linguistic, musical, logical-mathematical, spatial, bodily-kinesthetic, and

personal. For example, the personal refers both to access to one's own feeling life and to the ability to

notice and make distinctions among other individuals. A major criticism of Gardner's theory is that

some of his proposed "intelligences" may be better conceptualized as "talents" than as forms of

intelligence. Nevertheless, Gardner's views have attracted a great deal of attention from psychologists

and educators alike.

To cite another example of a theory of multiple forms of intelligence, Sternberg (1985, 1991) has

proposed a triarchic theory of intelligence. He maintains that people function on the basis of three

aspects of intelligence: componential, experiential, and contextual.

This approach deemphasizes speed and accuracy of performance. Instead, the emphasis is on planning

responses and monitoring them. The componential aspect refers to analytical thinking; high scores

would characterize the person who is a good test-taker. The experiential aspect relates to creative

thinking and characterizes the person who can take separate elements of experience and combine them

insightfully. Finally, the contextual aspect is seen in the person who is "street smart"-one who knows

how to play the game and can successfully manipulate the environment.

According to Sternberg, a person's performance is governed by these three aspects of intelligence. Other

investigators are particularly interested in social competence as an aspect of intelligence (Sternberg &

Wagner, 1986). However, whether all the foregoing can account for individual differences or is just a

theory of cognition is debatable.

Although Spearman, Thurstone, and others may seem to have given way to Cattell, Guilford, Gardner,

or Sternberg, clinicians' day-to-day use of tests suggests that they have not really outgrown the g factor

of Spearman or the group factors of Thurstone. The whole notion of a single IQ score that can represent

the individual's intelligence strongly implies that we are trying to discover how much g the person has.

At the same time, however, most current intelligence tests are composed of subtests, so that the total IQ

represents some average of subtest scores.

This implies that, to some extent at least, we have also accepted Turnstone's group factors. We seem to

want to identify and quantify how much intelligence the person has, yet we cannot escape the belief that

intelligence is somehow patterned-that two people may have the same overall IQ score and still differ in

specific abilities. Thus, it would appear that practicing clinicians think more in line with Spearman or

Thurstone and are as yet little affected by the recent information processing developments.

HISTORY OF INTELLIGENCE TESTING

Two important historical developments in the latter half of the nineteenth century greatly influenced the

ultimate introduction of measures of intelligence (Thorndike, 1997). First, compulsory education in the

United States and other countries resulted in a very diverse student body. Many students came from

"uneducated" families or families that did not speak the native tongue. As a result, the failure rate in

137

Clinical Psychology (PSY401)

schools shot up dramatically. In order to preserve resources, there was pressure to identify those most

likely to succeed in school. Second, psychological scientists believed, and ultimately demonstrated, that

mental abilities could be measured. Although early attempts focused primarily on measures of sensory

acuity and reaction time (for example, Francis Galton, James McKeen Cattell), the groundwork was laid

in place.

Alfred Binet and his collaborator, Theodore Simon, became leaders in the intelligence testing movement

when they devised the Binet-Simon test to identify individual differences in mental functioning. Binet's

original purpose was to develop an objective method of identifying those truly lacking in academic

ability (as opposed to those with behavior problems). Like others of the day, Binet and Simon regarded

intelligence as a "faculty" that was inherited, although they also spoke of it as affected by training and

opportunity. With the interest in quantifying intellectual performance and with the continuing growth of

compulsory education in Europe and North America, intelligence testing became firmly entrenched

(Thorndike, 1997).

Institutions such as schools, industries, military forces, and governments were, by their nature,

interested in individual differences (such as levels of intelligence) that might affect performance in those

settings; therefore, intelligence testing prospered (Herrnstein & Murray, 1994).For many years, the

critical importance and widespread use of intelligence tests went largely unchallenged. However, by the

end of the 1960s, everyone seemed to be attacking the validity of these tests. Basically, the argument

was that such tests discriminate through the inclusion of unfair items.

As a result of a lengthy civil rights suit (Larvy P v. Wilson Riles) begun in 1971, the California State

Board of Education in 1975 imposed a moratorium on the use of intelligence tests to assess disabilities

in African Americans. The court held that IQ testing is prejudicial to African American children and

tends to place them, without real justification, in allegedly stigmatizing programs for cognitively

impaired individuals.

Others (for example, N. Lambert, 1981) have disputed the court's judgment, however. Some African

Americans contemplated a court challenge of the ruling, claiming it assumed that African Americans

would do poorly on the tests. Still others argued that IQ testing is not a social evil but the principal

means by which we can right the wrongs imposed upon minorities by a devastating environment.

The most widely used intelligence tests in the United States are those originally developed by

psychologist David Wechsler during the 1940's and 1950's.building on existing tests of the day--

including the Standford-Binet the Army Alpha and Beta tests, and the Bellevue intelligence scale--

Wechsler first developed an individual test in intelligence for adults. Followed by a similarly structured

test for school-age children to age 16, and finally, a test for pre-school-age children.

These tests were influenced by Wechsler's belief that there is a total or global level of intellectual

capacity that can be measured-thus; these tests yield a score that represents the person's overall

intelligence. the tests developed by Wechsler also reflect the geographic metaphor of intelligence

described by Sternberg(1990).Wechsler's tests more than any others have shaped psychologist's

perception of intellectual functioning as comprised of separate but related verbal and performance(non-

verbal abilities).

Wechsler (1939) emphasized that an IQ test measures functional intelligence, not intelligence it self.

Functional intelligence is influenced by nonintellectual factors including motivation, configuration of

specific abilities, and emotional adjustment. According to Wechsler score on an IQ test is a reflection of

what one has learned, and which is a function of the opportunities to which one has been exposed and

one's ability to take advantage of those opportunities. The subsets on Wechsler's test represent samples

of behavior but they are not exhaustive.

138

Clinical Psychology (PSY401)

The need for continued adaptation of intelligence testing is represented in the history of the Wechsler

scales, because all three versions have undergone substantial revisions since their inception and even

after Wechsler's death. These changes have taken two forms: changes in the items of the tests to make

them more current and appropriate for new generations, and the testing of new normative samples to

provide up-to-date sources for normative comparison in generating scores.

CONCLUSION

There is little question that intelligence tests have been misused at times in ways that have penalized

minorities. There is also little doubt that some tests have contained certain items that have adversely

affected the performance of some minorities. We should, therefore, do everything we can to develop

better tests and to administer and interpret them in a sensitive fashion. However, banning tests seems an

inappropriate cure that may ultimately harm the very people who need help.

139

Table of Contents: