NON-REACTIVE RESEARCH:Recording and Documentation

<< EXPERIMENTAL RESEARCH (Cont.):Validity in Experiments

USE OF SECONDARY DATA:Advantages, Disadvantages, Secondary Survey Data >>

Research Methods STA630

Lesson 36

NON-REACTIVE RESEARCH

Experiments and surveys research are both reactive; that is, the people being studied are aware of the

fact that they are being studied. In non-reactive research, those being studied are not aware that they

are part of a research project. Such a research is largely based on positivistic principles but is also used

by interpretive and critical researchers.

The Logic of Non-Reactive Research

The critical thing about non-reactive or unobtrusive measures (i.e. the measures that are not obtrusive or

intrusive) is that the people being studied are not aware of it but leave evidence of their social behavior

or actions `naturally." The researcher infers from the evidence to behavior or attitudes without

disrupting those being studied. Unnoticed observation is also a type of non-reactive measure. For

example, a researcher may be observing the behavior of drivers from a distance whether drivers stopped

at red sign of the traffic lights. The observations can be made both at the day time and at night. It could

also be noted whether the driver was a male or a female; whether the driver was also or with passengers;

whether other traffic was present; and whether thee car came to a complete stop, a slow stop, or no stop.

Varieties of Non-Reactive Observations

Non-reactive measures are varied, and researchers have been creative in inventing indirect ways to

measures behaviors. Because the measures have little in common except being non-reactive, they are

best learned through examples like:

Physical Traces:

· Erosion: Wear and tear suggests a greater use. For example, a researcher examines children's

toys at a children's play centre that were purchased at the same time. Worn out toys suggest

greater interest of children in them.

· Accretion: Accumulation of physical evidence suggests behavior. A researcher examines the

soft drink cans or bottles in the garbage collection. That might indicate the brands and types

of soft drinks that are very popular.

Archives:

· Running Records: Regularly produced public records may reveal lot of information. For

example, a researcher may examine marriage records for brides' and grooms' recorded ages.

The differences might indicate that males marrying younger females are greater than the other

way around.

· Other Records: Irregular or private records can reveal a lot. For example, a researcher may

look into the number of reams of paper purchased by a college principal's office for the last 10

years and compare it with students' enrollment.

Observations:

· External Appearance: How people appear may indicate social factors. For example, a

researcher watches students to see whether they are more likely to wear their college's colors

and symbols after the college team won or lost.

· Count Behaviors: Counting how many people do something can be informative. For example

a researcher may count the number of men and women who come to a full stop and those who

come to a rolling stop at a traffic stop sign. This suggests gender difference in driving

behavior.

· Time Duration: How long people take to do things may indicate their intention. For example

a researcher may measure how long men and women pause in front of a particular painting.

Time taken may indicate their interest in the painting.

124

Research Methods STA630

Recording and Documentation

Creating non-reactive measures follows the logic of quantitative measurement, although qualitative

researchers also use non-reactive observations. A researcher first conceptualizes a construct, and then

links the construct to non-reactive empirical evidence, which is its measure. The operational definition

of the variable includes how the researcher systematically notes and records observations.

Content Analysis

Content analysis is a technique for gathering and analyzing the content of a text. The content refers to

words, meanings, pictures, symbols, ideas, themes, or any message that can be communicated. The text

is anything written, visual, or spoken that serves as a medium of communication. Possible artifacts for

study could be books, newspaper or magazine articles, advertisements, poems, letters, laws,

constitutions, dramas, speeches, official documents, films or videotapes, musical lyrics, photographs,

articles of clothing, or works of arts. All these works may be called as documents. The documents can

be:

· Personal letters, diary, autobiography.

· Non-personal interoffice memos, official documents, proceedings of a meeting.

· Mass media newspapers, magazines, fiction, films, songs, poems, works of arts.

Content analysis goes back nearly a century and is used in many fields literature, history, journalism,

political science, education, psychology, sociology, and so on. It is also called a study of

communication, which means who says what, to whom, why, how, and with what effect.

In content analysis, the researcher uses objective and systematic counting and recording procedures to

produce a quantitative description of the symbolic content in a text. It may also be called "textual

coding." There are qualitative versions of content analysis. The emphasis here is quantitative data

about a text's content.

Content Analysis is Non-Reactive: It is non-reactive because the placing of words, messages, or

symbols in a text to communicate to the reader or receiver occurs without influence from the researcher

who analyzes its contents. There is no interaction between the researcher and the creator of the text

under analysis.

Content analysis lets a researcher reveal the contents (i.e. messages, meanings, symbols, etc.) in a

source of communication (i.e. a book, article, movie, etc.). It lets him/her probe into and discover

content in a different way from ordinary way of reading a book or watching a television program.

With content analysis, a researcher can compare content across many texts and analyze it with

quantitative techniques (table, charts). In addition, he or she can reveal aspects of the text's content that

are difficult to see. For example, you might watch television commercials and feel that women are

mostly portrayed working in the house, cooking food, using detergents, looking after children. Content

analysis can document in objective, quantitative terms whether or not your vague feelings based on

unsystematic observation are true. It yields repeatable, precise results about the text.

Content analysis involves random sampling, precise measurement, and operational definitions for

abstract constructs. Coding turns aspects of content that represent variables into numbers. After a

content analysis researcher gathers the data, he or she enters them into computers and analyzes them

with statistics in the same way that an experiment or survey researcher would.

Measurement and Coding

Careful measurement is crucial in content analysis because a researcher takes different and murky

symbolic communication and turns it into precise, objective, quantitative data. He or she carefully

125

Research Methods STA630

designs and documents the procedures for coding to make replication possible. For example, a

researcher wants to determine how frequently television dramas portray elderly characters in terms of

negative stereotypes. He or she develops a measure of the construct "negatively stereotypes of the

elderly." The conceptualization may result in a list of stereotypes or negative generalizations about

older people (e.g., senile, forgetful, frail, hard of hearing, slow, ill, inactive, conservative, etc.) that do

not accurately reflect the elderly. Another example could be negative stereotypes about women.

Constructs in content analysis are operationalizing with a coding system, a set of instructions or rules on

how to systematically observe and record content from text. Look at the construct of "leadership role;"

for measuring this construct written rules should be provided telling how to classify people. Same is

about the concept of "social class." In case the researcher has three categories of upper, middle, and

lower class then the researcher must tell what are the characteristics that are associated with upper class,

middle class, and the lower class so that the coders could easily classify people in the three proposed

categories.

Observations can be structured: Measurement in content analysis uses structured observation i.e.

systematic, careful observation based on written rules. The rules explain how to categorize and classify

observations in terms of:

· Frequency: Frequency simply means counting whether or not something occurs and how

often (how many times). For example how many elderly people appear on a television

program within a given week? What percentage of all characteristics are they, or in what

percentage of programs do they appear.

· Direction: Direction is noting the direction of messages in thee content along some

continuum (e.g., positive or negative, supporting or opposed). For example the researcher

devises a list of ways an elderly television character can act. Some are positive (e.g., friendly,

wise, considerate) and some are negative (e.g., nasty, dull, selfish).

· Intensity: Intensity is the strength or power of a message in a direction. For example, the

characteristic of forgetfulness can be minor (e.g. not remembering to take the keys when

leaving home, taking time to recall the name of someone whom you have not seen in years) or

major (e.g., not remembering your name, not recognizing your children.

· Space: A researcher can record the size of the text message or the amount of space or volume

allocated to it. Space in written text is measured by counting words, sentences, paragraphs, or

space on a page (e.g. square inches) for video or audio text, space can be measured by the

amount of time allocated. For example, a TV character may be present for a few seconds or

continuously in every seen of a two hour program.

The unit analysis can vary a great deal in content analysis. It can be a word, a phrase, a theme, a plot,, a

news paper article, a character, and so forth.

Coding

The process of identifying and classifying each item and giving labels to each category. Later on each

category may be assigned a numerical value for its entry into the computer. In content analysis one can

look at the manifest coding and latent coding.

Manifest Coding: Coding the visible, surface content in a text is called manifest coding. For example,

a researcher counts the number of times a phrase or word (e.g. red) appears in the written text, or

whether a specific action (e.g. shaking hands) appears in a photograph or video scene. The coding

system lists terms or actions or characters that are then located in text. A researcher can use a computer

program to search for words or phrases in the text and have a computer do the counting work.

Manifest coding is highly reliable because the phrase or the word either is or is not present. However,

manifest coding does not take the connotation of word into account. The same word can take on

126

Research Methods STA630

different meanings depending on the context. The possibility that there are multiple meanings of a word

limits the measurement validity of manifest coding.

Latent Coding: A researcher using latent coding (also called semantic analysis) looks for the

underlying meaning in the content of a text. For example, the researcher reads the entire paragraph and

decides whether it contains vulgar themes or a romantic mood. His or her coding system has general

rules to guide his or her interpretation of the text and for determining whether particular themes or mood

are present.

Latent coding tends to be less reliable than the manifest coding. It depends on a coder's knowledge of

language and its social meaning. Training, practice, and written rules improve reliability, but still it is

difficult to consistently identify themes, moods, and the like.

Keeping in view the amount of work, often a number of coders are hired. The researcher trains the

coders in coding system. Coders should understand the variables, follow the coding system, and ask

about ambiguities. A researcher who uses several coders must always check for consistency across

coders. He or she does this by asking coders to code the same text independently and then checking for

consistency across coders. The researcher measures inter-coder reliability, a type of equivalence

reliability, with a statistical coefficient that tells the degree of consistency across among coders. The

coefficient is always reported with the results of content analysis research.

How to Conduct Content Analysis Research

Question Formulation: As in most research, content analysis researchers begin with a research

question. When the question involves variables that are messages or symbols, content analysis may be

appropriate. For example, how women are portrayed in advertisements? The construct here is the

portrayal of women which may be measured by looking at the activities they are shown to be doing, the

occupations in which they are employed, the way decision making is taking place, etc.

Unit of Analysis: A researcher decides on the unit of analysis (i.e. the amount of text that is assigned a

code). In the previous example each advertisement may be a unit of analysis.

Sampling: Researchers often use random sampling in content analysis. First, they define the

population and the sampling element. For example, the population might be all words, all sentences, all

paragraphs, or all articles in certain type of documents over a period of specified period. Likewise, it

could include each conversation, situation, scene, or episode of a certain type of television program over

a specified time period. Let us consider that we want to know how women are portrayed in weekly

news magazines. The unit of analysis is the article. The population includes all articles published in

weekly news magazines during 2001 to 2007. Make a list of English magazines that were published

during the said period. Define what is a news magazine? Define what is an article? Decide on the

number of magazines. Decide on the sample size. Make a sampling frame. Here the sampling frame

shall be all the articles published in the selected magazines during 2001 to 2007. Finally draw the

random sample using table of random numbers.

Variables and Constructing Coding Categories: Say a researcher is interested in women's portrayal

in significant leadership roles. Define "significant leadership role" in operational terms and put it as

written rules for classifying people named in the articles. Say the researcher is further interested in

positive leadership roles, so the measure will indicate whether the role was positive or negative.

Researcher has to make a list of adjectives and phrases reflective of the leadership role being positive or

negative. If someone in the article is referred to with one of the adjective, then the direction is decided.

For example, the terms brilliant and top performer are positive, whereas drug kingpin and uninspired

are negative. Researcher should give written rules to classify role of women as portrayed in the articles.

127

Research Methods STA630

In addition to written rules for coding decisions, a content analysis researcher creates a recording sheet

(also called a coding form or tally sheet) on which to record the information. Each unit should have a

separate recording sheet.

Inferences: The inference a researcher can or cannot make on the basis of results is critical in content

analysis. Content analysis describes what is in the text. It cannot reveal the intentions of those who

created the text or the effects that messages in the text have on those who receive them.

128

Table of Contents: