HISTORICAL-COMPARATIVE RESEARCH (Contd.):Locating Evidence

<< HISTORICAL COMPARATIVE RESEARCH:Similarities to Field Research

FOCUS GROUP DISCUSSION:The Purpose of FGD, Formal Focus Groups >>

Research Methods STA630

Lesson 41

HISTORICAL-COMPARATIVE RESEARCH (Contd.)

Conducting historical-comparative research does not involve a rigid set of steps and, with only a few

exceptions; it does not use complex or specialized techniques. Nevertheless, some guideline for doing

historical-comparative research may be provided.

Conceptualizing the Object of Inquiry

An H-C researcher begins by becoming familiar with the setting and conceptualizes what is being

studied. He or she may start with a loose model or set of preliminary concepts and apply them to

specific setting. The provisional concepts contain implicit assumptions or organizing categories that he

or she uses to see the world, "package" observations, and search through evidence.

Decide on the historical era or comparative settings (nations or units). If the researcher is not already

familiar with historical era or comparative settings, he or she conducts an orientation reading (reading

several general works). This will help the researcher grasp the specific setting, assemble organizing

concepts, subdivide the main issue, and develop lists of questions relating to specific issue.

Locating Evidence

The researcher locates and gathers evidence through extensive bibliographic work. A researcher uses

many indexes, catalogs, and reference works that list what libraries contain. For comparative research,

this means focusing on specific nations or units and on particular kinds of evidence within each. The

researcher frequently spends weeks searching for sources in libraries, travels to several different

specialized research libraries, and reads dozens of books and articles. Comparative research often

involves learning one or more foreign languages.

As the researcher masters the literature and takes numerous detailed notes, he or she completes many

specific tasks: creating a bibliography list (on cards or on computer) with complete citations, taking

notes that are neither too skimpy nor too extensive, leaving margins on note cards for adding themes

later on, taking all note in the same format, and developing a file on themes or working hypothesis.

A researcher adjusts initial concepts, questions, or focus on the basis of what he or she discovers in the

evidence. New issues and questions arise as he or she reads and considers a range of research reports at

different levels of analysis (e.g., general context and detailed narratives on specific topic), and multiple

studies on a topic, crossing topic boundaries.

Evaluating Quality of Evidence

As the H-C researcher gathers evidence, he or she asks two questions: Hoe relevant is the evidence to

emerging research questions and evolving concepts? How accurate and strong is the evidence?

The question of relevance is difficult one. All documents may not be equally valuable in reconstructing

the past. As the focus of research shifts, evidence that was not relevant can become relevant. Likewise,

some evidence may stimulate new avenues of inquiry and search for additional confirming evidence.

Accuracy of evidence may be looked at for three things: the implicit conceptual framework, particular

details that are required and empirical generalizations. H-C researcher evaluates alternative

interpretations of evidence and looks for "silences," of cases where the evidence fails to address an

event, topic, or issue.

Researchers try to avoid possible fallacies in the evidence. For example, a fallacy of pseudo proof is

failure to place something into its full context. The evidence might state that that there was a 50 percent

increase in income taxes, but it is not meaningful outside of a context. The researcher must ask: Did

other taxes decline? Did income increase? Did the tax incase apply to all income? Was everyone

affected equally?

147

Research Methods STA630

Organizing Evidence

As a researcher gathers evidence and locates new sources, he or she begins to organize the data.

Obviously, it is unwise to take notes madly and let them pile up haphazardly. A researcher usually

begins a preliminary analysis by noting low-level generalizations or themes. For example, in a study of

revolution, a researcher develops a theme: The rich peasants supported the old regime. He or she can

record this theme in his or her notes and later assign to significance.

Researcher organizes evidence, using theoretical insights to stimulate new ways to organize data and for

new questions to ask of evidence. The interaction of data and theory means that a researcher goes

beyond a surface examination of the evidence based on theory. For example, a researcher reads a mass

of evidence about a protest movement. The preliminary analysis organizes the evidence into a theme:

People who are active in protest interact with each other and develop shared cultural meanings. He or

she examines theories of culture and movements, then formulates new concept: "oppositional movement

subculture." The researcher then uses this concept to re-examine the evidence.

Synthesizing

The researcher refines concepts and moves toward a general explanatory model after most of the

evidence is in. Old themes or concepts are discussed or revised, and new ones are created. Concrete

events are used to give meaning to concepts.

The researcher looks for patterns across time or units, and draws out similarities and differences with

analogies. He or she organizes divergent events into sequences and groups them together to create a

larger picture. Plausible explanations are then developed that subsume both concepts and evidence as

he or she organizes the evidence into a coherent whole. The researcher then reads and rereads notes and

sorts and resorts them into piles or files on the basis of organizing schemes. He or she looks for and

writes down the links or connections he or she sees while looking at evidence in different ways.

Synthesis links specific evidence with an abstract model of underlying relations or causal mechanism.

A researcher often looks for new evidence to verify specific links that appear only after an explanatory

model is developed. He or she evaluates how well the model approximates the evidence and adjusts it

accordingly.

Historical-comparative researchers also identify critical indicators and supporting evidence for themes

or explanations. A critical indicator is unambiguous evidence, which is usually sufficient for inferring a

specific theoretical relationship. Researchers seek these indicators for key parts of an explanatory

model. Indicators critically confirm a theoretical inference and occur when many details suggest a clear

interpretation.

Writing a Report

Combine evidence, concepts, and synthesis into a research report. The way in which the report

is written is key in H-C research. Assembling evidence, arguments, and conclusions into a

report is always a crucial step; but more than in quantitative approaches, the careful crafting of

evidence and explanation makes or breaks H-C research. A researcher distills mountains of

evidence into exposition and prepares extensive footnotes. She or he weaves together evidence

and arguments to communicate a coherent, convincing picture to readers.

Data and Evidence in Historical context

Historical-comparative researchers draw on four types historical evidence or data:

1. Primary sources;

2. Secondary sources;

3. Running records; and

4. Recollections.

Traditional historians rely heavily on primary sources. H-C researchers often use secondary sources or

the different data types in combination.

148

Research Methods STA630

1. Primary Sources: The letters, diaries, newspapers, movies, novels, articles of clothing,

photographs, and so forth are those who lived in the past and have survived to the present are the

primary sources. They are found in archives (a place where documents are stored), in private

collections, in family closets, or in museums. Today's documents and objects (our letters, television

programs, commercials, clothing, and automobiles) will be primary sources for future historians. An

example of a classic primary source is a bundle of yellowed letters written by a husband away at war to

his wife and found in a family closet by a researcher.

Published and unpublished written documents are the most important type of primary source.

Researchers find them in their original form or preserved in microfilm or on film. They are often the

only surviving record of the words, thoughts, and feelings of people in the past. Written documents are

helpful for studying societies and historical periods with writing and literate people. A frequent

criticism of written sources is that elites or those in official organizations largely wrote them; thus the

views of the illiterate, the poor, or those outside official social institutions may be overlooked.

The written word on paper was the main medium of communication prior to the widespread use of

telecommunications, computers, and video technology to record events and ideas. In fact, the spread of

forms of communication that do not leave a permanent physical record (e.g., telephone conversation),

and which have largely replaced letters, written ledgers, and newspapers, make the work of future

historians difficult.

Potential Problems with Primary Sources: The key issue is that only a fraction of everything written

or used in the past has survived into present. Moreover, whatever is survived is nonrandom sample of

what once existed.

H-C researchers attempt to read primary sources with the eyes and assumptions of a contemporary who

lived in the past. This means "bracketing," or holding back knowledge of subsequent events and

modern values. "If you do not read the primary sources with an open mind and an intention to get

inside the minds of the writings and look at things the way they saw them, you are wasting time." For

example, when reading a source produced by a slaveholder, moralizing against slavery or faulting the

author for not seeing its evil is not worthwhile. The H-C researcher holds back moral judgments and

becomes a moral relativist while reading primary sources. He or she must think and believe like

subjects under study, discover how they performed in their own eyes.

Another problem is that locating primary documents is a time consuming task. A researcher must search

through specialized indexes and travel to archives or specialized libraries. Primary sources are often

located in dusty, out-of-the-way room full of stacked cardboard boxes containing masses of fading

documents. These may be incomplete, unorganized, and various stages of decay. Once the documents

or other primary sources are located, the researcher evaluates them subjecting them to external and

internal criticism.

External criticism means evaluating the authenticity of a document itself to be certain that it is not a

fake or a forgery. Criticism involves asking: Was the document created when it is claimed to have been,

in the place where it was supposed to be, and by the person who claims to be its author? Why was the

document produced to begin with, and how did it survive? Once the document passes as being

authentic, a researcher uses internal criticism, an examination of the document's contents to establish

credibility. A researcher evaluates whether what is recorded was based on what the author directly

witnessed or is secondhand information.

Many types of distortions can appear in primary documents. One is bowdlerization a deliberate

distortion designed to protect moral standards or furnish a particular image. For example, photograph is

taken of the front of a building. Trash and empty bottles are scattered all around the building, and the

paint is faded. The photograph, however, is taken of the one part of the building that has little trash and

is framed so that the trash does not show; dark room techniques make the faded paint look new.

149

Research Methods STA630

2. Secondary Sources: Social researchers often use secondary sources, the books and articles written

by specialist historians and other researchers, as an evidence of past conditions. It has its own

limitations.

Potential Problems with Secondary Sources: The limitations of secondary historical evidence include

problems of inaccurate historical accounts and lack of studies in areas of interest. Such sources cannot

be used to test hypotheses. Post facto explanations cannot meet positivist criteria of falsifiability,

because few statistical controls can be used and replication is not possible.

The many volumes of secondary sources present a maze of details and interpretations for an H-C

researcher. He or she must transform the mass of specialized descriptive studies into an intelligible

picture. This picture needs to be consistent with the reflective of the richness of the evidence. It also

must bridge the many specific time periods and locals. The researcher faces potential problems with

secondary sources.

One problem is reading the works of historians. Historians do not present theory-free, objective "facts."

They implicitly frame raw data, categorize information, and shape evidence using concepts. The

historian's concepts are a mixture drawn from journalism, language of historical actors, ideologies,

Philosophy, everyday language in the present, and social science. Most lack a rigorous definition, are

vague, are applied inconsistently, and are not mutually exclusive, nor exhaustive.

Second problem is that historian's selection procedure is not transparent. They select some information

from all possible evidence. From the infinite oceans of facts historian selects those, which are

significant for his purpose. Yet, the H-C researcher does not know how this was done. Without

knowing the selection process, a historical-comparative researcher must rely on the historian's

judgments, which can contain biases.

A third problem is in the organization of the evidence. Historians organize evidence as they write works

of history. They often write narrative history. This compounds problems of undefined concepts and the

selection of evidence. In the historical narrative, the writer organizes material chronologically around a

single coherent "story." The logic is that of a sequence of unfolding action. Thus, each part of the story

is connected to each other part by its place in the time order of events. Together all the parts form a

unity or whole. Conjecture and contingency are the key elements of the narrative form. The

contingency creates a logical interdependency between earlier and later elements.

With its temporal logic, the narrative organization differs from how the social researchers create

explanations. It also differs from quantitative explanation in which the researcher identifies statistical

patterns to infer causes. A major difficulty of the narrative is that the organizing tool time order or

position in a sequence of events does not alone denote theoretical or historical causality. In other

word, the narrative meets only one of the three criteria for establishing causality that of temporal

sequence.

Fourth and the last problem is that historiographic schools, personal beliefs, social theories influence a

historian, as well as current events at the time research were conducted. Historians writing today

examine primary material differently from how those writing in the 1920s did. In addition, there are

various schools of historiography (diplomatic, Marxist) that have their own rules for seeking evidence

and asking questions. It is also said history gets written by the people in power; it may include what the

people in power want to be included.

3. Running Records: Running records consist of files or existing statistical documents maintained by

organizations. An example of a running record is keeping of vital statistics by the government

departments in Pakistan; vital statistics relating to births, marriage, divorce, death, and other statistics of

vital events. We also have so many documents containing running records relating to demographic

statistics, and economic statistics being maintained by different agencies of UNO.

150

Research Methods STA630

4. Recollections: The words or writing of individuals about their past lives or experiences based on

memory are recollections. These can be in the form of memoirs, autobiographies, or interviews.

Because memory is imperfect, recollections are often distorted in ways that primary sources are not.

In gathering oral history, a type of recollection, a researcher conducts unstructured interviews with

people about their lives or events in the past. This approach is especially valuable for non-elite groups

or the illiterate.

Evaluating the Documents

Historical-comparative researchers often use secondary sources or different data types in combination.

For secondary sources they often use existing documents as well as the data collected by other

organizations for research purposes. While looking into the authenticity of these document researchers

often want answers to the questions like: Who composed the documents? Why were these written?

What methods were used to acquire the information? What are some of the biases in the documents?

How representative was the sample? What are the key categories and concepts used? What sorts of

theoretical issues and debates do these documents cast light on?

Problems in Comparative Research

Problems in other types of research are magnified in a comparative study. In principle, there is no

difference between comparative cross-cultural research and research conducted in a single society. The

differences lie, rather, in the magnitude of certain types of problems.

The Units being compared:

For convenience, comparative researchers often use nation-state as their unit of analysis. The nation-

state is the major unit used in thinking about the divisions of people across globe today. The nation-state

is a socially and politically defined unit. In it, one government has sovereignty over populated territory.

The nation-state is not the only unit for comparative research, but also frequently used as a surrogate for

culture, which more difficult to define as a concrete, observable unit. The boundaries of nation-state

may not match those of a culture. In some situations a single culture is divided into several nations

(Muslim culture); in other cases, a nation-state contains more than one culture (Canada). The nation-

state is not always the best unit for comparative research. A researcher should ask: What is the relevant

comparative unit for my research question the nation, the culture, a small region, or a subculture?

Problems of Equivalence: Equivalence is a critical issue in all research. It is the issue of making

comparisons across divergent contexts, or whether a researcher, living in a specific time period and

culture, correctly reads, understands, or conceptualizes data about people from different historical era or

culture. Without equivalence, a researcher cannot use the same concepts or measures in different

cultures or historical periods, and this makes comparison difficult, if not impossible. It is similar to the

problems of validity in quantitative research. Look at the concept of a friend. We ask some body how

many friends do you have? People living in different countries may have different meanings attached to

it. Even in Pakistan, we have variations in its meaning across the Provinces, and between rural and

urban areas.

Ethical problems are less intense in H-C research than in other types of social research because a

researcher is less likely to have direct contact with people being studied. Historical-comparative

research shares the ethical concerns found in other non-reactive research techniques.

151

Table of Contents: