TOOLS FOR DATA COLLECTION:Guidelines for Questionnaire Design

<< SELF ADMINISTERED QUESTIONNAIRES (CONTINUED):Interesting Questions

PILOT TESTING OF THE QUESTIONNAIRE:Discovering errors in the instrument >>

Research Methods STA630

Lesson 23

TOOLS FOR DATA COLLECTION

Broadly there are tools of data collection as part of communication surveys. These are:

1. Interview schedule

2. Questionnaire

3. Interview Guide

As discussed earlier interview schedule and questionnaires both are predesigned list of questions used

for communication with the respondents. In the case of interview schedule, the list of questions remains

in the hands of the interviewer who asks questions from the respondent, gets his/her response, and

records the responses. Questionnaire is also a list of questions, which is handed over to the respondent,

who reads the questions and records the answers himself. For purposes of convenience questionnaire

will refer to both interview schedule as well as questionnaire.

Interview guide is list of topics that are to be covered during the course of interview. Interview guide is

used for purposes of an in-depth interviewing. Questions on the topics are formulated on the spot. Most

of the questions are open ended. The interviewer may not use the same wording for each respondent;

the number of questions may be different; the sequence of questions may also be different.

Guidelines for Questionnaire Design

A survey is only as good as the questions it asks. Questionnaire design is one of the most critical stages

in the survey research process. While common sense and good grammar are important in question

writing, more is required in the art of questionnaire design. To assume that people will understand the

questions is common error. People may not simply know what is being asked. They may be unaware of

topic of interest, they may confuse thee subject with something else, or the question may not mean the

same thing to every respondent. Respondents may simply refuse to answer personal questions. Further,

properly wording the questionnaire is crucial, as some problems may be minimized or avoided

altogether if a skilled researcher composes the questions.

A good questionnaire forms an integrated whole. The researcher weaves questions together so they

flow smoothly. He or she includes introductory remarks and instructions for clarification and measures

each variable with one or more survey questions.

What should be asked?

The problem definition will indicate which type of information must be collected to answer the research

question; different types of questions may be better at obtaining certain type of information than others.

1. Questionnaire Relevancy

A questionnaire is relevant if no unnecessary information is collected and if the information that is

needed to solve the problem is obtained.

Asking the wrong or an irrelevant question is a pitfall to be avoided. If the task is to pinpoint

compensation problems, for example, questions asking for general information about morale may be

inappropriate. To ensure information relevancy, the researcher must be specific about data needs, and

there should be a rationale for each item of information.

2. Questionnaire Accuracy

Once the researcher has decided what should be asked, the criterion of accuracy becomes of primary

concern. Accuracy means that the information is reliable and valid. While experienced researchers

believe that one should use simple, understandable, unbiased, unambiguous, and nonirritating words.

Obtaining accurate answer from respondents is strongly influenced by the researcher's ability to design

a questionnaire that facilitates recall and that will motivate the respondent to cooperate. Therefore avoid

Research Methods STA630

jargon, slang, and abbreviations. The respondents may not understand some basic terminology.

Respondents can probably tell thee interviewer whether they are married, single, divorced, separated, or

widowed, but providing their "marital status" may present a problem. Therefore, asking somebody

about his/her marital status while the person may not understand the meaning of marital status is likely

to mess up the information. Words used in the questionnaire should be readily understandable to all

respondents.

3. Avoid Ambiguity, Confusion, and Vagueness.

Ambiguity and vagueness plague most question writers. A researcher might make implicit assumptions

without thinking of respondents' perspectives. For example, the question, "what is your income?" could

mean weekly, monthly, or annual: family or personal; before taxes or after taxes; for this year or last

year; from salary or from all sources. The confusion causes inconsistencies in how different respondents

assign meaning to and answer the question.

Another source of ambiguity is the use indefinite words or response categories. Consider the words

such as often, occasionally, usually, regularly, frequently, many, good, fair, and poor. Each of these

words has many meanings. For one person frequent reading of Time magazine may be reading six or

seven issues a year; for another it may be two issues a year. The word fair has great variety of

meanings; the same is true for many indefinite words.

4. Avoid Double-Barreled Questions

Make each question about one and only one. A double barreled question consists of two or more

questions joined together. It makes the respondent's answer ambiguous. For example, if asked, "Does

this company have pension and health insurance benefits?" a respondent at the company with health

insurance benefits only might answer either yes or no. The response has an ambiguous meaning and the

researcher cannot be certain of the respondent's intentions. When multiple questions are asked in one

question, thee results may be exceedingly difficult to interpret.

5. Avoid Leading Questions

Make respondents feel that all responses are legitimate. Do not let them aware of an answer that the

researcher wants. A leading question is the one that leads the respondent to choose one response over

another by its wording. For example, the question, "you don't smoke, do you?" leads respondents to

state that they do not smoke. "Don't you think that women should be empowered?" In most the cases

the respondent is likely to agree with the statement.

6. Avoid Loaded Questions

Loaded questions suggest a socially desirable answer or are emotionally charged. "Should the city

government repair all the broken streets?" Most of the people are going to agree with this question

simply because this is highly socially desirable. A question which may be challenging the traditionally

set patterns of behavior may be considered as emotionally charged i.e. it is loaded with such material

which may hit the emotions of the people. Look at some behaviors associated with masculinity in

Pakistani society. Let us ask a husband "Have you ever been beaten up by your wife?" Straight away

this question may be considered to be a challenge to the masculinity of the person. Hence it may be

embarrassing for the person to admit such an experience. Therefore, even if the husband was beaten up

by his wife, he might give a socially desirable answer.

7. Avoid Burdensome Questions that may Tax the Respondent's Memory

A simple fact of human life is that people forget. Researchers writing questions about past behavior or

events should recognize that certain questions may make serious demand on the respondent's memory.

Research Methods STA630

"How did you feel about your brother when you were 6 years old?" It may very difficult to recall

something from the childhood.

8. Arrange Questions in a Proper Sequence

The order of question, or the question sequence, may serve several functions for the researcher. If the

opening questions are interesting, simple to comprehend, and easy to answer, respondent's cooperation

and involvement can be maintained throughout the questionnaire. If respondent's curiosity is not

aroused at the outset, they can become disinterested and terminate the interview.

Sequencing specific questions before asking about broader issues is a common cause of question order

bias. In some situations it may be advisable to ask general question before specific question to obtain

the freest opinion of the respondent. This procedure, known as funnel technique, allows the researcher

to understand the respondent's frame of reference before asking specific questions about thee level of

respondent's information and intensity of his or her opinions.

9. Use Filter Question, if Needed

Asking a question that doesn't apply to the respondent or that the respondent is not qualified to answer

may be irritating or may cause a biased response. Including filter question minimizes the chance of

asking questions that are inapplicable. Filter question is that question which screens out respondents not

qualified to answer a second question. For example the researcher wants to know about the bringing up

of one's children. "How much time do you spend playing games with your oldest child?" What if the

respondent is unmarried? Even if the respondent is married but does not have the child. In both these

situations the question is inapplicable to him/her. Before this question the person may put a filter

question whether or not the respondent is married.

10. Layout of the questionnaire

There are two format or layout issues: the overall physical layout of the questionnaire and the format of

questions and responses.

Good lay out and physical attractiveness is crucial in mail, Internet, and other self-administered

questionnaires. For different reason it is also important to have a good layout in questionnaires

designed for personal and telephone interviews.

Give each question a number and put identifying information on questionnaire. Never cramp questions

together or create a confusing appearance.

Make a cover sheet or face sheet for each, for administrative use. Put the time and date of the interview,

the interviewer, the respondent identification number, and interviewer's comments and observations on

it. Give interviewers and respondents instructions on the questionnaire. Print instructions in a different

style from question to distinguish them.

Lay out is important for mail questionnaires because there is no friendly interviewer to interact with thee

respondent. Instead the questionnaire's appearance persuades the respondents. In mail surveys, include

a polite, professional cover letter on letterhead stationery, identifying the researcher and offering a

telephone number for any questions. Always end with "Thank you for your participation."

Table of Contents: