ZeePedia

USER RESEARCH: TYPES OF QUALITATIVE RESEARCH, ETHNOGRAPHIC INTERVIEWS

<< GOAL-DIRECTED DESIGN METHODOLOGIES: A PROCESS OVERVIEW, TYPES OF USERS
USER-CENTERED APPROACH, ETHNOGRAPHY FRAMEWORK >>
img
Human Computer Interaction (CS408)
VU
Lecture
19
Lecture 19. User Research Part-I
Learning Goals
As the aim of this lecture is to introduce you the study of Human Computer
Interaction, so that after studying this you will be able to:
·  Understand the difference in qualitative and quantitative research
·  Discuss in detail qualitative research technique
The outcome of any design effort must ultimately be judged by how successfully it
meets the requirements of both the user and the organization that commissioned it. No
matter how skillful and creative the designer. If she does not have a clear and detailed
knowledge of the users she is designing for, what the constraints of the problem are,
and what business or organizational goals the design is hoping to achieve, she will
have little chance of success.
What and how questions like these are best answered by qualitative research, not
metrics or demographics (though these also have their purpose). There are many types
of qualitative research, each of which plays an important role in filling in a picture of
the design landscape of a product.
Qualitative versus Quantitative Research
Research is a word that most people associate with science and objectivity. This
association isn't incorrect, but it biases many people towards the notion that the only
valid sort of research is the kind that yields the supposed ultimate in objectivity:
quantitative data. The notion that numbers don't lie is prevalent in the business and
engineering communities, even though we all rationally understand that numbers--
especially numbers ascribed to human activities--can be manipulated or reinterpreted
at least as dramatically as words.
Data gathered by the hard sciences like physics is simply different from that gathered
on human activities: electrons don't have moods that vary from minute to minute, and
the tight controls physicists place on their experiments to isolate observed behaviors
are next to impossible in the social sciences. Any attempt to reduce human behavior
to statistics is likely to overlook important nuances, which though they might not
directly affect business plans, do make an enormous difference to the design of
products. Quantitative research can only answer questions about how much or how
many along a few reductive axes. Qualitative research can tell you about what, how
and why in rich, multivariate detail.
Social scientists have long realized that human behaviors are too complex and subject
to too many variables to rely solely on quantitative data to understand them. Usability
practitioners, borrowing techniques from anthropology and other social sciences, have
developed many alternative methods for gathering useful data on user behaviors to a
more pragmatic end: to help create products that better serve user need.
169
img
Human Computer Interaction (CS408)
VU
The value of qualitative research
Qualitative research helps us understand the domain, context and constraints of a
product in different, more useful ways than quantitative research does. It also quickly
helps us identify patterns of behavior among users and potential users of a product
much more quickly and easily than would be possible with quantitative approaches. In
particular, qualitative research helps us understand:
·  Existing products, and how they are used.
Potential users of new or existing products, and how they currently approach
·
activities and problems the new product design hopes to address
Technical, business, and environmental contexts--the domain--of the product
·
to be designed
Vocabulary and other social aspects of the domain in question
·
Qualitative research cab also help the progress of design projects by:
·  Providing credibility and authority to the design team, because design
decisions can be traced to research results
·  Uniting the team with a common understanding of domain issues and user
concerns
·  Empowering management to make more informed decisions about product
design issues that would otherwise be based on guesswork or personal
preference
It is the experienced that qualitative method, in addition to the benefits described
above, tend to be faster, less expensive, more flexible, and more likely than their
quantitative counterparts t provide useful answers to impotent questions that leads t
superior design:
·  What problems are people encountering with their current ways of doing what
the product hopes to do?
·  Into what broader contexts in people's lives does the product fit and how?
·  What are the basic goals people have in using the product, and what basic
tasks help people accomplish them?
Types of qualitative research
19.1
Social science and usability texts are full of methods and techniques for conducting
qualitative research. We will discuss following qualitative research techniques:
·  Stakeholder interviews
·  Subject matter expert (SME) interviews
·  User and customer interviews
·  User observation/ethnographic field studies
·  Literature review
·  Product/prototype and competitive audits
170
img
Human Computer Interaction (CS408)
VU
Stakeholder interviews
Research for any new product design, through it must end with understanding the
user, should start by understanding the business and technical context in which the
product will be built. This is necessary not only to ensure a viable and feasible end
result, but also to provide a common language and understanding among the design
team, management, and engineering teams.
Stakeholders are any key members of the organization commissioning the design
work, and typically include managers and key contributors from engineering, sales,
product marketing, marketing communications, customer support, and usability. They
may also include similar people from other organizations in business partnership with
the commissioning organization, and executives. Interviews with stakeholders should
occur before any user research begins.
It is usually most effective to interview each stakeholder one-on-one, rather than in a
larger, cross-departmental group. A one-on-one setting promotes candor on the part of
the stakeholder, and ensure that individual views are not lost in a crowd. Interviews
need not last longer than about an hour, though follow-up meetings may be called for
if a particular stakeholder is identified as an exceptionally valuable source of
information.
The type of information that is important to gather from stakeholders includes:
·  What is the preliminary vision of the product from each stakeholder
perspective? As in the fable of the blind men and the elephant, you may find
that each business department has a slightly different and slightly incomplete
perspective on the product to be designed. Part of the design approach must
therefore involve harmonizing these perspectives with those of users and
customers.
·  What is the budget and schedule? The answer to this question often provides a
reality check on the scope of the design effort and provides a decision point
for management if user research indicates a greater scope is required.
·  What are the technical constraints? Another important determinant of design
scope is a firm understanding of what is technically feasible given budget,
time, and technology.
·  What are the business drivers? It is important for the design team to
understand what the business is trying to accomplish. This again leads to a
decision point, should user research indicate a conflict between business and
user needs. The design must, as much as possible, create a win-win situation
for users, customers, and providers of the product.
·  What are the stakeholders' perceptions of the user? Stakeholders who have
relationships with users (such as customer support representative) may have
important insights on users that will help you to formulate your user research
plan. You may also find that there are significant disconnects between some
stakeholders' perceptions of their users and what you discover in your
research. This information can become an important decision point for
management later in the process.
Understanding these issues and their impact on design solutions helps you as a
designer to better serve your customer, as well as users of the product. Building
171
img
Human Computer Interaction (CS408)
VU
consensus internally will help you to articulate issues that the business as a whole may
not identified, build internal consensus that is critical for decision making later in the
design process, and build credibility for your design team.
Subject matter expert (SME) interviews
Some stakeholders may also be subject matter experts (SMEs): experts on the domain
within which the product you are designing will operate. Most SMEs were users of
the product or its predecessors at one time, and may now be trainers, managers, or
consultants. Often they are experts hired by stakeholders, rather than stakeholders
themselves. Similar to stakeholders, SMEs can provide valuable perspective on a
product and its users, but designers should be careful to recognize that SMEs
represent a somewhat skewed perspective. Some points to consider about using SMEs
are:
·  SMEs are expert users. Their long experience with a product or its domain
mean that they may have grown accustomed to current interactions. They may
also lean towards expert controls rather than interactions designed for
perpetual intermediate perspective. SMEs are often not current users of the
product, and may have more of a management perspective.
·  SMEs are knowledgeable, but they aren't designers. They may have many
ideas on how to improve a product. Some of these may be valid and valuable,
but the most useful pieces of information to glean from these suggestions are
the causative problems that lead to their proposed solutions.
·  SMEs are necessary in complex or specialized domains such as medical,
scientific, or financial services. If you are designing for a technical or
otherwise specialized domain, you will likely need some guidance from
SMEs, unless you are one yourself. Use SMEs to get information on complex
regulations and industry best practices. SME knowledge of user roles and
characteristics is critical for planning user research in complex domains.
·  You will want access to SMEs throughout the design process. If your product
domain requires use of SMEs, you should be able to bring them in at different
stages of the design to help perform reality checks on design details. Make
sure that you secure this access in your early interviews.
User and customer interviews
It is easy to confuse users with customers. For consumer products, customers are
often the same as users, but in corporate or technical domain, users and customers
rarely describe the same sets of people. Although both groups should be interviewed,
each has its own perspective on the product that needs to be factored quite differently
into an eventual design.
Customers of a product are those people who make the decision to purchase it. For
consumer product, customers are frequently users of the product; although for
products aimed at children or teens, the customers are parents or other adult
supervisors of children. In the case of most enterprise or technical products, the
customer is someone very different from the user--often an IT manager--with
distinct goals and needs. It's important to understand customers and their goals in
order to make a product viable. It is also important to realize that customers seldom
172
img
Human Computer Interaction (CS408)
VU
actually use the product themselves, and when they do, they use it quite differently
than the way their users do.
When interviewing customers, you will want to understand:
Their goals in purchasing the product
·
Their frustrations with current solutions
·
Their decision process for purchasing a product of the type you're designing
·
Their role in installation, maintenance, and management of the product
·
Domain related issues and vocabulary
·
Like SMEs, customers may have many opinions about how to improve the design of
the product. It is important to analyze these suggestions, as in the case of SMEs, to
determine what issues or problems underline the ideas offered, because better, more
integrated solutions become evident later in the design process.
Users of a product should be the main focus of the design effort. They are the people
(not their managers or support team) who are personally trying to accomplish
something with the product. Potential users are people who do not currently use the
product, but who are good candidates for using it in the future. A good set of user
interviews includes both current users (if the product already exists and is being
revised) and potential users (users of competitive products and non-automated
systems of appropriate). Information we are interested in learning from users includes:
·  Problems and frustrations with the product (or analogous system if they are
potential users)
·  The context of how the product fits into their lives or workflow: when, why,
and how the product is used, that is, patterns of user behavior with the product.
·  Domain knowledge from a user perspective: what do users need to know to
accomplish their jobs
·  A basic understanding of the users' current tasks: both those the product
requires and those it doesn't support
·  A clear understanding of user goals: their motivations and expectations
concerning use of the product
User observation
Most people are incapable of accurately assessing their own behaviors, especially
outside of the context of their activities. It then follows that interviews performed
outside the context of the situations the designer hopes to document will yield less
complete and less accurate data. Basically, you can talk to users about how they think
they behave, or you can observe it first hand. The latter route provides superior
results.
Many usability professionals make use of technological aids such as audio or video
recorders to capture what users say and do. Care must be taken not to make these
technologies too obtrusive: otherwise the users will be distracted and behave
differently than they would off-tape.
Perhaps the most effective technique for gathering qualitative user data combines
interviews and observation, allowing the designer to ask clarifying questions and
direct inquiries about situations and behaviors they observe in real-time.
173
img
Human Computer Interaction (CS408)
VU
Literature review
In parallel with stakeholder interviews, the design team should review any literature
pertaining to the product or its domain. This can and should include product
marketing plans, market research, technology specifications and white papers,
business and technical journal articles in the domain, competitive studies. Web
searches for related and competing products and news, usability study results and
metrics, and customer support data such as call center statistics.
The design team should collect this literature, use it as a basis for developing
questions to ask stakeholders and SMEs, and later use it to supply addition domain
knowledge and vocabulary, and to check against compiled user data.
Product and competitive audits
Also in parallel to stakeholder and SME interviews, it is often quite helpful for the
design team to examine any existing version or prototype of the product, as well as its
chief competitors. Doing so gives the design team a sense of the state of the art, and
provides fuel for questions during the interviews. The design team, ideally, should
engage in an informal heuristic or expert review of both the current and competitive
interfaces, comparing each against interaction and visual design principles. This
procedure both familiarizes the team with the strengths and limitations of what is
currently available to users, and provides a general idea of the current functional
scope of the product.
Ethnographic interviews
19.2
It has been experienced that combination of one-on-one interviews and work/lifestyle
observation is the most effective and efficient tool in a designer's arsenal for
gathering qualitative data about users and their goals. The technique of ethnographic
interviews is a combination of immersive observation and directed interview
technique.
Hugh Beyer and Karen Holtzblatt have pioneered an ethnographic interviewing
technique that they call contextual inquiry. Their method has, for good reason, rapidly
gained traction in the industry, and provides a sound basis for qualitative user
research.
Contextual inquiry
Contextual inquiry, according to Beyer and Holtzblatt, is based on a master-
apprentice model of learning: observing and asking questions of the users as if she is
the master craftsman and he interviews the new apprentice. Beyer and Holtzblatt also
enumerate four basic principles for engaging in ethnographic interview:
Context:
Rather than interviewing the user in a clean white room, it is important to interact
with and observe the user in their normal work environment, or whatever physical
context is appropriate for the product. Observing users as they perform activities and
questioning them in their own environment, filled with the artifacts they use each day,
can bring the all-important details of their behaviors to light.
174
img
Human Computer Interaction (CS408)
VU
Partnership:
The interview and observation should take the tone of a collaborative exploration with
the user, alternating between observation of and discussion f its structure and details.
Interpretation:
Much of the work of the designer is reading between the lines of facts gathered about
user's behaviors, their environment, and what they say. These facts must be taken
together as a whole, and analyzed by the designer to uncover the design implications.
Interviewers must be careful, however, to avoid assumptions based on their own
interpretation of the facts without verifying these assumptions with users.
Focus:
Rather than coming to interviews with a set questionnaire or letting the interview
wander aimlessly, the designer needs to subtly direct the interview so as to capture
data relevant t design issues.
Improving on contextual inquiry
Contextual inquiry forms a solid theoretical foundation for quantitative research, but
as a specific method it has some limitations and inefficiencies. The following process
improvements result in a more highly leveraged research phase that better set the
stage for successful design:
· Shortening the interview process: contextual inquiry assumes full day
interviews with users. The authors have found that interviews as short as one
hour in duration are sufficient to gather the necessary user data, provided that
a sufficient number of interviews (about six well-selected users for each
hypothesized role or type) are scheduled. It is much easier and more effective
to find a diverse set of users who will consent to an hour with a designer than
it is to find users who will agree to spend an entire day.
· Using smaller design teams: Contextual inquiry assumes a large design
team that conducts multiple interviews in parallel, followed by debriefing
sessions in which the full team participates. Experiments shows that it is more
effective to conduct interviews sequentially with the same designers in each
interview. This allows the design team to remain small (two or three
designers), but even more important, it means that the entire team interacts
with all interviewed users directly, allowing the members to most effectively
analyzed and synthesized the user data.
· Identifying goals first: Contextual inquiry, as described by Beyer and
Holtzblatt, feeds a design process that is fundamentally task-focused. It is
proposed that ethnographic interviews first identify and prioritize user goals
before determining the tasks that relate to these goals.
· Looking beyond business contexts: the vocabulary of contextual inquiry
assumes a business product and a corporate environment. Ethnographic
interviews are also possible in consumer domains, though the focus of
questioning is somewhat different.
175
Table of Contents:
  1. RIDDLES FOR THE INFORMATION AGE, ROLE OF HCI
  2. DEFINITION OF HCI, REASONS OF NON-BRIGHT ASPECTS, SOFTWARE APARTHEID
  3. AN INDUSTRY IN DENIAL, SUCCESS CRITERIA IN THE NEW ECONOMY
  4. GOALS & EVOLUTION OF HUMAN COMPUTER INTERACTION
  5. DISCIPLINE OF HUMAN COMPUTER INTERACTION
  6. COGNITIVE FRAMEWORKS: MODES OF COGNITION, HUMAN PROCESSOR MODEL, GOMS
  7. HUMAN INPUT-OUTPUT CHANNELS, VISUAL PERCEPTION
  8. COLOR THEORY, STEREOPSIS, READING, HEARING, TOUCH, MOVEMENT
  9. COGNITIVE PROCESS: ATTENTION, MEMORY, REVISED MEMORY MODEL
  10. COGNITIVE PROCESSES: LEARNING, READING, SPEAKING, LISTENING, PROBLEM SOLVING, PLANNING, REASONING, DECISION-MAKING
  11. THE PSYCHOLOGY OF ACTIONS: MENTAL MODEL, ERRORS
  12. DESIGN PRINCIPLES:
  13. THE COMPUTER: INPUT DEVICES, TEXT ENTRY DEVICES, POSITIONING, POINTING AND DRAWING
  14. INTERACTION: THE TERMS OF INTERACTION, DONALD NORMAN’S MODEL
  15. INTERACTION PARADIGMS: THE WIMP INTERFACES, INTERACTION PARADIGMS
  16. HCI PROCESS AND MODELS
  17. HCI PROCESS AND METHODOLOGIES: LIFECYCLE MODELS IN HCI
  18. GOAL-DIRECTED DESIGN METHODOLOGIES: A PROCESS OVERVIEW, TYPES OF USERS
  19. USER RESEARCH: TYPES OF QUALITATIVE RESEARCH, ETHNOGRAPHIC INTERVIEWS
  20. USER-CENTERED APPROACH, ETHNOGRAPHY FRAMEWORK
  21. USER RESEARCH IN DEPTH
  22. USER MODELING: PERSONAS, GOALS, CONSTRUCTING PERSONAS
  23. REQUIREMENTS: NARRATIVE AS A DESIGN TOOL, ENVISIONING SOLUTIONS WITH PERSONA-BASED DESIGN
  24. FRAMEWORK AND REFINEMENTS: DEFINING THE INTERACTION FRAMEWORK, PROTOTYPING
  25. DESIGN SYNTHESIS: INTERACTION DESIGN PRINCIPLES, PATTERNS, IMPERATIVES
  26. BEHAVIOR & FORM: SOFTWARE POSTURE, POSTURES FOR THE DESKTOP
  27. POSTURES FOR THE WEB, WEB PORTALS, POSTURES FOR OTHER PLATFORMS, FLOW AND TRANSPARENCY, ORCHESTRATION
  28. BEHAVIOR & FORM: ELIMINATING EXCISE, NAVIGATION AND INFLECTION
  29. EVALUATION PARADIGMS AND TECHNIQUES
  30. DECIDE: A FRAMEWORK TO GUIDE EVALUATION
  31. EVALUATION
  32. EVALUATION: SCENE FROM A MALL, WEB NAVIGATION
  33. EVALUATION: TRY THE TRUNK TEST
  34. EVALUATION – PART VI
  35. THE RELATIONSHIP BETWEEN EVALUATION AND USABILITY
  36. BEHAVIOR & FORM: UNDERSTANDING UNDO, TYPES AND VARIANTS, INCREMENTAL AND PROCEDURAL ACTIONS
  37. UNIFIED DOCUMENT MANAGEMENT, CREATING A MILESTONE COPY OF THE DOCUMENT
  38. DESIGNING LOOK AND FEEL, PRINCIPLES OF VISUAL INTERFACE DESIGN
  39. PRINCIPLES OF VISUAL INFORMATION DESIGN, USE OF TEXT AND COLOR IN VISUAL INTERFACES
  40. OBSERVING USER: WHAT AND WHEN HOW TO OBSERVE, DATA COLLECTION
  41. ASKING USERS: INTERVIEWS, QUESTIONNAIRES, WALKTHROUGHS
  42. COMMUNICATING USERS: ELIMINATING ERRORS, POSITIVE FEEDBACK, NOTIFYING AND CONFIRMING
  43. INFORMATION RETRIEVAL: AUDIBLE FEEDBACK, OTHER COMMUNICATION WITH USERS, IMPROVING DATA RETRIEVAL
  44. EMERGING PARADIGMS, ACCESSIBILITY
  45. WEARABLE COMPUTING, TANGIBLE BITS, ATTENTIVE ENVIRONMENTS