You are here

Cognitive Interviewing
Share
Share

Cognitive Interviewing
A Tool for Improving Questionnaire Design



September 2004 | 352 pages | SAGE Publications, Inc

"As both an academic instructor in questionnaire design and a research design methodologist for the federal government, I feel this book is very timely, useful for students and practitioners, and unique in its use of real world practical examples that most everyone can relate."
—Terry Richardson, General Accounting Office

"The combination of theory and practical application will make this a useful book for students as well as professionals who want to learn how to incorporate cognitive interviewing into the questionnaire design process."
—Rachel Caspar, RTI International

The design and evaluation of questionnaires—and of other written and oral materials—is a challenging endeavor, fraught with potential pitfalls. Cognitive Interviewing: A Tool for Improving Questionnaire Design describes a means of systematically developing survey questions through investigations that intensively probe the thought processes of individuals who are presented with those inquiries. The work provides general guidance about questionnaire design, development, and pre-testing sequence, with an emphasis on the cognitive interview. In particular, the book gives detailed instructions about the use of verbal probing techniques, and how one can elicit additional information from subjects about their thinking and about the manner in which they react to tested questions. These tools help researchers discover how well their questions are working, where they are failing, and determine what they can do to rectify the wide variety of problems that may surface while working with questionnaires.

Cognitive Interviewing is ideally suited as a course text for advanced undergraduate and graduate research courses across the social sciences. Professional researchers and faculty in the social sciences, as well as practice fields such as medicine, business, and education, will also find this an invaluable reference for survey research. There is no other book on the market that covers cognitive interviewing as applied to questionnaire design.


 
ORIENTATION AND BACKGROUND
 
Introduction to Cognitive Interviewing
What is (and isn't) cognitive interviewing?

 
A broader perspective: The cognitive testing process

 
 
Setting the Stage: First Principles of Questionnaire Design
Questionnaire-related (and other) sources of error in self-report surveys

 
Why do survey questions produce response error? A sociolinguistic perspective

 
How can we avoid problems in survey questions?

 
If we are proficient designers, why do we need to test questions?

 
Chapter summary

 
Exercise: Question evaluation

 
 
The CASM Approach to Questionnaire Design
Origins of CASM

 
Applications of Basic CASM Research

 
Cognitive interviewing as Applied CASM Research

 
Chapter summary

 
 
Cognitive Interviewing in Practice: Think-Aloud, Verbal Probing, and Other Techniques
Think-Aloud interviewing

 
Verbal Probing Techniques

 
Concurrent versus retrospective interviewing

 
Choosing between think-aloud and probing techniques

 
Vignettes, card sorts, and field-based probes

 
Chapter summary

 
 
THE INTRICACIES OF VERBAL PROBING
 
Developing Standard Cognitive Probes
Examples of cognitive probing

 
Logical and structural problems

 
A systematic approach to probe development: The Question Pitfalls Model

 
Chapter summary

 
Exercise: Using the QAS to develop probe questions

 
 
Beyond the Standard Model of Verbal Probing
A classification of probe types

 
Proactive versus Reactive probing

 
Standardized versus free-form probes

 
Chapter summary

 
Exercise: Emergent probing

 
 
A Further Perspective: Probing as Expansive Interviewing
A broad view of cognitive interviewing

 
The Ethnographic Interview as an alternative perspective

 
Merging Cognitive and Expansive interviewing

 
Chapter summary

 
 
Avoiding Probing Pitfalls
Are we in danger of finding problems that don't exist?

 
Practices that avoid artificial problems

 
Chapter summary

 
 
THE COGNITIVE TESTING PROCESS
 
Training of Cognitive Interviewers
Who makes a good cognitive interviewer?

 
Technical background of the cognitive interviewer

 
How should cognitive interviewer training be accomplished?

 
Continuing education in cognitive interviewing

 
Chapter summary

 
 
Planning and Conducting Cognitive Interviews
Fitting cognitive testing into the overall design sequence

 
Preparing for the interview: Subject recruitment

 
The interviewing process

 
Logistic issues in the cognitive interview

 
Chapter summary

 
 
Analyzing and Documenting Cognitive Interview Results
Characterizing cognitive interview outcomes

 
The analysis of think-aloud interview results

 
Analysis of the probed interview

 
Persistent analysis issues

 
Chapter summary

 
Exercise: Analyzing cognitive interviews

 
 
OTHER ISSUES AND TOPICS
 
Special Applications of Cognitive Interviewing
Adjusting to survey administration mode

 
Cognitive testing of sensitive questions

 
Cognitive interviewing and establishment surveys

 
Testing questions on attitudes and opinions

 
Interviewing across the age range

 
Testing non-questionnaire materials

 
Chapter summary

 
 
Evaluation of Cognitive Interviewing Techniques
Theoretical perspectives

 
Empirical evaluation of cognitive interviewing

 
Are we evaluating the right outcome?

 
Limitations to cognitive interviews

 
Chapter summary

 
 
Beyond Cognitive Testing: Affiliated Pretesting Methods
Expert Review

 
Focus Groups

 
Behavior Coding

 
Reinterview surveys

 
How do pretesting methods compare?

 
Chapter summary

 
 
Recommendations and Future Directions
Twelve recommendations for cognitive interviewing practice

 
Future directions for cognitive interviewing

 
A final case study

 
In conclusion

 
 
References
 
Appendix 1. Sample cognitive testing protocol
 
Appendix 2. Example

“As both an academic instructor in questionnaire design and a research design methodologist for the federal government, I feel this book is very timely, useful for students and practitioners, and unique in its use of real world practical examples that most everyone can relate.”

Terry Richardson
General Accounting Office

“The combination of theory and practical application will make this a useful book for students as well as professionals who want to learn how to incorporate cognitive interviewing into the questionnaire design process.”

Rachel Caspar
RTI International

"The book has definite strong points. After reading Willis's book, I feel more comfortable about developing questions for customer surveys. It's a great reminder that people being surveyed may need more specific questions to provide an accurate response in the cognitive interview."

Rhonda Lunemann
Journal of the Society for Technical Communication

A great book! A MUST for everyone pretesting questionnaires!

Dr Eva Zeglovits
Dept of Methods in the Social Sciences, University of Vienna
March 12, 2012

This title is also available on SAGE Research Methods, the ultimate digital methods library. If your library doesn’t have access, ask your librarian to start a trial.