Bowling Green State University

Statement of purpose. Prior to conducting a selection, training, survey, or any sort of applied research project, a myriad of methodological and data analytic issues must be addressed. Take for instance an I-O psychologist conducting a 360-degree assessment. How should feedback providers be chosen? How many respondents are needed to produce a valid report? What types of information should and should not be presented to the feedback recipient (e.g., standard deviations)? How should the data be presented (e.g., graphical or tabular format)?

The quality and success of our research are directly impacted by our answers to these types of questions. Although one "correct" answer most likely does not exist for a methodological inquiry, we are still charged with making educated decisions. The objective of this column is to try to bring to the forefront information that, hopefully, will help with this charge. To do so a guest columnist and/or I will attempt to synthesize extant research and/ or present interviews conducted with "seasoned" practitioners on methodological issues.

Due to space (and time) limitations, each column will address just one applied methodological issue. This first column of Informed Decisons will focus on a salient survey research issue. Specifically, methods to facilitate survey response will be discussed. In order to make the column as relevant and useful as possible, I encourage you to email me at (rogelbe@ bgnet.bgsu.edu) or write me (Department of Psychology, Bowling Green State University, Bowling Green, Ohio 43404) with ideas or to let me know if you are interested in authoring a future Informed Decisions column.

Bowling Green State University

The advantages of the mail questionnaire, such as its low cost and ability to reach many geographic regions, have made it a widely used method for assessing the attitudes and opinions of employees. However, much to the chagrin of survey researchers, response rates appear to be on the decline (Schwarz, Groves, & Schuman, 1997). In this paper, we discuss why and how to increase mail survey response rates.

Why Should We Facilitate Survey Response?

There are several reasons why we should be concerned when individuals do not respond to our surveys. First, nonresponse can introduce bias into our findings. Specifically, generalizations from obtained data to a population are limited when nonrespondents differ systematically from respondents on survey relevant variables. Remember though, nonresponse in and of itself does not necessarily indicate bias (Rogelberg & Luong, 1998). Nonresponse may also result in not having enough data to run certain analyses in a statistically powerful manner (e.g., a department by tenure analysis can not be conducted). Finally, nonresponse may diminish the perceived credibility of our findings. Specifically, managers and employees alike may not take our recommendations seriously if they are based on the opinions of only a handful of individuals, even if those individuals are representative of the population. Given these problems and the prevalence and utility of the mail questionnaire, a myriad of studies have been devoted to examining the different techniques that are used to stimulate response rates.

Research on Response Facilitation Techniques

Typically, research on response rates to the mail questionnaire is atheoretical and appears to be a post hoc tag-on to some other research question. Furthermore, response facilitation techniques are often studied in isolation of one another such that the combined effects of techniques are relatively unknown. By synthesizing the extant quantitative and qualitative reviews (Fox, Crask, & Kim, 1988; Heberlein & Baumgartner, 1978; Kanuk & Berenson, 1975; Linsky, 1975; Yammarino, Skinner, & Childers, 1991; Yu & Cooper, 1983) as well as examining some recent or excluded research, we are able to present a discussion of research substantiated ways to facilitate survey response.

Techniques to Facilitate Response

In Table 1 we have classified response facilitation techniques into three categories. In the first category are techniques that we advise the use of, because they have been found to increase response rates in most studies. Mixed evidence exists regarding the effectiveness of the techniques in the second category; some researchers have found that they have positive effects on response rates, whereas others have found no effects. Because they have not been found to negatively affect response rates, we also suggest that you employ these techniques. Finally, we have placed in the third category little, or as-yet-unresearched, factors that we feel warrant attention because they may affect response rates.

Repeatedly contact your participants. Before sending your surveys, notify your potential participants (e.g., via email) that they will be receiving a survey in the near future. Interestingly, a pre-survey notification has been shown to be as effective as reminder notes in facilitating response. After an appropriate amount of time (10 days to 3 weeks) has passed since you first send the actual surveys, remind individuals in your data sample via telephone or mail to complete and return their surveys.

To prevent individuals who have completed the survey from being inundated with reminders, you may want to consider using a coding system or have individuals send a separate postcard, with their name, indicating that they have returned their survey (the postcard should be stamped and included with the original survey). Some research suggests sending out an additional copy of the survey with the 2^nd or 3^rd reminder notes, if applicable.

Although this may appear counter-intuitive, when contacting potential respondents, some data suggests not including a deadline date for when surveys must be completed and returned. Deadlines may increase the speed of returns from those who would likely respond anyway, but it may cause those who are less likely to respond to not do so as the deadline date approaches. If you must give a deadline, we suggest explicitly stating that late surveys will be accepted.

Entice and impress your participants. Providing incentives can significantly increase your responses. If you are uncomfortable providing cash, inexpensive items such as pens, key chains, or coupons for food/drink can increase responses as well. The main issue to keep in mind, though, is to include the incentive with the survey. This has been found to work better than the promise of a reward.

Include some sort of appeal in the cover letter. Ensure individuals that their participation is important. Tell them, for instance, that their participation is beneficial to research, or that this is an opportunity for them to express their opinions. In other words, make them feel that both they and the survey are important.

Be personable. By personalizing your surveys, participants may feel that their participation is more important. This can be done in many ways. The techniques that have been found to be most effective include using postage stamps (versus metered postage), individualizing salutations (e.g., address the participant by his/her name), and handwritten (instead of mimeographed) signatures. When you do personalize your surveys, be sure to explicitly guarantee your participants that their responses will be kept confidential. For highly sensitive survey topics, personalization should probably be avoided.

Don’t forget the survey itself. Try to keep the survey to a reasonable length. Ask yourself why you need each question and eliminate those questions that are not germane to your project. Periodically, attempt to downsize your survey’s content (you may even want to involve your internal customers in these efforts). Remember that a survey does not have to measure and assess all possible topics. Also, be careful not to use the "measurement for measurement sake" approach to survey design. Please note that we are not arguing against the use of a scale of items to measure a construct.

Given that interest in topic is a good predictor of one’s decision to participate in a survey project, to the extent you can, try to include content that will be seen as interesting to your potential participants. How you order your questions may also affect responses (e.g., research suggests placing the more interesting and easy questions first and demographic questions last). Finally, how your questionnaire looks may affect whether or not an individual will respond. For instance, your survey should be easy to read (e.g., do not use a font that is too small) and easy to complete (e.g., provide response scales that make sense). We also suggest that you consider, at times, alternatives to black ink on white paper.

Miscellaneous factors that may affect responses. In this last category, we discuss several variables, which may affect survey response. These factors have not, however, been extensively researched in the literature. Based on our research and experience, we suggest providing respondents with survey feedback once the project is completed. Besides general feedback of the survey’s findings, action plans and follow-up work should be subsequently shared (even explain when no action is to be taken). In other words, do not abandon your participants once you have gotten what you want. The likelihood that they will participate in future surveys depends on their experience with yours. Also, keep in mind that individuals probably get surveys from multitudes of sources. Hence, carefully consider how much and often you should be surveying your employees. Consider the use of random sampling as opposed to census surveys. Also, consider creating a survey registry within your organization to manage survey efforts.

Summary

Based on the extant literature, we have attempted to summarize and translate the vast research findings on facilitation techniques into practical suggestions, which we hope will help increase responses to your survey. Our main advice is to take all these suggestions into consideration when you design your process and tools. A total design approach will more likely increase your response rate than any single technique in isolation. Please contact us (luong@bgnet.bgsu.edu) with any comments on this article or any ideas you may have for facilitating survey responses.

Fox, R.J., Crask, M.R., & Kim, J. (1988). Mail survey response rate: A meta-analysis of selected techniques for inducing response. Public Opinion Quarterly, 52, 467–491.

Heberlein, T.A. & Baumgartner, R. (1978). Factors affecting response rates to mailed questionnaires: a quantitative analysis of the published literature. American Sociological Review, 43, 447–462.

Kanuk, L., & Berenson, C. (1975). Mail surveys and response rates: A literature review. Journal of Marketing Research, 12, 440–453.

Linsky, A.S. (1975). Stimulating responses to mailed questionnaires. Public Opinion Quarterly, 39, 82–101.

Rogelberg, S.A. & Luong, A. (In Press). A review and guide to handling survey nonresponse. Current Directions in Psychology.

Schwarz, N., Groves, R.M., Schuman, H. (1998). In D.T Gilbert, S.T. Fiske, & G. Lindzey (Eds.). The handbook of social psychology (4^th ed., pp, 143–179). New York: McGraw-Hill.

Yammarino, F.J., Skinner, S.J., & Childers, T.L. (1991). Understanding mail survey response behavior: A meta-analysis. Public Opinion Quarterly, 55, 613–639.