Many educators are familiar with a variety of surveys and questionnaires for the office and the classroom. These can include formal responses about work procedures or informal surveys such as where to hold a social event. Designing survey questions for research, however, requires strategic planning to make the survey effective. Clarity in question design, effective procedures for garnering responses, and usefulness for continuing research are all important facets necessary at the earliest planning stages. This article will focus on the planning stages of academic survey research: choosing a theme, designing questions, and gathering responses.
Choosing a Theme
Surveys can be used in a variety of ways at different stages of a larger project. Surveys can often help define a project, give focus to a project, or introduce new avenues of research. If a survey is too broad, the researcher may find themselves overwhelmed with responses they cannot collate in any meaningful way. A clear statement of purpose helps to give your survey focus. As Wolcott (2001) points out in reference to research papers, “You have your purpose well in mind when you can write a critical, clear, and concise sentence (p. 9).” He further points out that it does not have to be an eloquent statement in the initial draft because the goal is to get yourself writing so that your project can begin (Wolcott, 2001).
For the purposes of this column, I will use my own example of a survey intended as part of a larger research project about the experiences of university lecturers in Japan and how they felt about their academic careers. After brainstorming dozens of possible questions about academic careers, I decided to focus on professional development (PD) activities. At this point, I needed a clear statement of purpose:
This survey focuses on participation in professional development for part-time instructors in the Japanese university system. Participants should have part-time, non-tenured status. Professional development includes reading professional journals, research, publication, presentations, and conference participation.
The statement clarifies the focus of my survey and the participants I am seeking, and clearly defines the subject of my research.
Designing Questions
Once I had a clear statement of purpose, I designed a twenty-question survey. The first set of questions gathered biographical information such as gender, teaching experience, and work hours. In asking for biographical data (e.g., nationality, ethnicity), you should consider both privacy and sensitivity issues. It is important to consider whether or not such questions are necessary. The National Council of Teachers of English has a list of guidelines and suggestions for discussing issues of gender (DesPrez et al., 2018). For my survey, providing gender was optional. I also did not include questions about race/ethnicity because it was not relevant to my line of research. I did include questions about nationality as I was interested in conducting further research into native speakerism and implicit regional bias in Japanese hiring practices.
The second set of questions focused on PD. This included questions about membership in professional organizations, subscriptions to journals, hours spent on research, and access to grants and funding. Also, respondents were asked to assess on a 5-point Likert scale whether or not they felt the need to actively participate in research at their place of employment or as part of their personal career goals.
For academic research, it is important that the researcher determine what information they need to gather and how to make the survey questions as clear as possible. Using this question as an example, let’s look at the process of refining this question to make it more effective: What professional development (PD) activities do you participate in?
This was an open-ended question aimed at gathering short answers. With help from a mentor, I realized the question was too broad, causing confusion as to what constituted PD. After making a list of possible PD activities, I narrowed them down to (1) reading professional journals, (2) writing for publication, (3) doing academic research, and (4) conference participation. Next, I carefully crafted precise questions that could be easily presented in multiple-choice formats and would garner more responses from time-strapped participants. For example, my new questions moved from open-ended to quick concise drop-down menu options:
How much time do you devote to PD (reading, writing, research) in your field?
1. Less than one hour a week
2. One to three hours a week
3. Four to six hours a week
4. More than six hours a week
Another important reason to choose multiple-choice options for surveys is that many programs automatically create charts or bar graphs to display your research. When doing the final written paper, having ready-made graphic summations of research data can “broaden the appeal for those who appreciate having data presented in non-textual form” (Wolcott, 2001, p. 19).
The final question was open-ended so respondents could answer freely about their concerns as part-time instructors. I found that most of the respondents appreciated having at least one open-ended question in which they felt free to raise concerns. These responses provided areas of insight not considered in my original research plan.
Gathering Responses
When using a survey, the more responses you receive, the better. It is crucial, therefore, that your survey length, time frame, and avenues for gathering responses are well-orchestrated. First, the less daunting the list of questions, the more likely you are to get responses. For a survey of volunteer graduate students, for example, you may be able to do a longer survey with a variety of open-ended questions. For an online survey in which you want to elicit as many responses as possible, it is better to keep the survey length to 10 minutes maximum.
Second, a set time frame works best. A survey that is out there for too long is quickly forgotten. Also, your research may be time-sensitive if used for a publication. Mine was available for three weeks—the majority of the thirty-one responses I received came within two days.
Third, determine how you will gather responses. For my research, responses were primarily solicited online through a university posting board: the Association of Foreign Wives of Japanese, which has a subgroup for educators, Online Teaching Japan, and English Teaching Japan, all of which have active Facebook communities. The survey was conducted in English, although it was open to all language instructors. Professional organizations may allow you to post your survey in newsletters or journals, but be sure to investigate thoroughly all guidelines and policies first. Even after reading guidelines, courtesy goes a long way in gaining support for your project. One online group moderator encouraged me to give a personal introduction before posting a survey request and asked for follow-up reports of completed research. Not only did this help generate responses, it also helped me connect with another researcher in a similar field.
You should include brief information on who you are, your research interests, and the intentions of your survey. It is also important to provide contact information, for example an email address included directly after the statement of purpose.
Final Thoughts
It is strongly recommended that you get a second pair of eyes to look over your questions. Perhaps they are not as clearly understood by another person as they are by you. Perhaps it takes longer to complete the survey than you intended. No survey will be perfect, but having a second opinion certainly helps to make it more effective. The planning stages of determining your theme, designing questions, and gathering responses are crucial to creating an effective and concise survey that furthers your research.
References
DesPrez, E., Baca, D., Blackburn, M., Chen, A., Coles, J. A., Domínguez, M., Ife, F., Pennell, S. M., & Shelton, S. A. (2018, October 25). Statement on gender and language. National Council of Teachers of English. https://ncte.org/statement/genderfairuseoflang/
Wolcott, H. F. (2001). Writing up qualitative research. Sage.
Winifred Lewis Shiraishi is an educator in the English Immersion Program/Bilingual Program at Katoh Kindergarten in Numazu. She has taught English in Japan since 1998. She holds a Master’s degree in History from the University of North Carolina at Greensboro. Her current research focuses on professional development for educators.