by Arthur Prokosch
former Online Services Manager at TSNE

Online surveys have emerged as an efficient way for nonprofits to gather data from their constituents, donors - and even their peers - entirely through the Web. At their core, these systems include tools for defining and customizing survey questions, and tools for tracking and downloading the responses. Most modern systems, such as SurveyMonkey or Zoomerang, have additional useful features, including templates, tools for publicizing your surveys, or the ability to filter and search though the results. For recommendations on which online survey system to choose, articles at TechSoup and Wikipedia have good lists of software and features.

This article, however, assumes you have already chosen a system. What remains is much more important - and much more often overlooked. The success of an online survey will depend more on how you define what you’re looking for, how you relate to your target audience and how well you execute the details. This article describes these factors and gives you tips to follow to ensure you get the most, best data possible - and while we’re at it, the most satisfied respondents, too.

I. Engage Your Audience

Be realistic about how motivated and available your intended survey-takers are. If you’re not sure, take the time to run a test with 10% of your list before investing more resources.

With an audience that’s already motivated, providing an incentive for completion could be optional. Otherwise, be prepared to offer a knick-knack or white paper download, or a raffle entry for a larger prize. While responses from only the most motivated constituents can be informative, you want to hear from as wide a spectrum of your audience as possible.

If your survey software allows, break up your survey into multiple pages if it’s longer than two to three screenfuls. Be sure to indicate “page 1 of N” at the top, so that your respondents won’t get intimidated and give up, and can set aside an appropriate chunk of time. This has the added benefit of saving the responses on any completed pages, even if respondents do give up partway through. Also be conscious of length: if you go beyond three pages of this size, you’ll get fewer responses, many of which will be incomplete. Be prepared to cut survey questions mercilessly.

In any case, it is essential to arrange questions in such a way that maximizes engagement and the amount of useful data you get when respondents abandon the survey. To do this:

1. Be sure to have introductory text at the beginning of the survey to provide context and assurances of privacy.
2. Ask a couple of open-ended questions early on, to make sure the respondent feels heard. (But be careful not to overdo this; an entire page of text boxes will scare off most respondents.)
3. Group questions together by subject. Make sure that if there are multiple questions for which some respondents would select “N/A,” these questions are next to each other and can be skipped together.
4. Ask for demographic information (name, address, income,...) and any more personal questions that might make the respondent uncomfortable at the end of the survey. Hopefully they’ll be more at ease with the survey by that point; if not, you’ll still have their earlier answers.
5. Make sure that the “done” message or “landing page” of the survey thanks respondents, reminds them of the importance and privacy of their responses, and if applicable, asks them to refer friends to the survey as well.

II. Elicit Meaningful, Accurate Data

As you plan your survey, be as clear and specific as possible about what it is that you’re looking for. Write it down. Run it by coworkers. As you create and test the survey, refer back to this purpose to make sure the questions you ask will give you data that meets it. If you don’t, you risk not only vague questions and meandering surveys, but useless data.

To be sure that the questions you ask will give you the data you want, you need to ask them in the right format. There are two general goals here: first, for your respondents to answer on topic, and second, for the answers to be stored in a way that allows as much summarization and analysis as possible.

Closed Question Types

In terms of online surveys, a closed question type is one where instead of typing in a response, the respondent clicks an answer that’s already visible on their screen. Because it’s quicker and easier both for the respondents to answer these questions, and for the preparer to graph, total and manipulate the results, they get used more often than open-ended questions. Note: for multiple-choice questions, be sure to include options like "don't know," "N/A," and/or "other" as appropriate.

• Multiple choice/choose one is the most restrictive question type, but that makes it the easiest to analyze. The subtypes below illustrate the range of useful information you can elicit in this format. Sometimes you’ll have a choice between presenting these as a drop-down menu versus “radio buttons” that display at all times - go with the latter for ease of use if at all possible.
  • Subtype: “Please select the option that best describes…” (but consider “choose multiple” if many of your respondents won’t fit into only one box)
  • Subtype: A rating scale which allows the respondent to indicate how much they agree with an adjective or statement; best with 5 options, without labels other than the extremes (e.g., 1=strongly disagree, 5=strongly agree)
  • Subtype: Never/Rarely/Sometimes/Often/Always (but it’s better to use more meaningful labels, such as “Every Day” instead of “Often,” which the respondent may interpret differently than you intend)
  • Subtype: Numerical ranges, such as 0-1 / 2-4 / 5-9 / 10+ (but be sure that the ranges don’t overlap - people who want to answer "5" will be confused if their options include both "2-5" and "5-10"). You also get more accurate results when ranges start on a round number (i.e., don’t use 101-200 as an option; you’ll get fewer errors using 100-199 instead).
  • Subtype: Yes/No questions allow you to require the respondent to click either Yes or No, so you can be sure of their response. Use this instead of a built-in "Yes/No" option that uses a checkbox - otherwise, you’ll be left scratching your head as to whether respondents chose to leave the answer unchecked, or if they didn’t see the question.
• Multiple choice/choose many is a good alternative when you’re looking for a full spread of responses, or when it becomes obvious that a “choose one” question won’t elicit accurate answers. The downside is that you can’t compare the answers as easily - they won’t sum to 100%, and generally it will be more difficult to make comparisons between answer choices (because “both” will have been possible). These generally appear as checkboxes on the survey.
  • Best for: Any open-ended question with "Check all that apply" appended, and a clear set of possible answers

Deciding Between These

Let’s say you’re asking for a respondent’s job role. While it’s possible that the respondent might be shared between two departments, most of the time that will be distracting noise in data analysis. It is almost always better to let the respondent make the determination of which one is most relevant. Thus, phrase the question as “Please select the box that most appropriately describes your position,” and use “choose one.” Only when a full inventory is necessary should you use “choose many” – for example, “Which of the following services would you be interested in?”

Page:  1 | 2  Next

Keep up-to-date on our latest articles - sign up for the TSNe-Bulletin, a monthly e-Newsletter providing tips and ideas to help you strengthen your nonprofit’s impact with and for the communities you serve.