Getting Started with Online Surveys
About This Article:
This article was created by Wild Apricot, providers of membership management software for associations, nonprofits and clubs. It is designed for the staff and/or volunteers of small membership organizations who are new to conducting online (web-based) surveys.
Your organization has likely asked participants to complete feedback forms after events or sent out periodic surveys to members. But have you conducted any online surveys?
Conducting online surveys:
- Offers members an easy way to provide immediate feedback (automatic data collection speeds the process and allows for anonymity if desired)
- Demonstrates that you are listening to your members/supporters
- Provides opportunities to re-engage members/supporters
What kind of surveys?
We understand that in some cases, there is no substitute for personal outreach and interaction with your members. However, you might want to consider an online survey for:
- quick polls - gaining feedback on, for example: issues, programs, priorities, potential event dates or topics
- gathering post-event feedback
- gauging member satisfaction and promoting member retention. (A White Paper by Larry J. Seibert, Ph.D. of the Loyalty Research Center suggests Using Member Surveys for Retention – by identifying vulnerable members.)
Are online surveys right for you?
While online surveys can be really effective, you need to be sure that this format is right for your audience and that it will enable you to meet your objectives. Here are a few things to consider:
- Will all of your members be able to participate? If a proportion of your members are not regular computer users, you’ll be excluding them from responding. So it’s important to be clear on the nature of your target audience before you choose to do a solely web-based survey.
- Are you willing or able to act on the data or feedback you’re collecting? As an Inc. Magazine blog post notes, "If you are unwilling or unable to act on survey data, then the survey is a waste of your customers' time and an unnecessary distraction for your organization....it's important to remember that an online survey is not just a form – it is a customer [member] touch point, and an opportunity to build better customer [member] relationships.”
Great expectations - what kind of response rates can you expect?
If you are new to online surveys, you’re probably wondering what kind of response you can expect.
While there are no hard and fast rules around what to expect, companies such as SurveyMonkey and Zoomerang suggest “Response rates vary widely depending on a number of factors. For online surveys in which there is no prior relationship with recipients, response rates can be as high as 20% to 30%.”
Of course, there are many variables that can impact your survey response, including:
- The topic
- The survey length
See below for some tips on crafting effective online surveys.
Calculating Your Response Rates:
Here’s a simple way of calculating your response rates:
|# of completed surveys ||= Response Rate |
|# of people contacted |
(Image source: SurveyMonkey)
A healthy response rate means that you can be satisfied that your survey offers an accurate representation of your membership. So, it stands to reason that the larger the number of responses, the more likely you can trust the survey feedback to reflect the general membership’s opinions are represented.
A thoughtful blog post – 12 Common Membership Survey Mistakes – (Dugan & Lopatka) offers some cautionary advice that “a carefully designed survey will provide valuable input...” but “inaccurate results can create more problems than no results at all.” So in the spirit of effective design promoting effective results, here are some tips to keep in mind as you get started:
1. Set clear objectives:
To be sure you are clear on your objectives, ask yourself:
What are you trying to learn?
- What metrics do you need to help you achieve your goals?
- How will you use the data you collect?
- What sample size will you need to ensure your objectives are met?
2. Identify the Right Online Survey Tool:
There are a number of online survey tools out there. Most are hosted in the Cloud so they can be accessed easily via a web browser. These tools automatically collect the survey response and offer various ways of analyzing and sharing the results.
There are many different online survey tools – offering various bells and whistles and customization options with varying price points. While we haven’t tried them all, Idealware has created a great article - A Few Good Online Survey Tools - (available in our Membership Knowledge Hub) that offers summaries of some of the most popular survey tools. In this Idealware article, Eric Leland offers some great advice for choosing the right survey tool:
“Start by thinking about your needs. If you’re just looking to get your feet wet with a quick survey, one of the free or low cost tools will probably work fine. In fact, a more sophisticated survey package is likely to just be considerably more difficult for you to use. On the other hand, if you’re looking for survey software to support rigorous research, the more advanced packages are more likely to have the features you need. Whichever type of package makes sense for you, take advantage of the free versions to take the tools for a trial run. While many of the advanced features are not available in the free trials, the vendor may be able to give you access to these features as well.”
If you are new to online survey tools, you might be a little intimidated. But many of the providers offer resources to help you with both question development (e.g., question types and structure) as well as sequencing and layout. (See Resources below for a sample SurveyMonkey resource document.)
3. Get the Timing Right
There are a number of issues around timing that you need to consider. And as with all surveys – online or off – you need to understand and consider your audience (e.g., your members or supporters) to get the timing right for them. Here are some timing issues to consider:
- What is the optimal completion time? – e.g., how many questions are optimal and how long will it take to complete?
- When should it be scheduled? Finding the optimal time period for responding may improve your response rate. So you need to avoid busy seasons for your members (e.g., tax time for accountants, end-of year crunch, etc.); and work around religious and seasonal holidays as well.
- When do you need the results? If the data is time sensitive, be sure to provide enough time for participants to respond and for data analysis as well.
- What day of the week is best to launch the survey? In terms of timing, research suggests that you’ll get the best response when you send email invitations for surveys on Monday, Friday and Sunday.
Scheduling of numerous surveys:
In addition, if you plan to conduct a number of surveys over the year, you need to give some thought to how these will be scheduled to avoiding “survey fatigue.” In a recent blog post, Greenfield Services Inc. warned about “Beating Survey Fatigue.” While it’s great to show that you are listening to members, you don’t want to overdo it with too many surveys – set up a schedule (e.g., quarterly) that works for your members.
4. Keep it short and simple:
You won’t be surprised to hear that you’ll probably get a better response to a brief survey. When participants see a well thought-out survey with short, succinct and easy-to-answer questions, they will be more likely to complete it and hit the “submit” button. Here are a few tips to help you keep it simple:
- Start by explaining the goals of the survey and offer clear instructions
- Avoid using jargon, abbreviations, technical terms, and complex words. Try to use simple, commonly used language. For example, use the word “work” instead of “employment” and “about” instead of “regarding,” etc.
- Use a simple sentence structure – e.g., with a single subject and verb – and avoid combining two questions into one.
- Give participants specific choices (e.g., yes, no, rating scale, multiple choice) to make the questions easier to answer. And choosing the right question type is also important in ensuring you get the data you need.
- Think about phrasing questions using closed versus open-ended questions. Closed questions (e.g., with multiple choice answers, true/false and attitude scales) are easier to answer and interpret than open-ended questions. Open-ended questions DO allow participants to offer their own opinions (if feedback is what you are after), but these are harder to analyze or interpret.
- Save any personal or demographic questions until the end of the survey.
- Always end with a thank you for participation and clear instructions on how to complete the survey and if/how the results will be shared.
5. Proof it and test it:
Once you’ve drafted the survey, be sure to have someone proof read it to be sure you haven’t missed any typos (always very distracting). You should also test it – to be sure the content and format will offer the feedback or data you need to meet your objectives.
Initially, you should try to find a couple of folks who can offer an objective opinion. Next you should field test it on a small group of members (e.g., a few of your Board Members, other volunteers etc.). Be sure that field testers provide specific feedback, for example on:
- Was the survey’s objective clear?
- Were the questions clear and easy to answer quickly?
- Did the question formats (e.g., multiple choice; ranked; rating scale, etc.) offer effective metrics?
- Is the survey too long?
Gather feedback from the testers on the actual survey and also take a look at the results you receive, to be sure your format and questions will be able to deliver on your survey objectives.
You should also take the opportunity to test your email invitation – if that is the method you plan to use to promote and/or send out your survey. Ask the test respondents to rate the email and its subject line too (which is very important and can skew open and completion rates).
6. Promote it:
Promotion is key to achieving a high response rate in order to gain a true representation of your members’ viewpoints. There are a number of ways you can promote your online survey, including:
- Personalized emails – with survey link
- Posting the survey on your website and promoting it (e.g., on your home page or in your “News”)
- Posting on Facebook (or embedding on your Facebook page)
- Including a note and the link in your e-newsletter
- Writing a blog post or forum post with the link and timelines
- Including a note in any mailed correspondence (with URL and details about timelines and any incentives)
- Reminder emails
Ideally, you should consider multiple communication methods to launch the survey as well as to remind folks to participate.
7. Share the results:
While you are conducting the online survey to meet your own specific objectives, one of them is likely to ensure that you are listening to your members. So it’s important to share the results with the participants and your membership in general. This offers both an opportunity to thank those who participated and to leverage the results for additional member communications.
For example, we recently conducted a Membership Renewal Survey among our blog readers and Facebook followers. We then shared the results through a blog post and infographic and we included our survey findings in a Membership Renewal Guide as well.
8. Rinse & Repeat:
If you are creating a member survey that will be repeated regularly (e.g., quarterly or yearly) to track one or a number of data points or issues on a regular basis, you need to ensure that you repeat the exact questions in order to accurately track results.
One of the 12 Common Membership Survey Mistakes suggested by Dugan & Lopatka was “Not resurveying.” As they suggest, “a survey is a still photograph, not a motion picture, and it reflects opinions at only one point in time. Your first survey is a just a benchmark by which your subsequent surveys should be compared. …If you are really interested in opening the lines of communication with your members, don’t overlook doing annual or even semi-annual surveys.”
We hope this Getting Started Guide has been helpful, but if you want more information, check out the blog posts and additional resources below.
Here are some Wild Apricot blog posts with tips for online surveys:
- Using Member Surveys for Retention - Larry J. Seibert, Ph.D., Loyalty Research Center (ASAE Knowledge Center- White Paper): http://www.asaecenter.org/Resources/whitepaperdetail.cfm?ItemNumber=37001
- 12 Common Membership Survey Mistakes - Dugan & Lopatka http://www.duganlopatka.com/resources/articles/published-articles/39-not-for-profit-articles/251-survey-mistakes-article
- Smart Survey Design (PDF) – SurveyMonkey http://help.surveymonkey.com/euf/assets/docs/pdf/SmartSurvey.pdf
- Survey Fundamentals (PDF) - A guide to designing and implementing surveys (not just online surveys) – University of Wisconsin-Madison (this is very comprehensive and includes a “Checklist for Effective Questionnaires”) http://oqi.wisc.edu/resourcelibrary/uploads/resources/Survey_Guide.pdf
- How to Use Online Tools for Customer Surveys - Inc. - http://www.inc.com/guides/2010/07/how-to-use-online-tools-for-customer-surveys.html