Surveying is the systematic process of gathering information from a sample of entities for the purpose of examining or describing someone or something. Surveys play a critical role is gathering actionable data from students, faculty, and staff for assessing the effectiveness of college programs and services and for advancing the College’s mission and priorities.

Questionnaires

When most people think of surveys, they think of questionnaires, or forms developed to gather information about individual’s experiences, perceptions, or plans for the future. In fact, the terms survey and questionnaire are often used interchangeably, even amongst seasoned researchers. Today, most questionnaires are administered electronically using specialized software designed for this purpose. They typically produce quantitative data that can be analyzed using statistics.

While questionnaires can be a useful assessment tool, their utility is limited in terms of measuring student learning and teaching effectiveness. Data gathered from questionnaires are considered indirect evidence and are best used to complement data gathered from direct assessments such as exams. Another drawback of questionnaires is that it is becoming increasingly difficult to get individuals to respond to them. Consequently, data derived from surveys are becoming harder to interpret and make valid inferences from.

Designing and launching a survey is relatively easy and inexpensive. However, crafting a well-designed survey that will capture the types of information researchers will find useful requires careful planning and technical expertise.  Researchers need to carefully consider the wording of questions, the questionnaire design, the target population, the sampling frame, data collection methods, cost, timing, non-response, and post-collection processing of data and analyses. The Office of Institutional Effectiveness is available to assist researchers with developing a well-designed survey instrument that will help them make valid inferences.  Please see contact the Office of Institutional Effectiveness for the survey policy and for additional questions.

Focus Groups

Focus groups are a popular technique used by qualitative and mixed-methods researchers to gather feedback from individuals. Focus groups are interviews of small (6-10), often homogeneous groups of people, who have similar interests and experiences (e.g., students in a club). They produce more nuanced data than are typically captured in questionnaires, and for this reason, they are sometimes used to complement other assessment methods.

While focus groups can produce valuable information, they are several downsides to them. For example, it is often very difficult to recruit participants as they typically require an hour or more of a person’s time. Individuals can be incentivized to participate by offering small stipends, but resources often do not allow for this.  Another disadvantage is that focus groups are more resource intensive to conduct than questionnaires.  They can take a significant amount of time to organize and the data they produce often takes a long time to analyze.

Perhaps the most critical element of a successful focus group is the moderator.  Moderators need to be relatively familiar with the topic being discussed, but they need to remain objective when facilitating the group. Their job is to elicit feedback and to keep the conversation flowing between everyone in the room.  It is best to have a respected individual from outside the unit to serve as the moderator.

Another important consideration to keep in mind is that confidentiality cannot be assured in focus groups.  Thus, if there are very sensitive topics under discussion, other data-gathering techniques may be preferable.

Interviews

Interviews are one-on-one conversations between individuals. Like focus groups, interviews produce qualitative data that can be used to assess programs, departments or other entities. The chief advantage of interviews are their potential ability to produce deep insights into the phenomena being assessed that are not likely to occur using other assessment techniques. Interviews also are less susceptible to bias than focus groups.

However, like focus groups, interviews are resource intensive.  Since each interview is conducted in a one-on-one setting, it can take many hours to interview enough participants to obtain useful results.

For additional information about interviews and focus groups, please see the following article.

While focus groups can produce valuable information, they are several downsides to them. For example, it is often very difficult to recruit participants as they typically require an hour or more of a person’s time. Individuals can be incentivized to participate by offering small stipends, but resources often do not allow for this.  Another disadvantage is that focus groups are more resource intensive to conduct than questionnaires.  They can take a significant amount of time to organize and the data they produce often takes a long time to analyze.

Perhaps the most critical element of a successful focus group is the moderator.  Moderators need to be relatively familiar with the topic being discussed, but they need to remain objective when facilitating the group. Their job is to elicit feedback and to keep the conversation flowing between everyone in the room.  It is best to have a respective individual from outside the unit assessed to serve as the moderator.

References: