After an assessment is conducted using one or more of the methods described in Step 3, the penultimate step in the assessment cycle is analyzing results and deriving meaning from them. In this step, we are attempting to answer the question, “Are students meeting the expected student learning outcomes?”  According to Maki (2004), this step raises the following questions:

• Will the criteria used to evaluate student achievement be externally or internally established?
• What kind of results do externally and internally developed criteria of judgment provide?
• How useful are the results in promoting interpretations of student achievement within the context of educational practice?

There are two considerations to keep in mind when evaluating the results.

Norm Referencing

A norm-referenced approach to assessment involves comparing student(s) performance against other individuals who are performing the same assessment. This is often helpful when comparing performance against other institutions or in other programs.  Examples of norm-reference assessments include the SAT, Collegiate Learning Assessment (CLA), and certification and licensure exams. Norm-referenced assessments can also be developed within an institution based on shared criteria and standards of judgment, for example, a rubric or test used to assess general education competencies.

In practice, norm-referenced assessments are used less frequently in the classroom to assess student achievement of course level and program-level outcomes.  Norm-referenced assessments also provide limited information about students’ behaviors, attitudes, habits of mind, and problem solving-processes, so they are often not as useful as criterion-based approaches.

Criterion Referencing

In contrast to a norm-referenced approach, criterion-referenced assessments do not compare student(s) performance relative to others.  With a criterion-based approach, a person’s knowledge or skills is compared against a predetermined standard, learning goal, or other performance level. The emphasis is on the performance level relative to the established criteria, not how others performed relative to those criteria.

Faculty are familiar with criterion referencing; in fact, this is how most student learning outcomes are assessed.  When we identify an SLO, we often preface it by stating, “A student will be able to….”  By doing so, a criterion-reference approach is implied because the benchmark that will be used to measure student work is clearly stated. However, this does not mean that tools, such as the VALUE rubrics or locally developed tests, cannot be used to measure the achievement of SLOs with criterion-referencing. On the contrary, well-designed measurements allow for performance to be measured against multiple dimensions that can be used to enhance teaching, learning, and the curriculum itself.

Additional considerations in this step include how to best summarize and analyze the results. These decisions will largely be dictated by the type(s) of data the assessments produced: quantitative, qualitative, or both. As discussed above, quantitative data may be analyzed using statistics such as frequencies, percentages and averages, and they can be visualized using tables and charts. A person with knowledge of spreadsheets can be valuable in helping to summarize the results and perform basic analyses. For qualitative data, a person with content and thematic analysis experience will be very helpful in identifying themes and commonalities. The Office of Institutional Effectiveness is available to assist with analyzing both types of data if needed.

Summarizing and analyzing the results is just one part of this phase in the cycle.  Discussing and interpreting the results is also crucial. It is important that assessment results not reside in a vacuum, in the hands of the chair and assessment coordinator.  Results need to be distributed and discussed, preferably during department meetings, where many individuals can weigh in on their meaning.  Very often, data will lend themselves to varying interpretations based on each person’s experience and knowledge. It is important that multiple stakeholders’ voices be heard.  Among the questions to be considered in this step are:

• Were the learning outcomes met? If not, why not?
• Were there some aspects of the learning outcomes met?
• Do we need to gather additional data?
• What changes need to be implemented to improve the results?