Blog

Like death and taxes, evaluation ratings bias is a fact of life. Here’s how to manage it.

managing evaluation bias with the right tools

With a new semester approaching, we’re reminiscing about an article by University of Kansas’s Dr. Joey Sprague that appeared in Inside Higher Ed in June 2016.  Dr. Sprague’s study surrounds the bias in student course evaluations.  It’s an age-old concern and you may just recognize this very same problem at your institution.

Dr. Sprague emphasizes the need to “focus on the measures that apply a higher level of reliability” and “apply sound statistical analysis.”

Our clients strive to get the most out of student rating evaluation data by analyzing the data trends to point out potential biases, and while it might seem an impossible feat, we have provided better ways to work with this information.

Here are five examples of how CoursEval™ can help apply statistical analysis to student evaluations of courses and instructors.  Response bias is here to stay, but with CoursEval™ you can equip yourself with the right tools each semester to make better use of the data you have:

  • Examine distribution of evaluation scores: CoursEval™ reporting capabilities allow institutions to compare various measures of central tendencies such as the mean, mode, and group median (adjusts the median slightly upwards or downwards to provide more information than the standard median).
  • Study survey  items: Compare the distribution for  survey items across classes, departments, evaluated individuals, semesters, years, colleges, programs, and rotation blocks or sites with CoursEval™ reporting.
  • Test item reliability: Compare ratings on the same item for a faculty member across different classes using our Survey Intelligence Report. Do this for each evaluation period or examine trends over time.
  • Measure the variability of student responses: Use CoursEval’s standard deviation and standard error reports to gauge differences or similarities in opinion.
  • Identify potential biases within student evaluation results: Look for patterns across a number of statistical measures within CoursEval™ reports. Take the analysis even further to test variables outside of CoursEval™ with a raw data extract and data warehouse.

Have your own thoughts on ensuring reliability, leave your comment or get in touch with us and we can tackle it together.