Student Evaluations of Teaching (Mostly) Do Not Measure Teaching Effectiveness

Abstract

<p>Student evaluations of teaching (SET) are widely used in academic personnel decisions as a measure of teaching effectiveness. We show:</p><ul> <li>SET are biased against female instructors by an amount that is large and statistically significant</li> <li>the bias affects how students rate even putatively objective aspects of teaching, such as how promptly assignments are graded</li> <li>the bias varies by discipline and by student gender, among other things</li> <li>it is not possible to adjust for the bias, because it depends on so many factors</li></ul><ul> <li>SET are more sensitive to students&#39; gender bias and grade expectations than they are to teaching effectiveness</li> <li>gender biases can be large enough to cause more effective instructors to get lower SET than less effective instructors.</li></ul><p>These findings are based on nonparametric statistical tests applied to two datasets: 23,001 SET of 379 instructors by 4,423 students in six mandatory first-year courses in a five-year natural experiment at a French university, and 43 SET for four sections of an online course in a randomized, controlled, blind experiment at a US university.</p>