The Problems with Teacher Ratings

I've been a huge fan of teacher ratings. I use them to avoid bad classes, and I've wanted CMC to post teacher ratings online, so every student can use them. If students were smarter about picking classes, not only would the bad teachers receive fewer signups and be forced to improve, but the good teachers would be in high demand and the school would work hard to retain them. Teacher quality is the most important variable in student learning; having a good teacher has a much larger effect than having a small class size, even though the US News college rankings place much heavier emphasis on class size than teacher quality. Other studies have shown that bad teachers in introductory classes can dampen students' interest in majoring in that field. Because students are in class with teachers every day, the theory goes, they should be in the best position to evaluate those teachers. But what do teacher ratings actually measure? Two studies provide surprising results. A 1993 study showed that outside observers can predict a teacher's ratings by watching video of a teacher for 30 seconds, with the sound turned off.1Apparently, the words coming out of a teacher's mouth don't matter as much as their body language.

The second study shows that we could replace the 30-question survey with two questions (Does the teacher seem like a nice person? and Is the teacher organized?), and get the same results.2 This suggests that our criteria for "good teaching" boil down to factors that have nothing to do with how well students are learning the material.

We do not assess our teachers based on how well they're teaching the material; our standard model of viewing the interaction between student and teacher primarily as a mechanism for transferring learning may be wrong. Tyler Cowen, an economist at George Mason, argues that our relationship with teachers is more about giving us confidence, inspiring us to do great things, and "overriding the governor" by assigning difficult work, than it is about learning the course material. Although our opinions of teachers have little to do with how qualified they are, almost every school makes hiring decisions based on a candidate's published papers, and the prestige of the candidate's degrees. CMC considers student evaluations when making tenure decisions, although it is one of only a few schools to do so.

If CMC replaces a 4.00/6 quality teacher with a 5.00/6 quality teacher, how has the school improved? High ratings are a good predictor of a good classroom experience, and low ratings are a sign that you'll spend the semester resenting the time you spend in class. A great teacher can make students fall in love with a subject, or inspire them to do great work, but it's not clear that either is correlated with either the teacher's knowledge of the material, or student learning. Teacher quality is important for test scores, but it's unclear that better teachers have an impact on how much material we're learning.

----------------------------------------------------

1Nalini Ambady and Robert Rosenthal, "Half a Minute: Predicting Teacher Evaluations From Thin Slices of Nonverbal Behavior and Physical Attractiveness," Journal of Personality and Social Psychology, Vol. 64 No. 3 (1993) p. 431-441.

2Gerald M. Meredith, "Dimensions of Faculty Course Evaluation," Journal of Psychology, Vol. 73 No. 1 (1969), p. 27.