It is widely recognized in academia that teaching evaluations are dumb. Let me qualify that: it is widely recognized by anyone who has bothered to look in even the most casual way at the research on teaching evaluations. If you are too lazy to Google the overwhelming evidence, I am not going to bore myself providing links to each individual piece of evidence pointing to the same conclusion.
Briefly, there is a strong bias built in based on student expectations (what they expect from you based on their other courses, the "Well my other professors do _____" problem), biases (women and people of color get evaluated differently and generally in much more critical ways), grading (magically we become better professors when we're giving higher grades!), and really basic transfer-of-affect type stuff (students are more favorable if you give them cookies or some other kind of treat like extra credit or, as one former colleague famously did, announcing that the final exam was canceled immediately before handing out evaluation forms).
No matter how the questions are worded, what evals end up measuring is some nebulous combination of Likeability factors rather than learning. We might like the students to state objectively whether they had a good learning experience but ultimately they end up telling us whether they Liked the professor and Liked the class.
Anyone who fails to recognize the above at this point is ignorant – possibly by design – of the evidence, dumb, or a university administrator.
OK, so starting from the agreement that evaluations are, if not total garbage at least unacceptably noisy, the question that nobody seems prepared to answer is: What do we use to replace them? "It's the best of a bunch of bad options" is not a good defense of anything, yet that's exactly where we are. There are broadly three alternatives:
1. Not having any sort of metric or evaluation for teaching in higher education. This would not only do a disservice to students – simply starting from the assumption that everyone is teaching well simply by being hired to teach – but it would screw faculty by eliminating any mechanism for rewarding good teaching in tenure and promotion. If there is no metric to show I'm a bad teacher, there's also no metric to demonstrate that I'm a good one. And so research and "service" become the sole determinants of T&P. Hard pass.
2. "Outcomes-based" evaluation, or the kind of stuff they do in K-12. Administrators could determine our teaching effectiveness by looking at how well students do grade-wise. This is transparently ridiculous since we control the assignment of grades in our own courses. If I am told "Lower grades mean you're bad a teaching" then obviously everyone's getting an A. If some examination from outside the course is going to be used, then I'm spending all my time "Teaching to the Test" which is exactly what's happening in K-12. We know this doesn't work.
3. Some kind of peer evaluation. Terrifyingly, I've actually heard some people suggest this. So your senior, tenured colleagues watch you teach and determine whether you're a good teacher or not. Yeah great what could go wrong? This either puts junior faculty fully at the mercy of one or more senior colleagues who could range from "Brilliant academic" to "Bitter, hate-filled deadwood Associate" or it creates some kind of self-certification system wherein faculty take turns declaring one another Outstanding Teachers.
It's not that I see any merits worth defending in student evaluations of teaching. The bigger issue, instead, is that no viable alternative exists. I'm all for doing away with student evals. Really. Cancel them tomorrow. The worrisome thing is that the things they will be replaced with have the potential to do all the things that are bad about student evals, but worse.
Sure, let's try some subjective, "holistic" approach to evaluating classroom performance. It won't take long to reveal how vulnerable that system will be to interpersonal grudges and biases. Oh, the Dean doesn't like you? Your senior colleague was opposed to hiring you because he wanted a different candidate? Awesome. Those people now get to decide whether you're a good teacher. I'm sure they'll be fair and objective.