Evaluations.

I was thinking about the responses to the Dallas Cowboys series of posts, and it really does just come down to whether you believe anyone has the right or the tools to accurately guess what a stranger is capable of.

We can't all be professional profilers, after all.

Personally, I never believed in grades. I frequently got A's without trying, saw people cheat to get A's, and saw people freak out from insecurity like that described in this beautifully written post. It was clear to me from an early age that grades didn't mean much.

So if I have this fundamental belief that the way we assess students is wildly inaccurate except at the extremes of the curve, why the hell would I want to be a professor?

Maybe it's a religious discussion at the base of it, but to me it seems that teaching and learning need to be fluid things, tailored to different learning styles. This also means that assessment by just one method- a test, an essay, a presentation, a CV (!!!)- no one thing is ever enough by itself.

And yet, cuts are made based on one or two criteria. To get into college, you have to have a minimum SAT score... but there are exceptions. If you are an athlete, or a legacy.

You have to have a certain grade point average... but this varies among school districts.

It always drove me nuts that there isn't a better way to standardize grading, so that you don't get large intro courses evaluated based on who is the 'easy' prof and who is the 'hard' prof.

Even for courses where the criteria are ostensibly objective, e.g. anything involved calculations where the final answer is either the right number or it isn't, the questions on the exams can vary in difficulty depending on who is teaching the course.

The same thing drives me nuts in science, but it's even worse, because the perception of science is that it's totally objective, totally fair, all for the greater good, blah blah bullshit blah.

And it's even less true in science than it is in education. And the more cross-disciplinary departments become, the less they seem to grasp that some things are harder than others.

In my field, we almost never publish a result that was only arrived at by one method, and never by only one attempt. Everything is repeated, usually at least 3 independent times, 3 independent ways.

That means for every panel of every multipanel figure I've ever published, I had to do at least 9 experiments.

At least, that's how I do research. It means I don't have as many publications as some people, but it also means I'm pretty damn sure everything I've published is a real, reproducible result.

I know a lot of fields are not like this, for various reasons. Some experiments can only be done once, because of cost and/or time to do a single experiment. Some experiments can only be done one way, because the technology just isn't there (yet).

In some fields the standards are lower and nobody expects error bars at all, much less asks for a paper to be revised to include them.

So to me, to work so hard to get to the truth, and then be judged based on only a tiny sliver of what I really know and can do, is just plain insulting.

Worse than that, though, are the people who seem to think it's not even worth discussing whether the way we choose faculty right now could use some improvements.

There are tons of articles and forums devoted to the ongoing debate about grades, and everyone seems to agree that there are myriad ways of assessing and evaluating student progress. That it's a work in progress.

So why is it verboten to even try to start a discussion of new ways to assess and evaluate faculty candidates?

Amazing to learn what will trigger a knee-jerk, defensive reaction.

I must have found a button.