Response to comment on Dallas Cowboys post.

Anonymous said...

first time caller, not all that long time reader

"And the great irony is that, when it comes right down to it, even science judges (at least the first cut) on appearances alone, too."

Your publication record isn't your appearance. The record is an important description of your scientific contributions.


Anonymous,

Are you a scientist? What stage are you in the hierarchy?

This is actually a great question for scientists and non-scientists alike. I think most scientists fail to think critically about this issue, so I'm going to critique it here.

Your publication record is influenced by many factors. As a junior person (by that I mean, student or postdoc), you are not the person who chooses what gets written up, which figures are included, what journal it is submitted to. Even if you write the entire paper yourself, usually someone else has to weigh in before it goes out.

So I would argue that, while yes, it is correct to say that a list of publications is a description of your contributions, it is not an accurate or complete picture of a person's skills, accomplishments, or aptitude. It definitely does not describe whether you will be a good professor.

You could get a lot of information from a list of publications, but it would still be incomplete without further confirmation. It might describe whether you are a good collaborator if you have a bunch of authors from other labs, but more often than not your PI is the source of your collaborations. It might describe whether you are a good mentor if your students are co-authors, but again, the credit usually goes to the PI and nobody bothers to check the status of all the authors on a paper whose names are unfamiliar. It might describe if you are a sexist prick if you have never published with a female co-author once in your whole career, but again, usually names are listed as initials and nobody bothers to check.

aside: (I'm not making this up, I've seen papers with 30 authors where not a single one was female, and you have to wonder what's going on there. I've also seen a lot of papers- mostly older ones- where the authors are all male, and women are thanked in the acknowledgments for 'technical help' like doing all the experiments.)

I know plenty of people with high impact papers who contributed only a fraction of the data to their own papers, AND cannot accurately describe, much less teach, the methods their labmates and collaborators used for the figures they contributed.

(I know plenty of senior authors who fall in this category too, but we'll leave that for another post.)

I also know plenty of people whose work is top-notch, but it's not in high impact journals because their senior, tenured PI doesn't believe it matters where you publish, or more accurately, they know it doesn't matter where they publish and they don't care where you publish.

Or they just have no clue, or worse, no interest, in how to get a paper into a high impact journal. It's quite a bit different than getting a paper into a 'specialist' journal.

So a list of publications is subjective criteria because no one (by that I mean, almost no one) actually looks at a CV, then goes out of their way to download them all from Pubmed, read the papers and determine whether they have substance or quiz the first author on how much of the work they actually did.

While there is often a lot of attention paid to whether a middle author actually contributed anything significant, little attention is paid to whether a first author deserved that slot.

There's an assumption that goes along with first authorship that often is not deserved.

But nobody thinks about that, unless a second person's name receives a *contributed equally to this work.

And how the location of the publication influences different people is also subjective. For example, some people see a paper on a CV in an open-access journal and think nothing of it; others think "Hey, good for them!"; still others think, "Oh, it couldn't get in anywhere better so it ended up in one of these."

Within a field, a specialist journal may be well-respected; outside of that field, it's just another low-impact paper.

I'm a relativist. While data are objects, interpretation is subjective.

When you use something superficial, like a list of publications, to evaluate something bigger, like a faculty candidate, you're bound to miss good candidates and you risk getting nothing but attractive-looking mediocrity.