Tuesday, October 30, 2007

Response to comment on Dallas Cowboys post.

Anonymous said...

first time caller, not all that long time reader

"And the great irony is that, when it comes right down to it, even science judges (at least the first cut) on appearances alone, too."

Your publication record isn't your appearance. The record is an important description of your scientific contributions.



Anonymous,

Are you a scientist? What stage are you in the hierarchy?

This is actually a great question for scientists and non-scientists alike. I think most scientists fail to think critically about this issue, so I'm going to critique it here.

Your publication record is influenced by many factors. As a junior person (by that I mean, student or postdoc), you are not the person who chooses what gets written up, which figures are included, what journal it is submitted to. Even if you write the entire paper yourself, usually someone else has to weigh in before it goes out.

So I would argue that, while yes, it is correct to say that a list of publications is a description of your contributions, it is not an accurate or complete picture of a person's skills, accomplishments, or aptitude. It definitely does not describe whether you will be a good professor.

You could get a lot of information from a list of publications, but it would still be incomplete without further confirmation. It might describe whether you are a good collaborator if you have a bunch of authors from other labs, but more often than not your PI is the source of your collaborations. It might describe whether you are a good mentor if your students are co-authors, but again, the credit usually goes to the PI and nobody bothers to check the status of all the authors on a paper whose names are unfamiliar. It might describe if you are a sexist prick if you have never published with a female co-author once in your whole career, but again, usually names are listed as initials and nobody bothers to check.

aside: (I'm not making this up, I've seen papers with 30 authors where not a single one was female, and you have to wonder what's going on there. I've also seen a lot of papers- mostly older ones- where the authors are all male, and women are thanked in the acknowledgments for 'technical help' like doing all the experiments.)

I know plenty of people with high impact papers who contributed only a fraction of the data to their own papers, AND cannot accurately describe, much less teach, the methods their labmates and collaborators used for the figures they contributed.

(I know plenty of senior authors who fall in this category too, but we'll leave that for another post.)

I also know plenty of people whose work is top-notch, but it's not in high impact journals because their senior, tenured PI doesn't believe it matters where you publish, or more accurately, they know it doesn't matter where they publish and they don't care where you publish.

Or they just have no clue, or worse, no interest, in how to get a paper into a high impact journal. It's quite a bit different than getting a paper into a 'specialist' journal.

So a list of publications is subjective criteria because no one (by that I mean, almost no one) actually looks at a CV, then goes out of their way to download them all from Pubmed, read the papers and determine whether they have substance or quiz the first author on how much of the work they actually did.

While there is often a lot of attention paid to whether a middle author actually contributed anything significant, little attention is paid to whether a first author deserved that slot.

There's an assumption that goes along with first authorship that often is not deserved.

But nobody thinks about that, unless a second person's name receives a *contributed equally to this work.

And how the location of the publication influences different people is also subjective. For example, some people see a paper on a CV in an open-access journal and think nothing of it; others think "Hey, good for them!"; still others think, "Oh, it couldn't get in anywhere better so it ended up in one of these."

Within a field, a specialist journal may be well-respected; outside of that field, it's just another low-impact paper.

I'm a relativist. While data are objects, interpretation is subjective.

When you use something superficial, like a list of publications, to evaluate something bigger, like a faculty candidate, you're bound to miss good candidates and you risk getting nothing but attractive-looking mediocrity.

6 Comments:

At 9:07 PM, Anonymous Anonymous said...

I am sorry but this is BS. It's one thing to say: "I didn't get faculty job only because I am a woman".

But you are arguing that "I didn't get a job because my publication record sucks, and it sucks because I am a woman"? Seriously?

If you throw away publication records, and I can only guess that you wouldn't count letters of recommendation as they are also subjective, what *objective* measure do you suggest the search committees use to make hiring decisions? Date of birth? Weight and height? Astrological signs?

So let me get this straight - if your paper is not published/rejected/delayed, you think the only possible reason is because your advisor, your collaborators, all referees, editors and everyone else in the world are "in on it" - all of them are in the grand conspiracy to prevent you (and only you?) from becoming a successful female scientist - well... yeahhhh...suuurree... (roll-eyes).

Just because you are paranoid, doesn't mean they are not out to get you...

 
At 10:09 AM, Anonymous Anonymous said...

Oh! Excellent points! I'm going to scurry over to my chairperson's office and suggest that on our next job search, we don't just read the CVs and make an offer, but we actually bring them in for an interview or something. Why, we could even ask for letters of recommendation from not one but several people, and read those letters. We could ask the candidates to write, say, a three to five page summary of their research accomplishments and goals. We could have them give a research seminar AND a chalk-talk on their future grant proposals. THen we could, like, make assessments of all those great things you mentioned in your post. Why, we'd be able to figure out whether they actually understand all the data in their papers. ANd the chalk-talk thing might help us figure out whether they were forward thinking, capable of independence from their PI, and have the right baance of big ideas and realistic expectations!!

I propose that we allow other faculty and students and postdocs from the department to, you know, ask them questions about their work, too.

Hey, what about this idea, oh wise one. Let's say we invite not just our favorite one, (you know, the white guy with the most Cell, Science and Nature papers) but maybe five or six others. Then, we can, you know, like, compare them on criteria that we could not get just from reading their CVs and those other things-what were they-oh yeah, letters of recommendation. We could also have them meet individually with, like, everyone in the department, except for the scary a**holes, and see how they are to talk to. Why we could even go to dinner or do something social. From this we might be able to approximate a guesss about what kind of colleagues they would make.

Oh my god I am so psyched aobut this I can hardly contain myself.

This will REVOLUTIONIZE how we do job searches!!

YFS, you are BRILLIANT!!! Please, please apply for a job in my department. We hopeless paper-counting, H-factor calculating schmucks need someone like you!!!!

 
At 4:25 PM, Blogger Jenny F. Scientist said...

Speaking of publication records- I read a Nature paper this year, on something in my field (bricklaying) by a Nobel-prize-winning electrician's lab. It was utter crap. So the RECORD of the rest of the authors looked great, but the content was substandard. And search committees just look at pubs in the first round- they don't read them.

 
At 5:29 PM, Blogger yajeev said...

"I also know plenty of people whose work is top-notch, but it's not in high impact journals because their senior, tenured PI doesn't believe it matters where you publish, or more accurately, they know it doesn't matter where they publish and they don't care where you publish."

Ms.PhD,

I don't mean to be contrarian, but wouldn't even tenured professors care about where they publish? I mean, they still have to apply for grants, right?

Take it easy. I know it's hard sometimes.

 
At 6:11 PM, Blogger Ms.PhD said...

Yes, yajeev, you would think so!

But if your PI has no immediate funding concerns, they might not be in as much of a hurry as you are.

Funding cycles can go in 5 or even 10 year chunks, and if grant renewals are scattered (as smart people usually do) some PIs need more than a gentle push or a firm shove. For example, I know one PI who never worried about funding until just a couple of years ago, when multiple grants were not renewed in the same year, and suddenly there was this huge gap where they had always been money.

So they need a major gun to the head like that, or like
a) student's salary support is running out, so they have to graduate (and therefore have to get papers);
b) they see similar work presented at a meeting and realize they're going to be scooped.

That sort of thing.

And if they're thinking about retiring, but haven't told anyone that yet, you're screwed. You'll only figure that out later, and no one reviewing your CV will put two and two together.

So you can see how, if your timing is off by 1-2 years, it can make all the difference in the world.

Or by, you know, a decade, give or take 5 years.

I was thinking today about a grad student talk I heard recently, and this student was typical for the lab she's in, where numerous postdocs have died on the vine trying to get an insanely hard project to work, and then a grad student comes along just at the right moment and scoops up a bunch of protocols and goes to town.

In some ways this is academia at its best. Stuff does get done. Eventually. And only with a lot of hard work. But luck is a big factor.

 
At 3:36 PM, Anonymous Anonymous said...

The whole number of papers thing is crap anyway. 99% of what is published is artifact or redundant. At this time the major reason to publish is to get more than everyone else. And peer review is a joke. Just keep submitting it and it'll get in somewhere.

 

Post a Comment

<< Home