Falling down on the job.
JR sent this comment:
"YFS,
What do you think of these postdocs geting fingered for fabricating data? There was one at Penn a few weeks ago and here is another one from Dartmouth.
http://www.the-scientist.com/news/home/53493/"
Yeesh.
"To put 100% of the blame on the postdoc years after they have left the lab has reached a new low. You have to wonder if this is a modern interpretation of the good reference letter/bad reference letter where now if your former postdoc now independent PI is a threat to the mentor PI that you can now be exposed and have your career killed just as it starts to be productive. Just a thought."
Yes, especially since you bring it up and it's not mentioned in the article at all, I I am bothered about the lack of direct accountability for the PIs.
You have to admit, though, that some of that will rub off on the PIs indirectly.
And, it will also be bad for anyone who has worked in those labs since then.
I'm also not sure I think the punishment described in the article fits the crime.
I've probably said this before but I'll say it again. I think that anyone fabricating data is due to only three possible things:
1) mental defect or disease
2) pressure of the current scientific system
3) the arrogant conviction that no one will know the difference
I think this is a fundamental problem with the way the system is set up. Especially with big labs, the PI won't and usually can't micromanage. And maybe they shouldn't.
In some ways I think this is an example of how, as a postdoc, you're essentially a PI with most of the drawbacks and none of the benefits. You're frequently on your own, but they get to claim they're training you. You're basically doing everything yourself, but they get to be senior author on your paper and put your work in their grants. Etc. etc.
So it seems consistent that you get blamed for anything that goes wrong, even though the PI was supposedly in charge. They get all the credit, but never any blame.
In the case described in this article, it sounds like this guy is a great example of someone who wasn't trained properly in grad school about how to handle data processing. You don't pool data from experiments that were done differently. You don't amplify signals that are actually noise. And so on.
Or maybe he tried to fake it all. We'll never know for sure.
But this is a great example where everyone failed. The postdoc failed to seek advice. The PI failed to supervise. The reviewers - thesis committee? and/or journal? - failed to ask the right questions.
Weren't there any other pieces of evidence that seemed inconsistent with the interpretations? I find it hard to believe, if the research were really thorough, that there was nothing else that didn't match.
As usual, we're left wondering how many other pieces of published literature are equally wrong. Is this good for science? For the public that funds our work? No way.
Are we all afraid this could happen to us when we're PIs? You bet. What can you do?
1. Require that everyone keep a good notebook. Run a tight ship.
2. Ask to see the primary data for anything you, as PI, are going to publish. Although this could be a lot of work, I don't think it's unreasonable. In the future, I'd hope that all fields will shift toward always including all the primary data, instead of publishing only the very best hand-picked examples that suit the story.
3. Train people yourself when they first come in, and then let them loose when you're satisfied that they understand what's good scientific practice and what's not. Good labs already do this. Lots of labs do not.
4. Pay attention to the exceptions. Create an atmosphere where inconsistencies are valued instead of punished.
Any time you get a result you don't expect, that's telling you something. Often it's telling you your original hypothesis was wrong, or at least partly wrong.
Other times it says you have technical problems in your lab that are probably not affecting just this experiment, but others as well.
Try to have a lab where people feel comfortable sharing their problems instead of feeling pressured to hide them.
Try not to have a lab where the atmosphere is so miserable, that one spineless person (how common are they? are they more common in science than in the general population?) will do anything to get a paper, get a job, and get out.
I fear it's more common than anyone wants to admit, and it's only going to become more common if the system continues on as it is.
Labels: data, scientific method
posted by Ms.PhD @ 12:58 PM
9 comments
9 Comments:
I don't think the reason people falsify results is because of mental defect or desease.
The pressure of the current system may have a lot more to do with it.
Junior scientists grow cynical when they see that after years of taking abuse from PIs they do not get the credit they rightfully deserve, and their future is questionable despite all that hard work. So if they lack the necessary results to get that coveted job, and they can either tweak some graph that maybe will get them that job, or maybe will land them in trouble, OR they can be honest and lose that job anyways, their choices are not completely illogical, purely from "what happens next" viewpoint.
I am not advocating falsifying data, obviously, but I can understand the circumstances when this can be seen as last resort, and I can see the logic behind it too. Time and time again we see cheating in other competitive arenas - for example sports, or politics.
I think more minor offenses - such as showing only selected data, neglecting to report all the dirty laundry that may undermine your claims - I mean minor compared to downright making up data with no experiments or samples - is much more commonplace than we think. Making far stronger claims and conclusions, while dismissing possibility of artefacts or alternative explanation are a form of unethical behavior as well.
But then again, most of us have seen plenty of other unethical behavior - people getting on papers where their contribution is non-existent, dilution of authorship, misappropriation of credit, etc. And who is investigating those offenses?
Again, I am not condoning this behavior, but I can totally understand the calculation aspect behind it. I don't think these postdocs are much more morally corrupt than average PI out there. Everyone is gaming the system - from using what is clearly outlandish claims just to get funding, to forcing students and postdocs work slave hours for little wages and then "steal" the credit by making it seem like PI did all the work.
So excuse me if I am not shocked that there are postdocs out there who after working 60-80 hour weeks for 10 years through grad school and postdocs are faced with the stress of "you need to get this paper published or you are out on the street tomorrow, looking for a new career at age 32", who may decide that the risk of getting caught (and looking for a new career at age 32) is worth that job, as opposed to doing the right thing and essentially giving up on the last 10 years of their lives.
I also wonder how much "suggestive" fraud is going on, along the lines of PI constantly pushing postdoc or student to "find effect X" and essentially telling them that it must exist, and that they must find some evidence of it, and the project will not continue until evidence of X is found. I know a postdoc who spent 4 years of his life on nailing a few data points for a key publication. His advisor wouldn't let him work on any other project, no matter how hopeless getting those data seemed. Eventually, (miraculously) those data points were obtained, with reasonable error bars, supporting PI's theory. Never mind that previous measurements with huge error bars seemed to paint a different story. Did they keep taking data until they got what they wanted so they could stop and publish? I am not saying data was fabricated, just that you can use this "pick and choose" games to prove just about anything.
And once again - who is to blame here? "Your thesis is not done until we have evidence that X=Y" is not very different in my opinion from "You can't sleep or even lie down until you tell us Saddam has weapons of mass destruction, which we already know to be true and precisely what we need to hear".
i know someone who faked most of his published data. no one can replicate his findings but he jumped ship and landed a sweet job before anyone found out. in fact, the person who found out was a fresh postdoc who just joined the lab (aka "a nobody")
the 'reason' why this person faked the data was 'because' (according to him) his boss was a slave driver and needed results fast.
What ever happened to the scientific method?
When did academic research become science fiction?
How many of you have had the paper written before you even generated a single figure?
Others may disagree with me, but cherry picking your data is fabricating. You are misrepresenting the results of the experiments. Unless you can confidently invalidate a failed experiment (or an experiment that gives the undesired result) you don't have enough control over your system and cannot have any confidence that the data you are representing to be a true and accurate result.
The pressure to fabricate is so large. It is a huge problem and these publicized cases are only the tip of the iceberg. The pressure to publish novel, sexy science has forced many to turn a blind eye to the scientific method. I think this is a big problem and I am afraid that we are training a generation of scientists who see nothing wrong with misrepresenting their data just to reach the desired conclusion.
I remember once I performed a one off experiment that I showed the results to the PI. Without my awareness, within 2 hours the PI put that data in a paper and sent to Science only because it contained the desired result. I could never reproduce that result and I eventually fouund out that the observed result was just artifact. Fortunately the paper was rejected, but I often wondered what would happen if it was accepted.
for crying out loud. the problem is that people are led to believe that there will be a faculty job out there for them eventually. there is no guarantee. period. even if they get that science/nature/cell/etc paper.
but its not like you have to change careers if you can't get a faculty job. there are plenty of non-faculty positions out there that require post-doctoral level research training. most of them, despite what YFS has suggested in the past, require a lot of creativity to be carried out and you have a substantial amount of freedom.
yes, there are some internal deadlines like board meetings and compliance issues like HR training or GMP that make the work suck at times, but how is that different from grant deadlines and priorities like committee work and keeping your lab credentials (safety, recombinant DNA, animal protocols and clinical study protocols) current?
why are you doing science? to advance knowledge? what kind of knowledge? the kind that matters only to experts or the kind that transforms society or healthcare? what kind of lifestyle do you want? are you willing to sacrifice time and money for what you perceive as academic freedom?
the path to academic freedom is much longer than the path to an independent faculty position. you have to navigate the tenure process and learn departmental and institutional politics and become a recognized expert. maybe by the time you're 50 you will have academic freedom. can you wait that long? how many assistant and associate professors do you ever see on a campus if you don't work directly with them? go talk to a few.
Actually, come to think of it, I have a friend who left grad school after 6 years because he found out that a former postdoc (now a PI) had faked all his results.
The advisor wouldn't let my friend publish his thesis work, which largely corrected all the 'mistakes.'
He went to industry without getting a PhD.
I forgot about it because I never really heard all the details.
in response to the first reply, I definitely agree with the general premise. We all focus on scandals associated with data fabrication and other unethical behavior, but what are the underlying reasons for this behavior? everyone turns a blind eye to this huge problem - it's as if we ignore an elephant in the room but get shocked (Casablanca's "I am shocked!") when an elephant farts.
to the poster about other, non-academic opportunities - your comment is neither here nor there. Yes, there are *some* non-academic opportunities for *some* disciplines. But even the ones that do exist involve beginning at the entry level, company-wise speaking. whereas one of the major attraction of faculty jobs is that you are your own boss and almost instantly have a small army of your own postdocs and students.
Either way, the current system allows for PIs to abuse postdocs (and grad students) with very little remedy or feedback. Especially true for postdocs, who are in a "sink or swim" situation.
Some people engaged in cherrypicking data and other unethical behavior are probably morally corrupt individuals with fuzzy ethical boundaries. Or, maybe some of them are decent human beings who are more like cornered animals with no way out, under a lot of stress - who succumb to this stress temporarily due to "publish or perish" dilemma. I don't really know one way or the other, but I do agree with the top poster that just viewing everything through black and white prism of "they are morally corrupt, lying cheating sociopaths" is probably too simplistic.
in my experience, if you are a post-doc trained phd going into established biotech/pharma/chem industry in a research capacity you should at least get a tech within 6 months of when you join if not immediately (you may not realize it but practically everything is negotiable in business). of course you're going to be near/at the bottom of a new ladder -- you don't have the expected complement of skills to be anywhere else - i mean you'd start as an instructor/lecturer or assistant prof first, right? not as a tenured professor. but in industry you get paid more, the hours are more compatible with having a life outside work, the work environment is often more pleasant because there are shared goals and shared rewards, you aren't scrounging around for resources, etc.
please read intuition by allegra goodman. it's an excellently written account about the flaws of the scientific system and about the questionable practices of scientists... it seems like post docs in particular get the raw end of the deal. at least that's how it's portrayed in this book.
ena7800,
I was going to read this, a friend is supposed to lend it to me.
I think people are starting to understand that postdocs do get a raw deal. How long it's going to take before anyone tries to fix that is another problem altogether.
Ooh, just as I'm typing this I see that two people have linked to this post. I love the title of the first one...
Post a Comment
<< Home