Ethics of anonymous silence
This week in Nature there's an article titled "Under suspicion", discussing how Nature investigates allegations about data or author conduct.
This got me thinking again about one of my favorite topics: how many things have to go wrong before someone gets caught doing something really egregious and is actually forced to retract the paper.
More often I suspect truth dies by a thousand pinpricks, and the paper is published anyway, and even though many grad students and postdocs suspect entire fields are based on contaminated publications, and suspect we know why, those papers are never retracted.
Here are a couple of examples I know about peripherally, which, to my knowledge, have never been investigated or enforced in any way:
1. Author makes outrageous claim, paper gets reviewed by high-impact journal because the result is so "surprising". Turns out the crux of the claim is based on an extremely high sample number and/or vastly overstated statistical power. Reviewer suggests politely that the number in question is either a lie or a typo. Author revises the text to remove the "typo" but keeps the figures and conclusions the same. Paper is accepted.
Scary, part a: Nobody tries to verify that these authors actually tested even the revised, smaller number of samples (which is still way too many to be believable).
Scary, part b: Nobody can publish anything conflicting with the model based on these published claims, without reproducing the original results in at least as many samples, and no one can afford to do that because it's so outrageously expensive. And actually, if you calculate out the cost, it's clear that the original group couldn't possibly have afforded to do that many samples themselves, either. Suggesting that the only rational answer is that they... didn't.
I wonder if requiring some kind of accounting procedure for papers would help catch these kinds of exaggerations? Not that I'm favoring extensive bean-counting, but sometimes all it takes is the blank space on the back of an envelope.
2. Authors submit paper to high-impact journal with data that have clearly been processed incorrectly. Reviewer points this out. Paper is rejected. Authors submit same paper, with no revisions, to different high-impact journal. Paper is accepted.
Scary, part a: Reviewers at second journal apparently didn't notice? Or did the authors actually fix the problem and magically get the same results? Really? Magically?
Scary, part b: No one involved in the original anonymous reviewing process, neither the reviewers themselves nor the editor, is ethically (or, um, legally?) required to come forward and say anything? So they don't? It's like it never happened? Because it's anonymous, even though it's in the first journal's database, presumably, somewhere? Does that information just get deleted? What would people think if that information got out? Would we finally know which reviewers were completely spineless kowtowers?
Sometimes I wonder how often these kinds of things are happening. More often I wonder why everyone puts up with it.
I like to think we'd learn a heckuva lot if somebody would hack into those computers and find out the extent of all this nonsense. It would certainly be a fun data mining project, tracking the reviews and the papers across journals to see where they end up and how many accusations are made, investigated, or just lost in the shuffle from journal to journal.