public service message

PLEASE DON'T FEED TEH TROLL(S).

THAT IS ALL.

K THX.

Ethics of anonymous silence

This week in Nature there's an article titled "Under suspicion", discussing how Nature investigates allegations about data or author conduct.


This got me thinking again about one of my favorite topics: how many things have to go wrong before someone gets caught doing something really egregious and is actually forced to retract the paper.


More often I suspect truth dies by a thousand pinpricks, and the paper is published anyway, and even though many grad students and postdocs suspect entire fields are based on contaminated publications, and suspect we know why, those papers are never retracted.



Here are a couple of examples I know about peripherally, which, to my knowledge, have never been investigated or enforced in any way:


1. Author makes outrageous claim, paper gets reviewed by high-impact journal because the result is so "surprising". Turns out the crux of the claim is based on an extremely high sample number and/or vastly overstated statistical power. Reviewer suggests politely that the number in question is either a lie or a typo. Author revises the text to remove the "typo" but keeps the figures and conclusions the same. Paper is accepted.


Scary, part a: Nobody tries to verify that these authors actually tested even the revised, smaller number of samples (which is still way too many to be believable).

Scary, part b: Nobody can publish anything conflicting with the model based on these published claims, without reproducing the original results in at least as many samples, and no one can afford to do that because it's so outrageously expensive. And actually, if you calculate out the cost, it's clear that the original group couldn't possibly have afforded to do that many samples themselves, either. Suggesting that the only rational answer is that they... didn't.

I wonder if requiring some kind of accounting procedure for papers would help catch these kinds of exaggerations? Not that I'm favoring extensive bean-counting, but sometimes all it takes is the blank space on the back of an envelope.



2. Authors submit paper to high-impact journal with data that have clearly been processed incorrectly. Reviewer points this out. Paper is rejected. Authors submit same paper, with no revisions, to different high-impact journal. Paper is accepted.

Scary, part a: Reviewers at second journal apparently didn't notice? Or did the authors actually fix the problem and magically get the same results? Really? Magically?

Scary, part b: No one involved in the original anonymous reviewing process, neither the reviewers themselves nor the editor, is ethically (or, um, legally?) required to come forward and say anything? So they don't? It's like it never happened? Because it's anonymous, even though it's in the first journal's database, presumably, somewhere? Does that information just get deleted? What would people think if that information got out? Would we finally know which reviewers were completely spineless kowtowers?



Sometimes I wonder how often these kinds of things are happening. More often I wonder why everyone puts up with it.

I like to think we'd learn a heckuva lot if somebody would hack into those computers and find out the extent of all this nonsense. It would certainly be a fun data mining project, tracking the reviews and the papers across journals to see where they end up and how many accusations are made, investigated, or just lost in the shuffle from journal to journal.

this about says it.

more later.

for now, read this short post from Good Enough Woman.

Most post

At 726 posts in my dashboard (including a handful that were never posted), and in response to the intriguing Acadamnit meme:

Most Favorite Post (of mine)? Letter I won't send... yet.

Post with the Most Comments? As an American, I'm a minority in my profession

Most Memorable Post? Dear PI, it's your fault I'm depressed


Most Indicative of Your Blog Identity Post? Still true now

Most Humorous Post? Evolution of a project

Most Regrettable Post?(As in, I wished I hadn't had to) Alarming plagiarism of yours truly

Most Misunderstood Post? Just buying groceries, thanks

Most Satisfying to Write Post? Dear PI

Most Likely To Never Be Posted Post? On Obama vs. Clinton (never posted, now out of date)

Most Important Post? Ambassadors for Science

Most Appreciative Comments on this Post? Building Confidence

Where's my whistle?

This has been a week of uh-ohs.

The refrain that makes me want to hurl, because I've heard it multiple times from different people:

Uh oh, that gel from your collaborator seems to be completely fake

Luckily, none of these cases have affected me directly (yet?), but I have been thinking again about this question of what do you do when you suspect dishonesty in science.

It also got me thinking about how this is probably also contributing to why I'm not appreciated.

Yes, I've said it before and I'll say it again: I really am that good. But I sometimes wonder if the reason most people don't know real expertise when they see it is because they're willing to cut corners and falsify results, so they just assume I'm doing it too?

I'm tired of being treated like I'm mediocre, when at least I know my results are real, and I've had to watch several cases of liars getting High Impact papers and faculty positions.

But there is nothing I can do about it in the absence of hard proof or a confession that they spiked their samples, that the PI pressured them into producing the expected answers, etc.

I don't know what to do in these situations. It's the rare PI who will not get defensive if you even hint that maybe they didn't notice something fishy about that paper their favorite postdoc published last year.

And yet, I'm watching generation after generation of grad students get completely screwed, being made to feel inept when they can't reproduce data that probably never existed in the first place.

So I have to wonder, seriously, if I know a handful of these phonies are now professors, how many are there total? Are there more now than there were before? Will they ever get caught? Why do we tolerate it? How come nobody seems to know??

I also know a few people who left science during or after grad school when their PIs refused to admit that their new data invalidated the old, obviously massaged evidence published by past postdocs. They said they couldn't win, research wasn't what they thought it was, and went off to do other things.

I worry that unless we come up with a mechanism, maybe some kind of anonymous hotline, all of us trusting, honest souls will end up leaving out of sheer disgust, and there won't be any real science left.

And yet, the idea of being able to report people anonymously means you open up the possibility for false accusations. It's too 1984 for me, kids reporting their parents during the Cultural Revolution in China. It could be a whole new form of nastiness. Would that really be worth it?

Obviously, this is why we don't have The Truthiness Police. But I'm worried that science is hemorrhaging from being undercut by those who make it a game of ambition, while laughing at the noble pursuit of excellence.

Relative vs. Absolute

One of the analysis programs I use gives you choices. At one point when you're choosing how to display your data, the choices basically come down to the title of this post. It doesn't change the result, just the scale and portability of the results.

...

Yes, my therapist means to help me. Yes, my advisor may (or may not) have (at least some) good intentions, too. Yes, the same could be said about my parents, who could also be said to have screwed up any number of things about my personality and ability to function in adult life.

This week I've been thinking again about how, while intentions are nice, it doesn't really matter if the outcome is still fatally flawed.

Yes, it's nice to have someone on your side. But if that person is steering you wrong, and you're attaching to them only for the sake of having something to hold onto, that's not really going to help you make any progress.

If that person continually lets you down, whether through selfishness or a lack of appropriate expertise, would you keep on trying? If this is your partner, wouldn't you think hard about whether to continue the relationship? If it were your student, wouldn't you think hard about how many chances to give them? If this is your advisor, wouldn't you want to leave the lab?

At some point, good intentions are not enough.

And maybe not even relevant. Doesn't the bad guy usually think he's doing the right thing? Anybody see Watchmen?

...

I really believe that truth in research is relative. Because whatever we think is true now, it's probably only partly right, and years from now someone with better tools and more insight will realize that we were almost always at least partly wrong.

And yet, some things are absolute. Maybe only hindsight has this property: knowing what you know now, sometimes there was one answer better than the other. But you didn't know that then.

Somehow I find this concept easier to accept in research than in real life. Maybe because it's more clear to me how we couldn't have known. In research I read everything I can; I review my data as much as I can; I run all the analyses I can think of and that the software can manage.

In real life, I often find myself wondering if I could just have read the right books or talked to the right people, would I have known sooner what I know now? Because most of this is probably not new, not the way cutting-edge research is new. I'm sure most of my struggles in life and philosophy are old news. What I'm doing in life really is re-search.

So while intentions can only be relative, outcomes can be absolute.

...

At some point, you have to look at the data and say, is this working well enough to justify the time and cost?

I do this almost every day in research. I'm not sure everyone does- there must be a few labs with so much money, that it would be possible to get your PhD and sail through your postdoc never realizing how expensive it all is until you go to write your own R01.

But that isn't how my career has been. I'm always asking, usually before I even do a pilot run, can I afford this even if it does work? What will I do if it's working and I need to buy more and we can't afford that? How much information can I get if this is all I get to do?

It is all worth it?

It's really hard to work this way. It's like having a phobia of commitment. As a serial monogamist, I can tell you it's really a strain when your natural inclination is to throw yourself all in, but you know it's too risky because you'll just be heartbroken when it ends.

On the other hand, you have to start everything with a relatively open mind. There is no absolute intention, because we're all biased whether we mean to be or not.

So when we say "have an open mind' in science, we mean that you try to be objective, whether that means quenching your optimism or your pessimism, sometimes it depends on the person and the day of the experiment. Maybe you can't suppress your gut feeling, but you also know from (relative) experience, we're all wrong about 50% of the time. So you get used to acknowledging your fears and trying anyway. Some people call that brave.

...

Science has taught me a lot of things (so far?).

The length of diligence is always longer than you think.

Courage to try even when you think you'll fail again and again; even when you have failed.

Persistence doesn't even begin to cover how many times you have to pick yourself up and keep trying.

Patience with yourself can be harder than any other kind of patience. Patience with experiments can be easier than patience with other people or with circumstances.

Anger can be empowering.

Silence can raise your stock, but it isn't always powerful. Sometimes it's just passive.

Some people define truth from all angles.

Some people define truth like this:

if you just say it this way, it's technically true, and everyone will be happier.

Some people define truth as outcomes; some define it as implications.

One more excerpt from the same book by Margaret Rossiter.

I'm putting this here because it so eerily reflects things I have blogged about (30+ years since the 1970s when supposedly much progress was made- at least briefly).

My impression from reading this book is that there was a mini-revolution from 1968-1972, but then the momentum was lost: we're essentially moving forward now at a lazy snail's pace, with no major changes from what it was then.

I say this because I found myself writing still true now! still true now! still true now! over and over in the margins of this book, but especially this page. It touches on three major points I've raised on this blog, all of which were really contentious, namely

(a) Deniers, both male and female, who don't believe there is a problem or that anything needs to be actively done about it

(b) SuperWomen who are not actually useful as role models, and who pull up the ladder behind them as they go

(c) That foreign-born women scientists are treated differently from American women scientists, and have had more success not just abroad but also in the US

from page 381 (essentially the last page of the book):

"But if consciousness was running high and the outpouring of outrage was epidemic in some circles, such feelings were far from universal. Many eminent scientists, women as well as men, did not necessarily agree that there was a problem and wondered what all the fuss was about.

Having adjusted to it all years before and believing staunchly in individual virtues such as hard work, they were either oblivious to the problem or, when it was brought to their attention, adamant that it did not exist.

They were so much a part of the "system" that had treated them comparatively well that it was difficult for them, as it had been earlier for Jessie Bernard, to see a pattern and think of employers and colleagues, even sexist ones, as villains.

Often foreign-born, these faculty women clung to an individualistic view that all that mattered was doing very good work and lots of it; one's sex and marital status were irrelevant. By dint of a lifetime of hard work, considerable self-sacrifice, and perhaps a move to the United States, they had "made it", and they did not wish to criticize American institutions that had made their success possible. Their successful work and high rank on the faculty had blinded them to other views; instead they seemed proof that if, just if, a woman was good enough, she too would be promoted to the highest levels.

Their small numbers could be seen as indicators that a few women offered this successful combination rather than evidence that stronger credentials might be required for women than for men.

For example, German immigrant and Nobel laureate Maria Goeppart Mayer of the University of California at San Diego could not understand why the American Physical Society had created a committee on women in April 1971 or why it had put her on it: she had no interest or expertise in the area.

Similarly, Birgit Vennesland, Norwegian-born and long a full professor of physiology and biochemistry at the University of Chicago, ended her autobiographical statement for her fellow physiologists in the early 1970s with some angry remarks about the younger women who now expected to be put on university faculties just because they felt as qualified as men; for women to press to0 hard in this direction would, she felt sure, lower the quality of the faculty and thus in time endanger the strength of the nation. Academia should hold onto its proven ways and not give in to the merely political pressure of diversifying the faculty. "

progress = zero

In 1969, sociologist Alice Rossi presented the following data on women in her field, disturbingly similar to the current numbers for biology(and a variety of other fields, actually):

% of women at each stage:

undergraduate seniors planning to work in the field 43%
doctoral candidates in graduate school 30%
full-time assistant professors in the field 14%
full-time full professors in the field 4%

She goes on to say something relevant to all postdocs [although at the time research associate positions were dominated by women with PhDs who were generally not hired as faculty]:

"An excerpt from her report deplored the fact that research associates, even those with doctorates and ten or more years' experience, were still not allowed to apply for grants in their own name, whereas any new assistant professor could"

--from page 372 of Women Scientists in America, volume II by Margaret Rossiter