Saturday, March 18, 2006

help writing papers.

So here's a question for everyone out there:

In my field, papers are supposed to Be the Best They Can Be. Which is to say, if you haven't done all the experiments anyone could conceivably think of, no matter how technically difficult, it is extremely difficult to publish.

The reviewers always ask the question we all groan over: "Why didn't you do THIS (obvious but technically impossible experiment)?"

So I am trying to write a paper now, and because of personal problems, I switched labs in the middle of doing some of these experiments, and did some experiments in other people's labs, and so on. So the continuity is not ideal, and there are some experiments that, in the perfect world, I would go back and redo a little differently if I could, but it would be a huge time suck to do it now, and I don't think it would change the final result, it would just make the story a tiny bit tighter, and only for the more discerning readers.

I may still have to go back and redo stuff anyway, but I'd rather wait and see what the reviewers demand, since they always ask for both experiments I would expect, as well as some more than I'd expect.

I'd rather do the ones they want if it will get it published, than do the ones that might help slightly but I'm not sure if anyone will notice that I went the distance.

Nobody ever seems to have any idea how much work went into the papers I've published, anyway.

So I'm trying to figure out how to write this paper. One way is to try to discreetly say, "uh, we know this isn't perfect, but it is what it is." This approach will most likely severely limit the options for journals to publish it in, but I prefer it because it's more honest... and also a lot easier to write.

The other way is to try to hide the flaws by the order in which I present the data, so it's not necessarily obvious that I could (probably should) go back and redo the experiments at the beginning because the paper appears to build... you know, that thing everyone does where we act like we KNEW ALL ALONG that this is how it would turn out.

Instead of just telling a narrative, chronological story of how it actually happened. Like all the 'classic' papers are written.

I'm curious to hear what people think. Which is better? Got any great advice? Is it absolutely always better and worth it to go back and redo the experiments first? Am I wasting my time even trying to write it up now?

13 Comments:

At 8:00 PM, Blogger Psycgirl said...

In psyc, we tend to outline some of the more obvious problems up front in the discussion, where we talk about limitations and future directions. BUT, I don't have any publications yet, sooooo you might not want to listen to me!

 
At 8:28 PM, Anonymous Anonymous said...

i'd say, if it is pretty obvious to *you* what experiments are missing... then you'd better do them, because otherwise your paper will get shot down by the reviewers.

sounds more like you want to cut corners...

 
At 12:31 AM, Anonymous Anonymous said...

In my experience, those 'classic' papers that appear to be a chronological recounting of how it actually happened are - more often than not - fairy tales in that the experiments did not necessarily happen in the order described.

Is there really any such entity as the 'perfect' experiment? Yes, we all try, but we also know that we might have done things better or done additional experiments, etc. Sometimes you must decide that you have adequate data to support or refute your hypothesis and that doing more (in the real world that we deal with) is simply not the most productive use of your time. The risk is that if you routinely take this approach you will gain a reputation as the 'master of quick and dirty experiments.'

It is important to never hide known flaws in your experiments/data! In my book this represents intellectual dishonesty and is unfair to your peers in the field (as well as to yourself). If you still think that the experiments and data are valid, then present things as honestly as possible. If that doesn't fly with the reviewers, then so be it.

Good luck!

 
At 2:28 AM, Blogger Dr J. said...

In terms of how to structure a paper: here I think logic, understanding and a good story comes ahead of chronology everytime. Every scientist knows that some things occasionally happen out of the expected order - a surprise result, having to wait a long time for a reagent or machine or collaborator. What´s more important is that the paper makes sense, that it is easy to follow and understand and is engaging. I know a number of scientists scoff at this, but keeping your readers engaged and thrilled will ensure they remember this paper when it comes to writing the references for their next. And they´ll remember you at conferences and for interviews if they´ve liked your papers - and that means more than just the dry results.

Secondly - the doing all experiments possible. Are your results enough to stand up on their own or do they really need a bit more support? One very key phrase to me is "beyond the scope of this study". We all know that we could do more, find out more - but is it necessary to have ALL of that in ONE paper? It depends also on the journal you´re submitting too of course - you want Nature, back up with more. But in the end, say you have identified a new protein, the first paper doesn´t have to include it´s purification, sequencing, gene identification, enzymatic studies, subcellular location, interaction partners, mutations in obscure diseases, NMR data, EM picture, crystal structure, drug target validation, small molecule inhibitor screening and initial clinical trials.

Referees will criticise, that´s their job. You don´t have to do EVERYTHING they suggest, you simply have to give valid reasons why you did not or why you think that it is beyond the scope of this study.

 
At 7:43 AM, Anonymous Betsy said...

Over the years, I've learned that it is futile to try to submit the "perfect" manuscript. The reviewers will always find something more for you to add, especially if it's at a high-profile journal.

Go ahead and submit what you've got. You're never going to know what the reviewers want until it's out there. If you have time, redo some of the experiments while it's out for review so if they ask for it, it's done. You shouldn't be deceitful, but don't highlight the flaws either. It's not your job to point out what's missing from the paper, the reviewers will be more than happy to do that for you.

And always remember the story of The Admiral's Pipe:

http://blogs.msdn.com/brada/archive/2005/05/12/417064.aspx

 
At 10:04 AM, Blogger alamode said...

Rinse, wash, repeat...

Send in the paper, expect it to need modification of some kind. When the reviews come back, try to respond to them within reason, or go to a "lesser" journal.

We have a prof in the dept (Clinical oriented Mol. Bio/oncology) that routinely knowingly sends in incomplete papers and then does whatever experiments the reviewers ask for (within reason). Saves him a bunch of time trying to "out-think" the reviewers and in the long run saves everyone in the lab a lot of time doing those extra experiments that turn out to be unnecessary. His people all get papers, and his publications end up being quite good.

Do you get more recognition for having publications that were 100% perfect on the first submission or from those that took three rounds of review and re-submission? In the end no one cares either way, its just total number of your pubs with maybe some bonus for the overall "impact factor".

A long winded reply... sorry. Send in that paper!

 
At 10:15 AM, Blogger NeuroChick said...

You have to be careful, though, when sending in a paper that is glaringly incomplete. Journals don't send every manuscript that they receive out for review, and many won't accept a resubmission after that unless it's been drastically improved. If you're aiming middle-top to top journal, I wouldn't send in an incomplete paper. They get too many papers to waste time reviewing one that is obviously incomplete.

If you're missing an experiment that would just be more of a "kicker," then maybe you could get away with sending the paper in. But if the experiment is critical to understanding the other work in the paper, it's best to hold off, do the experiment, and then submit. If you've already done the experiment and it's just a matter of getting a prettier gel, picture, or whatever, then it's probably best to submit it for review, and then add in the new stuff when the revised version is sent in.

 
At 8:04 AM, Anonymous Katie said...

I'm going to apologize here, because I have absolutely nothing to offer in the way of advice. But when I read your post, I got so excited because I have some problems with a couple of my papers I've been holding back and can't figure out what to do with them. So, basically, I can relate. It sounds like it's a judgement call - and a relatively hard one - but I'm sure you'll figure it out. If it matters, I hope the review process goes abnormally smoothly for you. It sounds like you could use a break here.

 
At 10:28 AM, Blogger Meredtih said...

If your experiments are complete enough to tell a convincing story, at least write up a draft of the paper. I agree with Dr.J- write the paper so that the story makes the most sense and is most convincing- you don't have to present it in chronological order. Put the draft aside for a bit, and then read it again, being as objective as possible- and have a colleague read it. If, after reading it, you really feel like you need those additional experiments, then bite the bullet and do them. If the paper is convincing enough without them, send it in for review. I agree with the commenter that advised you not to turn it in of you know it is incomplete or needs a lot more work, because this will piss off reviewers and like the other poster said, the editor might not even send it out for review. The paper should be as complete as possible when you send it in. Will you be proud of the paper without those additional experiments, even if the paper gets published without them? You need to be happy with the paper, because people judge you by your publications. But by the same token, you will never publish in a timely manner if you are too much of a perfectionist. Good luck with your paper!

 
At 8:48 AM, Anonymous Anonymous said...

I pretty much think that the reviewers feel pressed to say something is wrong with the manuscript so they write it up and ask for one or two more things-regardless. I often prefer to hold back one thing and when they ask for it it's already there (or very close) and I can finish it up really quickly and then it gets accepted.

 
At 9:06 PM, Blogger Abel PharmBoy said...

I'm very late to the ball here, mostly because I've been backlogged with reviewing papers. Mostly for journals with IFs of 8-12, so probably not of the stature you're looking at, but here are my two cents:

If missing the other experiments make you look lazy, re-do them in your current lab. I agree with whoever said that some reviewers just need some one thing to cite, but still give you an "accept with revisions" decision. This shows how I've learned to play the game: I usually leave one easy thing out of the paper that we do while it is under consideration. This goes against my nature, but 15 yrs with my own lab has made this a survival behavior.

However, the point is well-taken that some very high-profile journals don't always send things out for review. If you're looking at journals with IFs above 20, go ahead and include the missing expts. Disclaimer, I've only had one paper sent out for review by Science, but not accepted - I think that my top papers are in JBC, but I had enough of them to get a tenure-track position and keep R01s for a decade or so.

I hate this fucking game sometimes, especially when I'm working on something that I know will help my wife's patients but can't get funded because it's not timely or (she HATES this term), sexy.

Good luck, and let me know if I can help.

 
At 12:04 PM, Blogger Bill Hooker said...

The other way is to try to hide the flaws by the order in which I present the data

Purely personal: I really hope you won't do this. I hope you will submit the honest version, with or without extra experiments, and be part of a change in the culture of science rather than going with its current flow. It's fine to structure the paper so as to best present your ideas, but that -- as you clearly understand -- is a different beast. The way so many scientists view the process as a game, as so many steps where you try to "get away with something", drives me bugfuck. I get the feeling it bothers you too, and if we (I'm a lowly postdoc too!) don't say "enough" it will just continue for another intellectual generation, and another, and another...

My $0.02 that may be of actual use to you: You can always argue. I've had several papers accepted over the objections of obnoxious referees, because in the end it's the editor's decision and if you're right on the merits then you'll get published.

 
At 7:36 AM, Anonymous Writing a Research Paper said...

Many institutions limit access to their online information. Making this information available will be an asset to all.

 

Post a Comment

Links to this post:

Create a Link

<< Home