Sunday, February 03, 2008

Where are the holes?

As a comment to my last post, Gosh asked how you know where to go in science to find something new, and says it sometimes feels like you'd have to read a lot of textbooks since it seems like everything is already known.

I've written a lot about how much I think reading helps. It's not the whole thing, but it is a big part.


One professor I had in college assigned us two giant textbooks to read over the summer.

Maybe I'm the only one who actually did (?!).

I liked the message it sent:

We're not going to go chapter by chapter and ask you stupid quiz questions every week. We're going to ask you to absorb as much of this as you can, so we can move on to more interesting discussion. But we can't talk as equals until you've been exposed to the vocabulary and the concepts.


There are at least three main pluses to reading well-written textbooks:

1. Outstanding questions. Things we've wanted to know that we still don't know. Good textbooks will just tell you plainly, "Here is an interesting idea. We still don't know how this works."

At the root of it, this is the main thing that made me want to get into science. The idea that everything is NOT already known!

I was shocked to learn this. Weren't you?

2. Historical context. Sometimes it's downright funny how wrong the old models were, but you can often see exactly why they thought they were right, given what they knew at the time.

3. Practice reading for assumptions. It's partly language, and it's partly smell - you start to sniff out the leaps in logic. Textbooks are a great place to learn how to tell when something has been 'assumed' but never actually tested.

Textbooks are where you learn the rules. How will you know an exception if you don't know the rules?

Change favors the prepared mind (no, that's not a typo).


Obviously, textbooks are are biased - everyone is biased, we admit it. And often outdated, sometimes dangerously so (at least in my field). Neither of these would be a problem, except that beginning students aren't always taught to keep these things in mind.

The first thing you need is a healthy dose of doubt. Don't take anyone's word for it that we actually know something. Anything. Figure out which stuff you believe. To do that, you have to read.


Then you have to learn the limits of the technology. What can you really say. Where are the holes. What's the limit of detection. What's the error.

I can't emphasize this enough. My main frustrations with colleagues lately are that they don't seem to understand the most basic techniques that we all use. What you can and can't say using qualitative or quantitative methods. When it's inappropriate to just report a mean, or just show one example. The difference between a good reagent and a bad one, and how to tell which is which.


To start to learn this, you do experiments. And you read papers. LOTS of papers.

When you read a lot, patterns emerge. In most fields, there are groups of authors who dominate the Big Journals, and their papers converge on an accepted model. It's the pat-each-other-on-the-back phenomenon. You'll know it because they all cite each other, and they don't discuss outstanding questions or exceptions (even if they were the ones who authored them in the first place).

But in every field, in other journals, there are odd little observations that don't fit with the accepted model. You usually can't find these by reading citation lists, you have to go directly to the library (Pubmed, Google Scholar, whatever).

You have to read both kinds of papers.

Usually the accepted model is mostly right.

But it's almost NEVER Completely Right (!).

Usually the odd little observations are in odd little systems that are less well understood, obscure organisms or uncommon techniques. And that is why they are easily ignored or dismissed. They are often badly written and almost always hard to understand.

But biology is all about exceptions, because every rule gets broken somewhere.

You have to read both kinds of papers to find the holes. The mainstream papers are all about "pay no attention to the man behind the curtain!"

If you just read those, of course it will seem like everything is already known.

It's not.


Sometimes when a rule breaks, it's an example of another way to solve a biological problem. Parallel evolution, maybe.

May be telling you something important.

Sometimes an exception is just an exception. Differences can be tolerated, even if they don't excel.

The gamble is guessing, when you find an exception, whether it's just tolerated in a marginal system, or if it's telling you to think about a paradigm shift. Or a way to do something in vitro.

A great example are thermophilic bacteria. Once upon a time, people would have told you, there is no way life can survive in a place that hot.

Fast-forward a couple of decades, and there is no way we could do the things with PCR that we do now, thanks to polymerases from those thermophiles that can survive 30 cycles in a heat block.


So reading is great, but I actually get most of my really new ideas when I'm observing the systems I work in by doing experiments.

Beware the risky all-or-nothing projects: they just lead to dead ends.

Here's how to tell the difference.

For a good observational project, if it's designed correctly with controls and a broad enough hypothesis, you're going to learn something no matter what.

You'll have the opportunity to make observations by stressing the system just enough.

For a risky all-or-nothing project, you'll often hear the phrase, "If this works..."

The reason "if" features so prominently is that if it doesn't work, you've got nothing. Not even new information about what to do next.

For better or worse funding, I design my experiments so that, if all the controls work the way they should (and they don't always work as expected!), I will learn something.

Something new.

Usually along the way, I see something I didn't expect to see. And then I go design an experiment to test whether that's real.

So to summarize this whole diatribe: pay attention to the exceptions. If it's reproducible, it's probably not just an anomaly.

The trick is having to nerve, the time, the resources and the permission to find out which ones are anomalies by testing them.

Labels: , , ,


At 2:27 PM, Anonymous JR said...

You know...I am really starting to reconsider you. Maybe you really are only fit for an academic career. Do you think any of your readers, including the poor kid that asked the question, actually read this post completely?

At 7:52 PM, Blogger Ms.PhD said...


I'm not sure how to take that. You might have noticed a really negative (I'm being euphemistic) comment on another recent post, so perhaps you should clarify whether you're poking fun or what?

Do you mean my list is too long and wordy, and therefore very academic?

I will take that as a fair criticism. But I suspect you meant something more.

At 8:13 PM, Anonymous JR said...

Nothing more. Just long and wordy.

At 9:01 PM, Anonymous Anonymous said...

I read it completely :)

At 10:03 PM, Anonymous Anonymous said...


At least one anonymous reader (me) read the post in full.

At 7:07 AM, Anonymous Anonymous said...

This is incredibly insightful. Really. My only criticism is that it has a little too much of the "everyone else is doing science wrong and I am the only one who knows how to do it right", but if you can strip that out of it, you really have very good things to say.

I'm not poking fun. I'm completely serious.

A couple of years ago when I was preparing my teaching statement for my tenure files, I realized that I had been modeling my teaching after experiences I had as a student that resulted in some kind of enduring memory. I've tried to integrate all the good learning experiences I had as a student and eliminate all the less useful stuff. And by and large the student response has been favorable. That's essentially what you are doing here, and something that we often forget when we step in front of our class to lecture. WE're here because we've had some incredible learning experiences, and thoe are what we need to pass on.

You have some excellent ideas on learning and teaching science, and you present them very clearly. Lose the 'tude, and develop it a little more.

At 7:30 AM, Anonymous bsci said...

Really nice post.

You glanced at this, but the most significant thing for me was to attend journal clubs starting as an undergrad. Sitting with a group of faculty and grad students as they tear apart papers written by famous people in respected, peer-reviewed journals does more for realizing all the questions aren't answered and also for figuring out how to find those unanswered questions. If someone is at a place where this doesn't exist, make one. I've done that a few times too.

It's also sometime fun to trance back references. Getting back to 100 year-old papers sometimes leaves you in awe of some really amazing research and sometimes leaves you thinking that a whole field was based on this?

At 9:56 AM, Blogger Jennie said...

I love this post.
One of the main concepts I've learned from my graduate adviser is to never accept anything at face value. While I thought that during undergrad and my research there I learned to question things I realized I actually didn't once I started grad school. I had this assumption that well, this is a research paper in a journal it has to be correct. But I'd talk about data in some publications that didn't make sense to me with my adviser and he'd think for a while and say "their data must be crap"
while this may sound pompous it really wasn't meant this way and it's taught me to be more critical about what I read.

At 2:43 PM, Anonymous lost academic said...

I hope you will not be too insulted when I say that for the first time in a long while, this is a post of yours I have not been disappointed to read from you. You've got so much going for you, and even knowing the blog is yours to vent with, I get worried and upset reading it--plus not a little fatalistic about your chances. However, I thought you'd appreciate hearing again something that sprang to mind as I read over what you'd written--I'm sure you've seen the movie Men In Black, and recall what Tommy Lee Jones said to Will Smith, sitting on the park bench.

"A thousand years ago everybody knew as a fact, that the earth was the center of the universe. Five hundred years ago, everybody knew that the Earth was flat, and fifteen minutes ago, you knew that humans were alone on it. Imagine what you'll know tomorrow."


Post a Comment

Links to this post:

Create a Link

<< Home