Tuesday, June 24, 2008

The Messy Phase of a Scientific Problem

-juniorprof started an interesting discussion about whether to be a producer or consumer of the literature. I started to write a comment in response, but it sorta ballooned up, went off on a bit of a tangent, and so I moved it here. Be sure to check out his post there (and his blog in general, it's a good one).

Overall, I find the message behind the dictum of the famous neuroscientist that juniorprof relates, which is to "be a producer not a consumer [of the scientific literature]" to be kinda ambiguous; of course all scientists need to have some engagement with the body of work they're involved in, and of course it can be easy for some folks to be too wrapped up in the literature to the detriment of their own creativity and experimental production. As juniorprof added in the comments, with which I agree, it's best to start a new project by going through the literature, and then after that to let your experiments guide you.

As part of his post juniorprof also warned against getting too wrapped up into the minutiae of details that can at times overcome a particular subfield (the state of which can be diagnosed by the presence of negative results against this or that hypothesis). I like that too in principle, but I can think of a couple examples where it took the field several years and a lot of negative experiments to converge on a better understanding of a process, and in so doing helped to reveal a more broadly interesting and relevant results.

Cases in point are the signalling mechanism that causes M current inhibition by receptor activation, and whether long-term potentiation (LTP) occurs as a result of changes in pre- or postsynaptic properties. It took a long time before it became generally agreed that PIP2 depletion inhibits M channels, or that postsynaptic glutamate receptor insertion underlies many forms of LTP, and I can think of lots of experiments with negative results done to assess the multiple alternatives. Sure, in isolation any one of those results isn't terribly informative, unlike a good positive result would have been; yet in the aggregate they reflect the shared knowledge of the field which had to be accounted for in the final scheme. And I would argue that only the existence of these negative results enabled the final positive result. (Still though, if you do have a negative result, stick it somewhere where you also have some amount of positive results and appreciable march towards the final answer. And by the gods do NOT highlight the negative result in the title. GAH!).

Also, the final papers which cemented the current understandings of these processes were published by people who were actively researching these questions throughout the messy period when lots of negative results and back and forth papers were being produced. So, I find it unlikely that any single researcher could sit out the ugly phase, work on something else, and then "swoop back" in and make a strong positive situation). Instead, I think that only by those people actively involved and invested in the line of inquiry were likely to have the ability to make the final connections and apply them to their question.

Now, I'll grant first that spending time on expts in the "dirty" phase isn't glamorous, and might be a hard sell to funding and promotion folks. I'm more thinking about how this process plays out for science per se (naive and unwise, I understand). Secondly, you have to make sure your initial process has some level of importance before you decide to commit to it. In this way it helped that M current (as an example of ion channel modulation more generally) and changes in synaptic strength such as LTP were intrinsically important questions. These two also had the lucky effect that their answers became important in and of themselves, as examples of PIP2 modulation of ion channels and activity dependent insertion of ion channels, respectively. Maybe it's not lucky though, since you should probably assume that a question which takes that long to settle is likely to generate an answer that is pretty novel when all is said and done.

So where does that leave us? Hmm, my main dictum is, "Do good science".

2 comments:

juniorprof said...

Like all generalities, this one has some glaring weaknesses and you point out many of them. I think that the point should be not to sacrifice experiment for reading. I, for one, live in fear of talking myself out of creativity so I have created rules for myself to help me avoid this situation.

You're dictum is, of course, the standard of our profession. In the end, nothing else matters.

Nat Blair said...

As I tried to convey in my post (though with uncertain success) I do agree with the underlying message you were getting at. Especially with regards to the creativity aspect. I think I was reacting to the dismissal of the literature (especially with regards to the "messy" phase) which I likely applied more weight to than you probably intended. I honestly think that the messy phase is a natural and indeed necessary stage of the evolution of a piece of scientific knowledge (hmm, I feel another blog post coming...).

I hadn't really thought about the likelihood that excessive engagement with literature might be an impediment to creativity, but that's a great point, and I agree with you that it certainly could for some people. It occurs to me that not only is this the case with what's actually in the literature, but what's not out there. I can think of many times when early grad students have pooh-poohed their own experimental idea, because it seemed so obvious that it must have been tried, and that its absence in the literature meant it failed. That's clearly the wrong tack to take. You gotta do your own experiment, trust your own hands, and see what happens.

Overall, creativity, and how is relates to being a scientist is really pretty fascinating subject. In some respects I see some interesting parallels between the process of science and the process of creating art. That's a subject on the list of things to blog about. Maybe I'll kick it up the line.