The medical hockey stick

Climate research isn’t the only field suffering from misplaced expectations and outcomes. Johb Timmer says We’re so good at medical studies that most of them are wrong at ars technica and describes inherent problems encountered when trying to make conclusions about phenomena that has many inputs and much complexity.

The consensus seems to be that we simply can’t rely on the researchers to do it. As Shaffer noted, experimentalists who produce the raw data want it to generate results, and the statisticians do what they can to help them find them. The problems with this are well recognized within the statistics community, but they’re loath to engage in the sort of self-criticism that could make a difference. (The attitude, as Young described it, is “We’re both living in glass houses, we both have bricks.”)

in the mean time, Shaffer seemed to suggest that we simply have to recognize the problem and communicate it with the public, so that people don’t leap to health conclusions each time a new population study gets published. Medical researchers recognize the value of replication, and they don’t start writing prescriptions based on the latest gene expression study—they wait for the individual genes to be validated. As we wait for any sort of reform to arrive, caution, and explaining to the public the reasons for this caution, seems like the best we can do.

At least in medicine there is whiplash as there are many different studies that reach conflicting conclusions. That is a contrast to the monolithic climate research output. Which is better?

Comments are closed.