The Procrustean Bed of Science
Figures present in Greek mythology can often provide startling comparisons to what happens in our modern world. That is why we love them, and the stories in which they reside. Better yet, it is often easier to more saliently recognize our flaws, our failings as humans, and our silly missteps through the eyes of some folklore hero.
Procrustes, “the stretcher,” son of Poseidon, bandit from Attica, held a residence on “the sacred way” between Athens and Eleusis. Here, Procrustes had a bed in which he invited every traveler who passed by to spend the night. Problem was, nobody seemed to fit this bed. Procrustes, being the metal worker he was, set out to work on his ill-fitted visitors and made them fit the bed. A guest proving to be too tall for the bed would simply have his or her legs amputated to the desired length. Too short? Procrustes would simply stretch them to fit.
What motive or moral imperative Procrustes had that impelled him to literally force his guests to fit this bed, I do not know. Perhaps, a gracious host, he was overridden by his desire to see his guests comfortable in the bed, and therefore forgot about the much more serious matters of, you know, bodily mutilation. It would seem that Procrustes near-sighted goal (i.e. get his guest to fit into this one-size-fits-all bed) essentially blinded him to the larger moral, philosophical, and ethical issues along the way which, had he listened, would have guided him otherwise.
Had he taken a second or two to consider other options (new bed?) or even the actual comfort of his own guests, Procrustes could have developed alternatives, and maybe saved a leg or two….
We know refer to a Procrustean solution as one in which some sort of data or output are “tailored” to fit a preconceived notion. To use an example in the field of statistics — instead of first finding the line of best fit for a scatterplot (perhaps to show a correlation), one might first decide the line (or correlation) one wishes to observe, and then pick the data that best fit this line, disregarding any data that does not.
Instead of a typical “win-win,” a Procrustean solution is “win-lose” — the winner being Procrustes, who, at the expense of the guest, gets his desired outcome. At least in Procrustes’ mind “the ends justify the means.”
Science has its own Procrustean problem.
Hypothesis testing by nature, is “semi-Procrustean.” When we “feel” like some phenomenon might work one way, and want to prove it, we go out of our way to develop experiments and trials that might lead us toward a conclusion. However, where the problems lie is when hypothesis generating encounters one of two issues.
- We “know” our hypothesis will be true from the outset, and therefore are destined to “prove” our theory to be right. (Maybe testing a “duh” hypothesis)
- We are “married” to our hypothesis so much in that our scientific method includes the “perfect storm” of variables which are likely to lead to our desired results. Basically, we fit the trial to our outcome.
(Okay sorry one more)
3. We search for data that supports our hypothesis (or conflict of interest) and knowingly disregard the other side of the coin — data that contradicts it.
The world of science, in particular peer-reviewed publishing, has a bias towards positive results. That is, we want something “proven” rather than a finding of “no difference” or even worse, a finding in direct opposition to what we thought might happen.
This positivity bias contributes to the Procrustean problem of scientists developing experiments and “fishing” for data that supports a theory they like, or have developed themselves.
Say, for instance, that I want to write a paper discussing the unhealthy properties of cashews (particularly salient, as I’m eating some now.) I can do a literature search on “cashews,” perhaps throw in a few key words like “heart disease” and “cancer” and “high blood pressure” and, for good measure, maybe add “choking” (a very probably risk!) What the database spits out might contain a variety of articles, a few likely referencing the adverse effects of cashew consumption on heart disease risk (probably an association study).
Hypothesis supported.
Of course, the literature search will also provide me with numerous articles on the benefits of cashews — “they’re heart healthy!” However, the Procrustean scientist, tools of self-validation at hand, can simply ignore these articles as if they were never published.
A recent example of this can be seen in the much maligned “AHA Presidential Advisory” on Dietary Fats and Cardiovascular disease which concluded
Taking into consideration the totality of the scientific evidence, satisfying rigorous criteria for causality, we conclude strongly that lowering intake of saturated fat and replacing it with unsaturated fats, especially polyunsaturated fats, will lower the incidence of CVD.
Whether this is true or not likely is quite complex and, like most things in science, can probably be summed up by “it depends.” The point here is not the conclusion, but what critics of the conclusion and the paper overall cite as the major flaw: cherry picked data.
What the AHA could have easily done (and probably did) was search for studies showing a link between saturated fat and CVD, the benefits of polyunsaturated fatty acids for CVD, and disregard all the rest. Large, observational diet studies which confirmed the pre-formed hypothesis were included; those which didn’t, well, you know the drill…
All of this assumes that the AHA had ideas about heart disease and saturated fat from the start. Unfortunately, the history of the medical profession suggests they most likely did. Conflict of interest may have shaped the investigation. This, however, may have been no diabolical plan on the part of the AHA (let’s hope not). Conflict of interest influences us all, whether we know it or not. Like Procrustes, we all have a desire to cut and stretch this world to fit the theories that we’ve concluded about it. Most scientists would like to have their work supported by data.
Knowing a problem exists allows us to address it. As scientists, trained in ethics, rigor, and unbiased approaches, we can proceed carefully into the realms of hypothesis testing, seeking to find an answer — unembellished as it may be — to a question; whether it supports our theory or not.
Rather that stretch, mangle, and force our data (our guests) to our hypothesis (our Procrustean bed), it seems best to find our bed, and then wait, patiently, until our guests, those too tall and too short, have all made their pass, and the one guest who can lie comfortably finally passes by. A win-win for all — a true fit.
Otherwise, we might just have to find a new bed.
In the end, Procrustes reign of terror ended, when Theseus, passing along the sacred way, captured and then “fitted” Procrustes to his own bed.
Let Procrustes’ fate serve as a warning. If we insist on fitting the world to our standards, rather than adjust our theories accordingly, we may, in the end, be stretched too thin, perhaps broken in half, by the world we refused to listen to.
