When data changes

[Barrett Dorko]

When the character Data on Star Trek sought to change he gravitated toward something closer to human – which means he wanted to be less certain and somehow more emotive than what he was originally. Occasionally, he managed to make some progress in this direction. Every trekky knows that this came with a price.

Anyone interested in the findings we rely upon as evidence-based practitioners would be foolish not to listen to this week’s episode of On the Media, especially the last segment done in conjunction with Radio Lab.

If I understood this correctly, the degeneration of the numbers found in an initial study is both inevitable and gradual. The former perhaps represents the effect of tightened controls and brings to mind the letter Jason Silvernail and I wrote titled The Method is Not the Trick a few months ago. The latter - a gradual movement toward something less dramatic, remains a total mystery. This fact really got my attention.

Data behaves this way when many factors are involved that go unheeded or cannot be controlled.

Now, am I talking about the numbers research gives us or am I referring to the robot?

[John W]

If you look up the word "mean" you'll find that aside from it having many different meanings and usages, its derivation is from the German and Dutch words for "in common" or "common".

Since, in behavioral statistics parlance, we're talking here about "regression to the mean", the inexorable tendency for data to revert towards the most base and common level upon repeated testing, then I think it's helpful to understand the many usages of the word "mean".

Many who show up at Soma Simple think we're mean here. And I don't mean in the sense of being common or average, but in the sense of being not nice. This is generally a misconception, and I mock it in the title above my avatar. We are mean, in the sense that we employ Occam's razor.

I'd go as far as to say that we wield a mean Occam's Samurai sword.

[joebrence9]

There was a great piece about this concept in "The New Yorker" last year. It explained the history behind the decline effect, regression to the mean, publication bias, etc. Check it out.

[Barrett Dorko]

I copied it back then and thought it would become a blog post but, like many others, it fell by the wayside.

A great article.

Thanks Joe.

[Seth]

I also thought this was a fascinating piece on radiolab. From what they said it sounds like regression to the mean was only one piece of the puzzle of why the statistically significant data seems to disappear over time. The hypothesis was thrown out there that perhaps somehow the act of observation itself could be a factor. A crazy thought... Barrett, you mentioned on facebook that the original study on the multifidus and it's relation to back pain was never repeated?? Has everyone really been integrating this reductionistic approach into clinical practice based on one study? Hopefully, the growing trendiness of neuroplasticity will overtake the current trend of the core strenghtening regime...

[Barrett Dorko]

Perhaps someone here knows of some replication. The other problem was a misinterpretation of what it actually demonstrated and what that meant.

Anyone?

Seth, Thanks for posting. We're all wondering who you are.

[Diane]

Hi Seth,
Welcome to SomaSimple. Here is a link to our Welcome Forum, so you can start a thread and tell us a bit about you, and where we can welcome you properly.