Well said Jonah.

The story appeared in the New Yorker in December about the alleged ‘decline effect’, a mysterious effect that attenuated scientific findings. Basically, some scientists are finding that after they make an important experimental discovery, each additional experiment they do shows the strength of the effect they found to be smaller than before.

That is super weird. Or at least would be if there weren’t a number of explanations more plausible than a ‘mysterious force’. No doubt due to spatial constraints–even though Lehrer had 5,000 words to work with–Lehrer can only include a limited number of examples of this alleged effect. But my main question is why wouldn’t this effect be more widespread?

Imagine looking at a series of coin flips—let’s say a series of a million coin flips. Somewhere in there you’ll probably find ten heads in a row or a lengthy series of alternating heads and tails. In fact, you’re likely to find a lot of different patterns in there. Does it mean the coin is rigged to produce these patterns? No. A fair coin toss will produce all sort of patterns if flipped enough times.

If Lehrer was writing a story about coin tosses, he would tell you all about the amazing patterns he found, but what about the rest of the tosses where we see nothing? In other words, a few researchers here and there are showing this bizarre decline effect but if that’s the pattern you’re looking for among thousands and millions of publications, you’re sure to find it. Does it mean science is rigged to produce a decline effect somehow? I’ll start believing that when the effect is more widespread than just a few hand-picked examples.

I’ll have more to say about this tomorrow or Friday. There’s a lot of interesting worthwhile stuff in this article but there’s also a lot of stuff presented as weird which has good explanations.

Lastly, I was wondering, would the decline effect itself decline if we studied it more until it didn’t exist anymore? But as the decline effect itself declined it would have less effect therefore it would decline less which mean that it would remain constant. Oh. A paradox.