Tuesday, February 16, 2010

Today's do-it-yourself tip

Suppose you get an irate phone call or eighty over the next few days from readers demanding to know why you and your Marxist colleagues are suppressing the bombshell story in which the chief Climategate perp confesses that global warming is a hoax. (It has to be true; after all, Rush said it, and there it is at the Fair 'n' Balanced Network!)

Well, sorta. Here's how Fox puts it:

The embattled ex-head of the research center at the heart of the Climate-gate scandal dropped a bombshell over the weekend, admitting in an interview with the BBC that there has been no global warming over the past 15 years.

Wanna know what he really told the BBC?

Do you agree that from 1995 to the present there has been no statistically-significant global warming?

Yes, but only just. I also calculated the trend for the period 1995 to 2009. This trend (0.12C per decade) is positive, but not significant at the 95% significance level. The positive trend is quite close to the significance level. Achieving statistical significance in scientific terms is much more likely for longer periods, and much less likely for shorter periods.

See why we keep on complaining about people who ignore confidence levels? He's not saying there isn't a trend; he's saying it's a trend that's close to significant at conventional levels (95%) but not quite there.

So what do you tell the reading public? OK, imagine you have some dread disease -- infectious palinitis or something, usually fatal within a year of diagnosis. There is no cure. But there's a new miracle drug we've been testing, and people seem to be symptom-free two years later. Want some?

Dunno. How good is the study? It's very good, but as with any such study, there's a chance we got our result by accident; it could be a false positive, meaning we got a good result by some accident unrelated to the treatment. There's about a 7 percent chance that it won't help if you take it. Still want some?

I would, even though we're hovering around 93% confidence rather than the arbitrary 95% level that the story mentions. That's the difference between "approaching significance" and "significant," or the difference between "no global warming" and "no statistically significant global warming" if you get your news from Planet Fox.

Your callers aren't necessarily stupid (though you shouldn't rule that out). But they've demonstrably been lied to -- by Limbaugh, which isn't a surprise, but also by a purported news outlet, which should be. Who knows? They might appreciate finding out what they're missing.

Labels: ,


Blogger John Cowan said...

And even if you get statistical significance right, the really important question is "What is the effect size?" You may have a highly robust result, significant at the 99.9999% level, showing that drug A is better than drug B; but if drug A is only .0000001% better than drug B, it's not really a big deal.

12:15 PM, February 16, 2010  
Anonymous Anonymous said...

Or as the journalism professor who introduced me to media research liked to say, "People get real carried away insisting on the 95% or 99% confidence level. But if someone gave me odds on a horse winning a race at the 65% confidence level, I'd put a lot of money on that horse." Oh, and I hope I never get palinitis. Maybe that's what's been wrong with me.

12:57 PM, February 16, 2010  

Post a Comment

Links to this post:

Create a Link

<< Home