Sunday, September 11, 2016

'Well within the margin of error'

In a routine-ish campaign tale, this line stood out:

The latest polls suggest a tight race in North Carolina. Real Clear Politics polling average gives Clinton a 1.2-point lead, well within the margin of error

Apparently it made quite an impression elsewhere, appearing in another story and a blog post (with a touch of editing) in the same day:

The latest polls suggest a tight race in North Carolina. Real Clear Politics polling average gives Clinton a 1.2-point lead, well within the margin of error.
 

The latest polls suggest a tight race in North Carolina. The Real Clear Politics polling average gives Clinton a 1.2-point lead, well within the margin of error.

Come to that, it's had a pretty good run since convention season:

Aug. 4: Polls show a tight race in North Carolina, a key battleground. An average of recent polls by Real Clear Politics shows Clinton with a half-point lead in the state, well within the margin of error

Aug. 3Polls indicate a tight race in the state. An average of recent polls by Real Clear Politics shows Clinton with a 2-point lead in the state, well within the margin of error.

July 25: The candidates are virtually tied in North Carolina, with Clinton enjoys a 2-point advantage in the state – well within the margin of error – according to Real Clear Politics’ polling average.

July 23: Despite the spending disparity, polls show the candidates virtually tied in North Carolina. Clinton enjoys a 2-point advantage in the state, well within the margin of error, according to Real Clear Politics’ polling average.

OK, spoiler alert. Whatever you think about the phrase "within the margin of error" (which is basically journalistic gear-grinding), it doesn't apply here, because the RCP "polling average" doesn't have one. No, really: Look at the top of the chart and you'll see that RCP doesn't even pretend that the average has a "margin of error." That doesn't cause the "average" to be a meaningless number, but it is a result of the "average" being a meaningless number. Hold those thoughts for a moment while we talk about the circumstances under which journalists use adverbs and other modifiers to indicate their evaluation of the events they present to the public.

One is simple social orientation, as in the familiar Anchorperson Adverb: "Fortunately, no one was seriously hurt." The anchorperson isn't suggesting a belief in the mystic power of fate; like bitching about the weather when you enter a sparsely populated elevator, this is a way of reminding strangers in close proximity to us that we're regular people too. We're not going to pull out a chainsaw and take their limbs off in the few moments we share between floors. 

Evaluation can be much more ideologically valenced, indicating not just that you're among the normal people but that you're on the side of all right-thinking Americans. The Fox story at right never says "tasteless," but the hed has a different mission: in agenda-setting terms, it isn't telling you what to think as much as telling you how to think about what you think. (Why this was so much bigger a deal than, say, an apparent truce in Syria that evening is a different level of agenda-setting.) But the perceived need to make us-vs.-them evaluations helps explain why the Observer can take a distinct stand on what "raises questions" in some cases:
... and yet remain so neutral on, say, a national political campaign that matches a babbling reality-TV huckster against a candidate with moderate experience in the executive and legislative branches of government.

Other adverbs are shorthand for news routines and news values; they tell the audience that you're doing your job. That's how "just" got into the running for Most Popular Hed Word in the Washington Post's email updates:

Hillary Clinton’s health just became a real issue in the presidential campaign

The Clinton campaign’s bad damage control just made the health story even worse

Did Hillary Clinton just make her own ’47 percent’ gaffe?

Things just got very, very heated between Rudy Giuliani and Chris Matthews

Mike Pence just upped the pressure on Donald Trump to finally renounce birtherism

 
"Just + [past tense]" is how Kids Today say "this just in": something happened, and you're hearing it here first. "Well within the margin of error" works similarly, even when there is -- literally -- no "margin of error" to be within. "Well within" is a signal of accuracy and impartiality. It means you've vacuumed under the fridge; in case anyone is trying to put one over on the audience, someone has done the out-of-the-way work that keeps everything in order.

Except that -- oh, hell, is this getting too obvious? The "Real Clear Politics average" is bogus. There might be a true value in the population -- whichever of several populations is allegedly represented at any given time -- that resembles the "average," or there might not. That's a matter of coincidence, not statistics. A mean of results from different-sized samples drawn from different populations (and, at the national level, a panel study from USC*) does not produce a number that you can carry out to one decimal point. That's no different from proclaiming that the dinosaur bone you found eight years ago is now 65,000,008 years old. It doesn't matter how often the Post and the Times repeat it; it's not the sort of number that allows you to calculate things like confidence intervals.

"Margin of error" is a pretty simple calculation.** If you can figure out an ERA, you do your own, and that's a good way of launching a bullshit-vaccine campaign. But it won't paint you out of corners that you've painted yourself into, particularly when it doesn't exist for the numbers you're describing.

*This is not to suggest that longitudinal surveys are bad science. They aren't. But they are also very much not the same creature as the cross-sectional surveys in the mix.
** For a random sample of size N, multiply the square root of (.25/N) by 1.96 to get the largest possible "margin of error" at 95% confidence.

Labels: ,

0 Comments:

Post a Comment

<< Home