That's why it says 'plus OR minus'

Here's the key graf from Sunday's lede story (right), showing a 2-point gap in the Ohio governator's race. Ready to play "spot the error"?

Pollsters interviewed 839 likely Ohio voters by home phone or cell phone Oct. 14 through Monday. The poll has a margin of sampling error of plus or minus 3.3 percentage points, meaning the Kasich lead could be as much as 5.3 points or Strickland could be ahead by 1.3 points.

Got it? Very good! The confidence level applies to both proportions, and it could be "plus or minus" for each. Kasich's lead could be 8.6 points, or Strickland could be up by 4.6 points, and we'd still be in the realm of nonchance cases -- a sample that, given our 95% confidence level, accurately reflects the population of "likely voters."

That's worth a correction, of course, but it's a correction in a good cause, so it's worth a moment's look. The Dispatch is trying to tamp down some of the mystical cozmik certainty that newspapers tend to use when they're talking about surveys, and they shouldn't be discouraged by -- oh, you know, a few arithmetic errors on the front page.* So let's talk about how to get it right next time.

Using specific examples to show what a result like "49-47" means is a good start, but it ought to go a bit farther. Specifically, it shouldn't just talk about the range but about the likelihood that some result falls at a certain place in the range, and thus about how and whether a 4-point Kasich lead is more likely than a 4-point Strickland lead.

The magic word is "random." Random samples of some normally distributed phenomenon are eventually going to form a normal distribution themselves. (Go ahead and call it a "bell curve" if you want.) Draw a horizontal line with tick marks for every unit from 40 to 55 inclusive. Draw a vertical line for each proportion (49 and 47). Now draw a horizontal line 3.3 ticks in each direction from the base of each proportion. Draw a nice big Christmas bell for each proportion, with its base 6.6 ticks wide. That's the area within which, 95% of the time, a properly drawn sample of the population -- all likely voters -- will fall.

You can see that there's a pretty good amount of overlap: more than there would be if the gap was 3 points, and a lot more than if the gap was 6 points.** It's clearly possible that Strickland is comfortably ahead, but it isn't very likely, and that's the distinction the story needs to explain. This result is most likely to suggest a population that's somewhere between "slightly behind" and "about even."

In hed terms, that's pretty boring. It should be. Surveys tend to produce incremental and tentative results, even if newspapers tend to use them as an excuse to drag out the "CLAWS HIS WAY BACK INTO FIGHT" stuff (which is almost never what it's cracked up to be, so don't). But done right, they're not only quite expensive but actually quite informative, so we ought to take some pains to get the nuances in order. As in this from last month:

Like all polls, the Dispatch Poll is subject to possible error other than sampling error. Other sources of error include unintentional bias in the wording of questions, data-entry error and nonresponse bias. Nonresponse bias means that those who responded might not necessarily reflect the views of those who did not participate.

That's a pretty good summary, and it's worth doing often because -- ahem, well, the problem isn't that the peasants are at the gates, it's that the well-financed lying gasbags who claim to speak for the peasants are at the gates. Part of Limbaugh's interminable rant the other day was about those liberal media polls -- how they're not designed to measure public opinion but to influence it. That's the sort of crap a well-informed 5-year-old could see through, but if the 5-year-olds aren't well informed, they aren't going to do their part.

Your Editor was actually polled tonight*** on a statehouse race, and it was pretty clear what steps had been gone through to keep assorted biases out. It's equally clear when an outfit like the Fair 'n' Balanced Network, which generally buys a pretty honest survey, is deciding that it needs to stack the deck: Do you think it's time the Kenyan Muslim socialist in the White House stops blaming Bush, or do you really not care if he accepts any responsibility for driving the country to ruin? That's the sort of thing we can influence, maybe, a little, in the long run, if we're unfailingly open about methods and unfailingly consistent about results.

* Yeah, it's from one of the cooperating papers in this brave new initiative. That doesn't mean you don't check the arithmetic -- and call the originating paper if it's wrong.
** Seven points would be a statistically significant lead; there would be no nonchance cases of overlap. I know some of you out there in readerland don't want to maintain this technical meaning for "significance," but -- hey, social engineering in the media has its virtues.
*** Leftovers aren't a problem, but -- please don't screw around if I'm actually cooking, all right?