Support your local copydesk: A slight return

If you didn't see this McClatchy gem, there could be a couple of reasons. It could have been squeezed out by late-breaking Hannah Montana coverage. It might have fallen out to make room for some service journalism. Or it might have been headed for print when some skeptical editors (kudos here to the S.C. bureau's northern coastal office) pulled it over and asked for some identification. Let's watch:

WASHINGTON — Democrats and Republicans in Iowa, New Hampshire and South Carolina have dramatically different views of the nation's priorities, according to a new McClatchy-MSNBC poll. The fact that the two parties' bases don't even agree on which issues matter most may help to explain why the people they send to Washington have such a hard time agreeing on anything.

That's not a bad concept for a poll to explore, and this isn't necessarily a bad poll. But by cheating on methods and reaching for a Profound Conclusion that the data don't (and can't) support, it ends up making for a really silly story.

While pollsters didn't offer each side all the same choices, Democrats and Republicans in all three states differed widely when asked about the same issues. Voters were asked to identify which issue they felt was the most important.

Guys? You're doing it wrong. The traditional way to run a "most important problem" survey is open-ended: You ask people what they think is the most important problem facing the country and write down what they say. You do get a fraction of respondents identifying the Sinister Masonic Conspiracy as the MIP, and a higher fraction worried about how Kids Today got no respect for the law and blah blah blah, but you end up with responses that you can usefully compare by gender, party ID, ethnicity and the like. What Mason-Dixon appears to have done at McClatchy's request* is to read half a dozen issues** to the Democrats and half a dozen overlapping but not identical ones to the Republicans. Republicans weren't asked about Social Security or "environment, energy and climate change"; Democrats weren't asked about "moral issues" or "taxes and government spending."

OK. You could argue some a priori reason to ask different stuff of different subsets, and that's true as far as it goes. You can justify not asking men whether they're worried about getting ovarian cancer, but that's very much not the same thing as refusing to ask Democrats if they're concerned about "moral issues" or Republicans if they're worried about Social Security. There are propositions you could test this way, but it's the wrong way to test the one claimed in the lede. That's cheating.

The top-down methodology points to another bunch of dubious conclusions as well. Among the evidence supporting the "they're from Mars, we're from Venus" conclusion in the lede is the significant difference on questions of "national security or terrorism" (Republicans think this is a big deal) and "Iraq" (big points from Democrats). The lede is assuming that the conceptual dust has settled a lot more than real life suggests -- basically, that "terrorism" and "Iraq" are exclusive categories in public opinion the same way as "Clinton" and "Obama," or "boxers" and "briefs." That's a really risky assumption, regardless of your views of the War On Terror® and its relationship to Iraq. But it's not an unusual one; CNN had a poll (conducted by Opinion Research Corp.) up last week with separate "who's winning" questions for Iraq, Afghanistan and the War On Terror®. We might be able to fix that sort of confusion at some point, but we can't pretend it isn't there now.

Can you cure that with a traditional MIP approach -- letting respondents generate the categories and drawing conclusions from there? Not entirely. You still can't tell for sure whether "terrorism" means Iran, the FARC or Timmy McVeigh. But you do have a case that you're measuring what's on the tip of the public tongue to start with, rather than what's placed there, and you have something you can use longitudinally.

There's a lot of other silly stuff in the lede. Sampling "likely caucus voters" or "likely primary voters" in three states doesn't allow for inferences about what the "parties' bases" think. "Most important issue" means exactly that; it isn't a measure of whether or how much other issues do or don't matter. And the idea that different parties identify different leading problems might be, oh, sort of why we have party systems in the first place.

Anyway, lots of talk that shouldn't take away from the main point. A local copydesk looked upon a ponderous submission from the Washington bureau and, finding it bizarre, sent it to sleep with the fishes. You'd like to hope that sort of well-founded assertiveness merits a nice note from the front office.

* Pay fiddler, call tune. Welcome to the band.
** The order was rotated, and "other/not sure" is counted as a separate category but not offered to the respondent. Pretty standard, but always nice to see it spelled out.