Asking simple questions

Honest, I'm not trying to pick on this particular survey, or all stories depending from it, or anything like that. But we do seem, once again, to have a story that's eating up a bunch of front-page space without, apparently, adding anything to the conversation. Let's walk through it step by step and see if we can raise the questions an editor should have thought of earlier.

As the immigration debate heats up across the country, a new study shows Latinos in Charlotte-Mecklenburg are as divided over immigration reform as any other group — and possibly more so.

That's a lot of assertions and implications for one lede, so let's skip the bit about whether we measured the degree to which the immigration debate is heating up across the country. Is there a new study, is there a measurable thing called "divided over immigration reform," does the study measure it, and are Latinos higher on it? Let's see.

The Crossroads Social Capital study, which measured social ties in the community, found almost six out of 10 Latinos (58 percent) in Charlotte-Mecklenburg feel immigrants are “too demanding in their push for equal rights.”

Oh. Would this be the Crossroads Charlotte Social Capital Benchmark Community Survey, reported back in the first week of June? In which the "too demanding" question was the 17th graf? Can someone explain why "too demanding in their push for equal rights" is a measure of support for "immigration reform"? What does it mean for a greater proportion of a subgroup to agree with this question: is a 50-50 split more or less divided than a 60-40 split?

“I'm upset at some of the demands I hear some parts of the illegal community making,” said Ricardo Mata, a Venezuelan native who has lived in the country for two decades. “Sometimes, I get fed up at the double standards I see.”

So when we shift to real people, it becomes a little clearer that the survey and the lede are talking about different things.

... Critics of the study's findings say they reflect only a small segment of the community and not the majority of Latinos who do support immigration reform.

Really? Do we have comparable data on how this majority feels? Local or national? How are the assertions in this graf going to be supported?

“I think the people who were surveyed were mostly established Latinos who are not having to face this issue,” said Angeles Ortega-Moore, executive director of the Latin-American Coalition.

Here we're getting data confused with opinions again. The survey either did or didn't draw a representative sample; if you don't like the results, that's fine, but this isn't the sort of question that's addressed by asking people what they think.

One hundred seven people who identified themselves as Hispanic or Latino participated in the Crossroads study. The full study's margin of error was plus or minus 3.24 percentage points.
Didn't we discuss something like this before? Why report the confidence interval for the full sample* when we're interested in a subgroup? For which, on this question, the margin of sampling error is 9.4 percentage points?**

While the findings don't appear to track national trends, they do seem to follow economic and generational lines. The longer and more successful Latinos have been in Charlotte-Mecklenburg, the more likely they are to think newly arrived immigrants are too pushy.

What trends don't they "appear to track," and who(m) don't they appear to track it too? Are we comparing questions about immigration reform or questions about whether immigrants are too pushy? Should we note along here somewhere that not all Latinos are immigrants and not all immigrants are Latinos?

The study also shows that Latinos are not monolithic thinkers and that some disagree with parts of the immigrant rights movement.

Uh ... does anybody outside the Observer newsroom really need to be reminded of those points?

Anyway, then we're off into Random Interviewland, talking with people who do or don't espouse the views that the study doesn't address. None of it gets near addressing the points in the hed or the lede, which -- presumably -- were the selling points for putting the story on the front.

The hed, of course, is abysmal; it's a Stupid Question, for one thing, and it's an incorrectly formed question at that. The deck raises issues that the story at least touches on in passing, but if the study's reported correctly here, they're factors in how people view the "demanding" question -- nothing, to bang on the main point again, that has anything to do with views on immigration reform.

And that's really the problem. If the story was sold as a bit of survey data that shed light on how people view immigration reform, it was mis-sold. It doesn't. It rehashes a finding that was a bullet point in a story two and a half weeks ago and dresses it up with quotations that sometimes address the main point and sometimes don't. It's hard to see how the sum of human knowledge would have been hurt by holding this story up until a few fundamental questions were answered.

* Just report it to one decimal place, OK? Two is spurious precision (though it's not as bad rounding to a whole number, as the NYT does).
** At 95% confidence (kids, don't forget those confidence levels!)