Friday, November 23, 2007

Stuff that surveys don't do

Pity the poor misunderstood public opinion survey. One set of people thinks survey research is Good Magic, a sort of cosmic GPS that allows, say, presidential candidates' progress to be tracked with astounding precision like so many off-road vehicles in some bizarre reality show. Another set thinks surveys are Evil Magic concocted by liberal professors from California who use trick questions so that when you say "I like Christmas," your answer comes out in the paper as "I spit on the flag." And it has your picture next to it with one of those Bin Laden hats on.

The truth is a lot more boring. A properly done survey gives a pretty good snapshot of what people say about what they do, or think, or might do. As does the survey discussed today. But not all its findings are equally relevant or useful, which is the problem with the the 1A lede treatment this particular finding got.

Dorothy Runyan and her husband are driving to visit their family in Virginia only twice this year instead of the usual three times.

... Runyan, 73, of Whitehall, said she and her husband try to keep the gas tanks filled on the pickup he drives and the SUV that she uses, but that they try to drive hers as much as possible because it gets better mileage.

They'll need it, polltakers say.

A whopping 82 percent foresee $4-a-gallon gasoline within a year.

That's the sort of conclusion polls don't allow. Whether people expect the price of gasoline to reach $4 a gallon has nothing to do with whether it actually gets there, and the findings of "polltakers" have nothing to do with whether our anecdotal family does or doesn't need better mileage. (And "a whopping 82 percent" overdramatizes the irrelevant, given that almost half those people say it's "somewhat" likely that they'll pay $4 a gallon "sometime in the next year.")

Too bad, because on the whole, this is a pretty well done survey with some moderately interesting results. People say they're driving less and shopping around for good gasoline prices, but in about the same proportions, they say they aren't changing holiday or other travel plans. There's an unusually high proportion (about one in five) of "don't know/refused" answers to a question about whether respondents have started carpooling or taking public transit. And the "yes" answer there is about the same as for a question about whether people have bought a car that gets better mileage. Hmm.

Interesting bunch of numbers about what people say about (leading to some entertaining inferences about what they don't say or what they lie about) how worried they are about the energy situation. It might or might not show that people are changing their energy habits, but it does suggest that some energy habits are easier to change -- or admit to not changing -- than others. Hard to see why that qualifies as a lede -- except that polls are expensive, and this one has a staff byline and a shirttail indicating that seven other staffers contributed, suggesting some pressure to get some mileage out of the thing.

Give this one a B+. It's not inherently evil, but it toots the wrong horns a bit too loudly about the wrong things.


Post a Comment

Links to this post:

Create a Link

<< Home