A nice survey to ignore

Want to save some space in the Wednesday paper? Go ahead and ignore the story about the Pew Form survey on American religious ignorance. (Oops -- already on the front page? Well, enjoy the cautionary rant anyway.)

I'm not suggesting it's a poorly conducted or faulty poll. (It has some conceptual design issues that should be clear in a second, but don't we all?) The problem is that it's the sort of instrument that says very little of any real interest while being attractive enough to be trumpeted for all the wrong reasons by all the wrong sorts of people.


Since the Pew Web site seems to still be busy,* here's the AP's take:

A new survey of Americans' knowledge of religion found that atheists, agnostics, Jews and Mormons outperformed Protestants and Roman Catholics in answering questions about major religions, while many respondents could not correctly give the most basic tenets of their own faiths.

As a rule, you should avoid being shocked by polls that find more Americans can name all four Beatles than some particular number of Supreme Court justices. People learn about the Beatles and the Supreme Court in different ways, and frankly, most of us are much more likely in day-to-day life to run across a situation where we have to name a Beatle at random than a Supreme Court justice. If anything, it suggests we've managed to come up with some sort of good working compromise with the world when it comes to how much information we're supposed to cart around every day.

So the "so what" question is foremost, but let's look at a couple of really stupid ledes first:

Holy cow, Americans are ignorant about their own religions. (New York Daily News)

If you want to know about God, you might want to talk to an atheist. (Los Angeles Times)

Americans are clear on God but foggy on facts about faiths. (USA Today)

The first question you should ask about a survey (once you've established that it comprises a proper sample of the population in question), even before the one about the money, is: What did it measure? The LATimes is the head evildoer here, because -- duh -- the survey doesn't ask what people "know about God." There are some gods in it, sure, but it's mostly about whether people know about Joseph Smith, Martin Luther, Maimonides and people like that. The writer's confusing religious history with religious belief -- exactly the sort of cluelessness the study finds in one of its more illuminating comparisons.

In general, though, it's the sort of dreary stuff that happens when you give multiple-choice tests to a random sample of the population. It's distinguished mostly by the ways writers find of using irrelevant facts to support their preconceptions. Here's the first factoid the Daily News highlights:

More than 50% of Protestants asked did not know that Martin Luther sparked the Reformation.


Uh, OK. You have to nail a reasonable amount of the Small Catechism to rate confirmation as a Lutheran, but how often does the guy come up among your Southern Baptist (that'll be the nation's largest Protestant denomination) friends? That's the sort of card-stacking trivia reporters specialize in: telling people to be shocked, without giving them the background to figure out whether they agree.

You can go find the rest of the questions elsewhere yourself. The bigger problem with this sort of polling is not that it's wrong or evil on its own, but that having already misused it to support its own biases, the press is poorly placed to call out other people who do the same thing. Here's Steven Prothero of BU at CNN, flogging his own book while renewing his call for "mandatory public school courses on the Bible and the world's religions":

In "Religious Literacy," I described our collective religious ignorance as a civic problem of the first order. How to hold politicians who pin their public policies to the Bible without knowing something about that text?  And how to make sense of religious conflict in the Middle East without knowing something about Judaism, Christianity and Islam?

Forgive me if I'd like religion profs -- with journalists right behind them -- to be at the head of the line in debunking this sort of thinking. If politicians are pinning policy to the Bible, the last thing we want to do is assess their fidelity to it. If we can't start by figuring out which conflicts in the Middle East are religious and which aren't,*** we're not doing the world much good. And how do you figure the balance will stack up in a course in "the Bible and the world's religions"?

I'm not worried by what people don't know about religion. I'm prepared to be quite worried about how much they know that ain't so -- how we managed to get such a flock of people loose at Fox News slinging around terms like "dhimmi" and "taqiyya." That's a study that might be kind of fun to see on the front page. Call me when you see it.

* From which you may draw conclusions about how the survey's relevance is playing out in the real world.

** If you keep up with this sort of stuff, you shouldn't be shocked by anything that survey research turns up, which is another good reason for keeping it off the front page.
*** If secular European Jewish socialists and secular Arab Christian socialists start shooting at each other over the same plot of land, you might fairly conclude: It's the land, stupid.