One born every minute

Everything you could want in a Monday front, isn't it? Football! Shopping! Weather! Lovely atmospheric photo of Orthanc the B-of-A tower! Oh, and a top news story too: Those teenagers! They lie, cheat and steal! And it has to be true, because Survey Says!

Let's admire the lead story for a bit, then, as a reminder that not all the evils done with (or to) survey research are the fault of statistical dishonesty or mendacious question design. Often, evil results come about because of elementary failures in the news judgment process: What makes these results news? What would it mean for them to be true? What's the context any resulting claims are based on? Who benefits if this is at the top of my page?

That's all assuming the survey is properly run, so let's assume it is (neither the version appearing here nor the longer one from the AP bothers to say, which is inexcusable*). What is it that's worth putting at the top of the front page?

In the past year, 30 percent of U.S. high school students have stolen from a store and 64 percent have cheated on a test, according to a new, large-scale survey.**

Oops! Fault one. Neither the paper nor the AP nor the folks who did the survey know how many students have stolen from stores or cheated on tests. This is what "self-report" means; all you know is how many of them say they did. More on this later, but a writer who doesn't understand the distinction really shouldn't be allowed to write about survey data.

Educators reacting to the findings questioned any suggestion that today's young people are less honest than previous generations. But several agreed that intensified pressures are prompting many students to cut corners.

Ah, so it's a trend piece! The pressures of modern life are prompting students to cut corners. But what makes that a top-of-the front trend today; do the anecdotes really illuminate any data, or are they just the same top-of-the-head stuff you could find interviewing people on the street?

The Josephson Institute, a Los Angeles-based ethics institute, surveyed 29,760 students at 100 randomly selected high schools nationwide, both public and private. All students in the selected schools were given the survey in class; anonymity was assured.

Apparently the AP is so impressed by the "large-scale" survey that it doesn't need to ask any more questions (or even do basic stuff like report the confidence interval; does the AP think that's only relevant to political preference polls?). For that, we have to look at the institute's own data, which note that for questions with 20,000 responses, the "accuracy" is 0.7%*** -- a figure "verified by the Department Chair, Decision Sciences & Marketing, Graziadio School of Business & Management, Pepperdine University." (You didn't have to go to the chair for that; a seventh-grader with a calculator could give you the same answer.)

And, under the subhed "Worse than it appears?" the institute provides this gem:

As bad as these numbers are, it appears they understate the level of dishonesty exhibited by America’s youth. More than one in four (26 percent) confessed they lied on at least one or two questions on the survey. Experts agree that dishonesty on surveys usually is an attempt to conceal misconduct.

Uh, no. "Experts" do tend to agree on social desirability bias, meaning that some people will give an answer that's closer to what they think the interviewer wants than to what they really think. But that's a very different sort of "dishonesty" than plagiarism or shoplifting; it's more like saying you're "undecided" even though you've already made up your mind how to vote. It takes a pretty strange attitude to class it as "an attempt to conceal misconduct."

Which brings on another question. A fourth of Our Younguns have "confessed" that they lied on the survey: Do any of you other high school refugees out there have an image of everybody in the row next to you giggling away as they check "10 or more times" for each of the sins? That "lying" in this setting might not be as unidirectional as it tends to be on questions like "did you vote in the election last month?"

What can be done about it? Let's ask the pollster:

Josephson contended that most Americans are too blase about ethical shortcomings.
“Adults are not taking this very seriously,” he said. “The schools are not doing even the most moderate thing. … They don't want to know. There's a pervasive apathy."

Cool. Five assertions of fact about public opinion, none of them having anything to do with what the survey measured. Is there a solution?

Josephson also addressed the argument that today's youth are no less honest than their predecessors.
“In the end, the question is not whether things are worse, but whether they are bad enough to mobilize concern and concerted action,” he said.
“What we need to learn from these survey results is that our moral infrastructure is unsound and in serious need of repair. This is not a time to lament and whine but to take thoughtful, positive actions.”

Don't tell me. Could we start by, oh, taking a Character Development Seminar from the Josephson Institute? Could we even become certified Character Development Specialists? Have those credit cards ready, kids!

Every two years, in short, this guy declares that the world is going to ethics hell in a handbasket (not only is all that shoplifting bad for businesses, but -- think of that next generation of Hummer salespersons!), and newspapers give him acres of free space to bemoan things in. It's pretty much the same story whether things go up or down: in 2002, the cheating rate had "soared" to 71% -- suggesting we needed at least a good plummet (if not an out-and-out "crater") to get down to the 61% (2006) from which we climbed back to 64% this year. But there's a certain sameness about the 2006 press release too:

According to a national survey of high school students by Josephson Institute, today’s young people reveal deeply entrenched habits of dishonesty.

There's no news in these surveys, and the "context" is the depressing need for journalists to stare into their navels and ask why things are so much worse than they used to be, and the only guy who benefits is the one who got the free ad at the top of your front page. It's kind of a pity more people don't have ethics issues with running decontextualized fictions in the guise of news.

* Sorry, but -- having been a high school student, I find it very hard to believe that "all students" in every chosen school completed the survey instrument.
** At least give the Obs desk credit for cutting the tail off the original AP lede, which read: In the past year, 30 percent of U.S. high school students have stolen from a store and 64 percent have cheated on a test, according to a new, large-scale survey suggesting that Americans are too apathetic about ethical standards. Any such suggestions about ethical apathy are entirely the invention of the writer.
*** That's "percentage points," if you've been paying attention.