Adventures in agenda-setting: The science beat

Of all the accounts of peer-reviewed health research in all the gin joints in all the world, why is this the one you saw Tuesday if you read a certain major Southeastern daily?

Alcohol's helpful? No proof, scientists say
Several reasons -- well, one primary reason and a bunch of interrelated secondary reasons. Taken together, they all help shed some light on how and why "science" looks the way it does in the popular press.

The secondary reasons are familiar from other critiques of the press: conflict's unwarranted primacy as a news value, the Maverick Against the System syndrome, the assumption that all "-ologies" are equal, the inability to distinguish proof from probability, a fondness for simplifying good ideas into logical fallacies, that sort of thing. But first to the big reason. Why is this in the paper?

Have a look at Tuesday's New York Times front and you'll find a big old clue. Unless things have changed a lot, sometime around 8 p.m. Monday, papers that subscribe to the NYT news service got the frontpage budget over the wire, and the alcohol story would have been listed among the frontpage reefers. That makes it just about the easiest decision in American journalism. A zombie could walk into a 1A conference dripping with fresh brains and the rot of the grave, and if the first thing it said before ripping the photo editor's arm off was "Let's go with the alcohol story; the Times is fronting it," the strongest response at the AME level would be "Can we get a graphic?"

That doesn't mean the Times's decisions are always good. It means that -- science, Fractious Near East, politics, whatever -- it's easier to point to the authority of the wire than to try to build an argument based on independent judgment. This is understood up and down the food chain. Two decades ago, the word-of-mouth guideline at the fishwrap in question was that if a story was on the front of the Times, the publisher* expected it to be somewhere in the paper, and most NYTNS subscribers probably have similar tales.**

OK. So we've established that, if the AP says "Cancer cured" and the NYT says "Put down that glass of red wine," you're more likely to read about wine. That doesn't make "is wine good for you?" a bad story -- but the one you read in the Obs (or the Times) was a bad story, so it's worth looking at some of those secondary reasons to see why.

Let's start with the hed: "Alcohol's helpful? No proof, scientists say." Dreadful on all counts. First, the story isn't about "helpful" (if you're planning to get drunk, alcohol isn't just helpful, it's damn near indispensible). Second, sun rises in East: Of course "scientists say" there's no proof, because "scientists" wouldn't have said there was. To quote a cherished preceptor:*** "If you want proof, go to seminary." We're in the probability business. Research proposes and tests hypotheses, and it finds or doesn't find support for them, but it doesn't "prove" stuff.

That's not just quibbling. The classic example is the tobacco industry, which argued for many years that no one had "proved" that cigarettes kill people -- quite correctly, in that designing a proper study that would allow genuine causal inferences about smoking would require randomly assigning people to smoke, which is sort of Tuskegee-level immoral. When that avenue isn't available, you defer to looking at boatloads of correlational data. Nothing inherently wrong with that, unless you want to go back to reminding your readers that Tareyton is better and charcoal is why, or that global warming is a fraud.

The Times's hed isn't a lot better (the Obs did a bad job of editing the story, but it was condensing a bad story, not turning a good story into a pile of garbage): "Alcohol’s Good for You? Some Scientists Doubt It." Welcome to the land of logical fallacies. All scientists "doubt it," because none would say in advance that alcohol is "good for you"; something that correlates with longer life in the average overweight 45-year-old white dude might lead to a much worse outcome in "you," for any of a number of reasons. The science bit lies in telling you which "you" seems to be associated with which benefits.

You can pick up on a bit of that if you read the entire story (that's the 30 NYT-size paragraphs you find at the Times, as opposed to the 10 Obs-sized grafs you find at the Obs). There is, for example, this:

“People who would not be able to stop at one to two drinks a day shouldn’t drink, and people with liver disease shouldn’t drink,” Dr. Klatsky said. On the other hand, “the man in his 50s or 60s who has a heart attack and decides to go clean and gives up his glass of wine at night — that person is better off being a moderate drinker.”

Do you get the idea that what "experts" say is that limited drinking is probably a good idea for some people and very much not so for others? Good. With a more even-handed (I'm trying really hard not to say "more honest," and up until now I was doing rather well) editing approach at the Observer, you might have gotten the same fairly accurate impression of the general state of scientific play.

That points to a systematic failing of news routines: The standard response to "Gee, this sounds complicated, nuanced and inconclusive" isn't "Great! That's right!" but "Oh, hell -- we'd better make it sexy, exciting and conclusive, then." Which is what happens when the Observer selects all its quotes from one side of the story. It's suggesting not just that this apparent conflict is new, but that it's the central element of the issue, and even worse that it's a conflict between "hey, have another" and "STAY AWAY from that stuff," rather than a (fairly boring but legitimate) dispute about whether the categories being counted will yield the sort of results that people on either side want to draw appropriate inferences from.

Just as not all disagreements are equal, not all starting points are equal. Sociology and epidemiology aren't the same craft, and they don't necessarily argue about the same stuff.**** That's fine. Apples aren't oranges, with all due respect to the source that the Obs selectively quoted, but good apples and good oranges are both cases of good fruit. When we turn everything into a fight, we lose sight of the sensible things that apples and oranges have to say to each other.

And in this case, the final answer is something like: Nothing very dramatic is going on here. "Science" doesn't know anything today that it didn't know yesterday (the Observer might have cheated to make the story sexier, but the NYT story was fundamentally misleading to begin with). The maverick dissenter isn't always better or smarter or righter than the annoying spokespersons of the status quo. The Times probably had something better to do than turn this bunch of random speculation into news; the Observer could have exercised its own independent judgment, rather than deferring to the NYT. It's quite all right to be annoyed or angered at this display of carelessness. But at this point, no one has any grounds to be surprised.

* Rolfe Neill, one of the genuine good guys to ever hold that title. Also one of the few mentioned kindly in a journalistic memoir (for having taught Charles Kuralt how to draw a page, if I recall correctly).
** Raleigh in the Sitton era seemed especially zombified by the Times (Sitton was an old Times hand), but that vintage of N&O was good enough to justify a little heuristic cluelessness.
*** And Evil Dissertation Methodologist, if he's listening.
**** Language Czarina's dissertation actually has "science" in the title. Mine has a repeated-measures analysis of variance in the middle somewhere. If you think you can tell which one is better, you have to buy the wine.