How to kill a rogue story

Wondering about that funny-looking chunk of election-oriented social science that just landed in your basket for tomorrow's paper? Here are some quick tests to help you tell whether it's ready for the front page or the big sleep. And there's a real news story to practice on!

[That's actually the serious part, because this story has been floating around the McClatchy Web site since yesterday, suggesting it's probably headed for print at some shops that use the McClatchy service. It really, really needs to be killed, so if you see it headed for print ...]

Here's the story:

Small study suggests McCain ads lampooning Obama hurt

(Several assertions in the hed are either questionable or flatly wrong, but let's move on to the text first)

John McCain struck again on Friday, releasing a Web video suggesting that his Democatic rival, Barack Obama is "The One," a semi-religious figure sent to save the world. The spot includes footage of Charlton Heston as Moses, parting the Red Sea.

The ad was the second released this week by McCain intended to make fun of Obama. Earlier, the campaign issued an ad that likened Obama to Britney Spears and Paris Hilton in an effort to take the shine off the huge crowds Obama drew in Berlin during his European tour.

Friday's ad takes that theme one step further, lampooning Obama's soaring rhetoric and suggesting that the Illinois senator suffers from a Messianic complex.

Wondering when we'll get to the study? That's a good question. First, though, you might want to point out a contradiction between the lede and the second graf. What the McCain camp has done is to release a video. It's not an "ad" until someone buys time or space* for it (which is why we say stuff like "tell him to buy an ad" when a candidate tries to pitch a story about the other candidate's messiah complex). "Ads" also run the risk of disapproval by the outlet's ad-standards department. You can make fun of McCain's Intertube skills all you want, but he's getting some pretty good viral-campaign mileage out of this.

... A small study of people's reactions to the Britney-Paris ad suggested, however, that while people don't like the ad, it caused them to doubt Obama, and small percentages who'd said before viewing the ad that they'd vote for him said afterword that they wouldn't.

OK. Now we're out of "normative" territory (is it appropriate to write a story that, in effect, replicates a dishonest ad?) and into nuts-and-bolts land. If this is a "study," there's a set of questions we need to know the answers to before it can run. First, what kind of "study" is it? Masscomm research breaks down broadly into two kinds:

1) Studies that count stuff, test the resulting numbers and draw inferences, and
2) Everything else

"Everything else" is a huge range of of domains and methods: history, law, Saidian critical discourse analysis, focus-group discussions of pizza ads, and more. You shouldn't sell it short, but today we're talking about the first kind of study.

That settled, we need to ask what kind of quantitative study it is, because methods aren't interchangeable. Content analysis can tell you that the War on Terror® looks different on Fox than on the BBC, but it can't tell you what effect that difference has. Surveys can tell you what people say, but not what sort of content makes them say it. The story isn't complete if it doesn't tell you what "study" means. And when that's settled comes the fun stuff: What did they measure, how did they measure it, and what do the results look like?

Those declines didn't result in more support for McCain; doubting Democrats and Republicans instead moved into the undecided column. Independents who moved away from Obama did say they'd vote for McCain.

The study, of 320 Americans, found that a majority of Republicans were "disturbed, skeptical" and "saddened" after viewing the ad and that 61 percent of Republicans had a negative view of the ad.

...While viewing the ad, participants indicated their levels of agreement by moving their computer mouse from left on a continuum. The responses were recorded in quarter-second intervals and reported in the form of curves. Participants were also asked pre- and post-viewing questions.

McClatchy hasn't bothered to say, but at this point you can figure out that the "study" is an experiment, not a survey. That puts the hed in a different light: 320 would be small for a survey,** but it's really big for an experiment. (And it's a study of only one ad, so the hed's wrong on that count too; we can't talk about what the "ads," plural, are doing, because we aren't measuring it.)

Is it a good experiment? Yet another set of questions. For starters, participants aren't randomly assigned to conditions. That's not a deal-breaker (after all, you can't randomly assign people to smoke or be pregnant), but it puts us in the category of "quasi-experiment. It's a single-shot, pretest-posttest design with no control group. That means any conclusions about the effect raise an immediate question: Compared to what?

If the ad does anything, we don't know how it compares to the effect of no ad at all.*** We don't know whether McCain ads have more impact than Obama ads, or whether an "acclaim" (pro-McCain) ad has more impact than an "attack" (anti-Obama) ad. With only one stimulus, we have no idea what element of the ad -- visual, voice, music, content -- might be having the effect. Which puts this ominous paragraph in a whole different light:

But the results that may have been most telling were the changes in whom the participants would vote for and suggested that such advertising could have an impact, especially among independents.

If McClatchy thinks this result is the "most telling," why isn't it the one that the researchers emphasize? Why do the researchers note, to the contrary, that "the ad did not move voters"? That gets to what's measured and how, so let's try to tease some numbers out of the story and the original report and see what we can do with Excel and that nice VassarStats link to the right.

The researchers probably didn't mention this ominous sign of the Power of Evil Ads because it's irrelevant. Put the changes for "who would you vote for today?" in a chi-square and the P value comes out to about .89. That represents less than one chance in eight that there's any difference related to watching the ad. We don't know where the almost imperceptible change -- for the record, three original Obama voters went to "other" and three to "undecided" -- came from. We do know that whatever is happening on that question (not "doubt," which seems to have been made up by the reporters****) is almost certainly not a result of the experimental treatment.

That doesn't mean there aren't significant results of the experiment. There are. If you treat "very favorable," "somewhat favorable" and the like as nominal data, the ad has no effect on opinions about Obama but a significant negative effect on opinions about McCain. If you squint a bit and assume that the intervals from 1 (very negative) to 4 (very positive) are equal, the ad makes Republicans significantly more positive about McCain and Democrats and Independents significantly more negative. The mean differences are small (with an N of 320, you don't need much to reach significance, but that's another issue), but they almost certainly didn't come about by chance.*****

Whether the "if the election was today" question is a better predictor of voting behavior three months in the future than the "your overall opinion" question is a matter for debate. The results aren't. To the extent we can say anything at all about its impact, the ad affects opinion but not intent. And if the ads "hurt" anyone, they "hurt" McCain.

If you've been following the playbook -- what sort of study, what was measured, what do the numbers say -- the conclusion ought to be pretty clear. Kill the story. Right now. Or ask McClatchy to provide a version that accurately reflects what the study found, rather than what the reporters speculate.

* Unless MCT is donating the space as a public service, like an antismoking campaign, and it really doesn't want to go there.
** MCT has certainly been happy to draw inferences from smaller subsamples in the past, though.
*** With 320 people, you could have 160 watch a 2-minute clip from "The Daily Show" (80 with ad, 80 with no ad) and 160 watch a 2-minute clip from "The O'Reilly Factor" (80 with, 80 without). Now you have a 2 (show) x 2 (ad) x 3 (affiliation) design, and that's going to start being fun in a hurry.
**** As does the bit about whether participants "dislike" the ad. If it wasn't measured, you can't say it was. Period.
***** You can, and should, try this at home. To see whether the average "before" response differs from the average "after" response:
1) Find the original data and convert the percentages back to raw numbers
2) Create an Excel sheet with three columns: party (1, 2, and 3, just to make things easier), before and after. The first 104 cases are the Democrats (1 in the "party" column), the next 108 are the Republicans (2) and the next 108 are the Indys (3)
3) In "before" and "after," enter the number of responses that correspond to each level of the variable. Among Democrats, four have a "very favorable" opinion of McCain before seeing the ad, so the first four rows in "before" get a 4.There are 21 "mostly favorable," so the next 21 rows get a 3. In "after," the first 5 rows get a 4 (for "very favorable"), the next 18 get a 3, and so on.
4) Run a T-test (under "data analysis") on the second and third columns, then select just the Democrat, Republican and Independent conditions. The test will tell you what the averages are for before and after and confidence level of the test statistic -- whether the difference is statistically significant.

Open mike for interns, 2008 edition

A note in yesterday's comments section spotlights one of the great dilemmas of this odd little craft we call editing, so it's time to bring an old tradition* out of mothballs. A few quick comments, and then -- I hope -- it's over to the interns.

Here's the comment from Luke**: "It's a struggle finding where what you've been taught was ideal in J-school applies and when exceptions are OK." He's quite right, and it may not be much consolation to know that the boundary can remain fluid long after the J-school and internship stage. But the matter does touch on a lot of issues that keep on coming up here, and interns are often in an especially ill-defined position when it comes to negotiating those issues.

There's the problem, for starters, of exactly how grammatical you want to get. One of the fun secrets of the newsroom is that the people who talk the loudest about "the language" often don't know very much about it (if you frequent the peevology section, you might have noticed that journalists -- and, to be fair, J-profs -- aren't alone in this regard). If someone tells you to stop sending over passive heds, you might need a decoder ring to know whether "passive" means "passive" or just "boring."

In many ways, you and your slot editor are navigating by throwing rocks in the dark and waiting to hear if any glass breaks. You're trying to figure out what the slot means by "grammar" and "rules," and the slot's wondering whether you let a split auxiliary go through because you exercised solid professional judgment or because you've never heard of the one "rule" the publisher remembers from college. Who's going to ask first? That's tricky even in a relationship where power is about even.

(The faculty*** is in a bit of a bind too. There's a lot of stuff in the secret-handshake category that really shouldn't be taught anymore -- certainly not as "grammar." But practically, we can't stop teaching the "over/more than" thing until editors stop testing for it. I expect lots of people on both sides will be happy to attend the opening ceremony of the disarmament talks, but until then, it'd be a brave journalism school that decided to teach the AP Stylebook as an important cultural artifact with nothing whatsoever worth reading about grammar and usage.)

Figuring out the grammar part is only the beginning. At some point in any good journalism program, students are introduced to the Forbidden Ledes: "It's official," "Christmas came early for ..." and all the other ones that should never stain a dead pine tree. The list tends to leave out a mundane but highly pertinent observation. Writers use the Forbidden Ledes. A lot. What's going to happen the first time you kill a "Christmas came early" on the local front centerpiece?

Did somebody say all journalism students ought to take at least one stats course? Well, they should. Does that mean someone's going to listen when you suggest that Star Reporter's keen political analysis of the latest poll is entirely unsupported by the data? You may be right, but don't expect to win.

So with that cheery screed aside, I hope this summer's journalism interns will take a moment to offer some advice. Two specific questions are posed:

1) What's the biggest surprise of your summer in journalism?
2) What's the one thing we should have taught you and didn't?

If you're an intern, please hit the button and comment. If you work in a shop with some interns, let 'em know they're invited. If you have some comments from your recent experience on either side**** of the divide, you can play too.

I hope to hear from you.

* Dating back to the turn of the millennium, when HEADSUP-L was an actual Listserv with a mere few dozen subscribers
** More visitors from KU would be welcome, by the way.
*** A whole bunch of us are going to get together and talk about you guys next week, by the way. Just so's you know.
**** I know. See above about the AP Stylebook.

Active aggressive

I'm getting really tired of the active voice. OK, not exactly. The active voice is easily one of my two favorite verb voices. What I'm getting tired of is every time a bit of information asks for a straight-bladed screwdriver, some copy editor insists on fetching a Phillips head.

That's the problem with today's hed.* It comes with a pretty unforgiving count. (1.5/42/2? Or is it a 36?) If you want to pack some meaning in, you'll need to work carefully, and this looks like a first-thing-that-fit sort of hed. The easiest way to make it better is to ignore your J2100 textbook** and ask for the appropriate item from the part of the grammar toolbox called "verb voice."

Count it up. The first line -- half the hed -- is gone by the time you're finished getting the subject out. The verb is another fourth of it. So you have the last quarter of the hed to spend on the most important noun, which is the object. The passive voice is the tool that lets you turn the hed around and emphasize the object.

Indictment heds are classics of the cop-and-court variety because there's almost never any need to emphasize the subject. The Woodward Dream Cruise doesn't issue indictments. The AARP doesn't issue indictments. That's what grand juries do. So the meaning of the first 75% of the hed is pretty much contained in the verb. That's space we can spend on who was indicted, which (trust me, I've been reading these things for a long time) is almost invariably more interesting than who did the indicting.

What makes up a senator's identity? Name, party and state almost always show up, though sometimes a career characteristic gets pride of place. ("Senator No" can be a clearer identifier than "R-N.C."). Almost any of them would make the hed better by their mere appearance. Let's see how they fit:

Grand jury
indicts senator

Alaska Sen.
Stevens indicted

Seven-term GOP
senator indicted

Another GOP
solon accused
(sorry, had to get "solon" in)

King of Senate
pork indicted
(if the Freep can declare him a "lion," I can make him a "king")

And British-style active, for the fun of it (kids, don't try this at home):
Alaska senator
'lied about gifts'

If the passive voice is so good (and good for you), why does the Strunk & White book call for preferring the active? It's a freshman comp book! It's a guide for people who need to write adequately, even if they never need (or learn, or want) to write well. And rules like "use the active voice" are a good way to write adequately.***

None of this is to say that the passive is "better." Verb voice is a tool that helps put meaning into place. The better one is the one that does that well. In case you're wondering, here's a hed that misuses the passive. Look in the relative clause:

Birth certificate of child linked to Edwards lists no father

"Linked" by whom? If you don't have the space to say "linked by the National Enquirer," you need to say hello to the active voice. As you should consider doing whenever you venture into the territory of election-year mudslinging.

Are we telling you anything you don't know? Hed writing isn't a walk in the park. You need all the stuff in the toolbox. Don't be scared away from the passive voice by some half-remembered thunder from J-school.


* OK, "yesterday's hed." Busy few weeks at the Manor.
** You should be making a habit of this anyway.
*** S&W also gives a nice example of how the passive voice can serve the writer's intent. Read it before you slag it.

Ministry of Truth

Just in from the Fair-n-Balanced Network, and too good to pass up:

WASHINGTON — A federal judge ruled Thursday that Bush administration advisers are not immune to congressional subpoenas.

The decision gives Democrats on Capitol Hill a major victory in their attempts to hamstring the Bush administration through a number of ongoing investigations.

Those darn Democrats on Capitol Hill! Always skulking around Rock Creek Park and waiting for Bin Laden's couriers to drop off their weekly shipment of cocaine and counterfeit $100 bills attempting to hamstring the Bush administration!

Sort of makes one wonder what coverage of the Watergate hearings might have looked like on Fox. But then again, we already have some hints about that.

Fear factor Foxtacular

It's a good day to keep the kids indoors, judging by today's news menu at the Fair-n-Valenced Network.
In addition to the kids-in-peril tales shown at right, a few more have been added:

Three kids shot in back seat of mom's Mercedes

Sitter charged with having sex with teen in her care

Amber Alert issued for 4 children in Massachusetts

Scared yet? Fox apparently hopes so.

Gaily indight, a gallant knight

Sometimes it's fun to just sit back and watch Fox play at copyediting, isn't it?

Credit where it's due. Fox put its own name on the Stevens story, with no less than three staffers contributing, and the story was getting better play earlier in the afternoon (what you see above is the top "latest news" item). That's notably more staff input than the average Missing Mom story (to be fair, Washington is Washington, and the Southeast general mayhem beat reporter is tied up elsewhere). If you're interested in the shenanigans of Monica Goodling, Kyle Sampson and their playmates at the Justice Department, though, you're going to have to work for it.

There's a word for that, you know

It's official: Christmas came early. Call it pain relief at the pump.

How else to explain the pinch-us-we're-dreaming tone that surrounds this minor statistical occurrence?*

It's really true -- gas dips below $4 a gallon
("Really true" distinguishing it from all those heds that are merely "true")

The signs don't lie: Charlotte-area gas prices really have dropped below $4 a gallon.
(That's good to hear. Because when the signs do lie, we have a word for that. It's called "fraud.")

Declining oil prices in the last two weeks – from a record-high of more than $145 a barrel to less than $124 Friday – have meant a little less pain at the pump.
(Don't hyphenate adj-noun combinations like "record high." Unless you're trying to take your readers' minds off "pain at the pump.")

Smart Shopperette of Charlotte filled up her white Mitsubishi Galant – whose license plate read LUVISLUV** – Saturday afternoon at the Quik Shoppe at East and South boulevards, where regular unleaded was $3.95 a gallon.

Ah, the beauty of news routines. A story like this isn't really official until a Real Person joins in -- i.e., until somebody buying gasoline is prodded into acknowledging that yes, it's usually better to pay less money for the same amount of stuff. And the white Galant is there -- well, for the same reason we always ask what kind of dog it was. More facts make a story truer.

“I drove over here to get this gas,” which was cheaper than stations near her home off Wilkinson Boulevard, Shopperette said, adding that saving only a few cents “makes a big difference.”

You know what might be more useful than playing Names Make News at the local gas station? Figuring out the point at which her statement becomes true. Let's keep it simple: 5 cents a gallon, 20 gallons a tankful -- five cents saves you a dollar a tank. At a ballpark figure of $4 a gallon, if you burn a quarter of a gallon getting to and from the 5-cent discount, it's a wash.

Isn't that fun? You wouldn't get to leave the office, but then again, you wouldn't have to ask people dumb questions like "Would you rather pay less money or more money for gasoline?"

* Copy editors: For real fun, next time you get a story about the "average price" of a gallon of gasoline, ask the originating desk for the standard deviation. Then ask for the sample from which the mean is calculated and whether the change from last week is significant. They'll love you!
** Forgive me for being unimpressed. Now if it had read LUVISNOTLUV THATALTERSWHENITALTERATIONFINDS, that'd be worth putting in the story.

If they didn't, they should have

Today's lesson in the Virtues of Not Making Stuff Up comes to you courtesy of -- well, just about everybody. Here's a lede from today's local guardian of the public weal:

Prosecutors, defense attorneys and an eager public will have to wait again -- this time until at least next month -- for Judge Ronald Giles to rule whether any of 200 text messages from Mayor Kwame Kilpatrick and his former aide Christine Beatty will be revealed.

Got it? OK. What does that lede have in common with this hed from America's Fair'n'Balanced Network?

Italians Shocked After Sunbathers Relax on Beach Near Dead Bodies

Anybody ready to take a try? Exactly! Neither one is, well, you know, exactly true -- but if they aren't, they ought to be. The Freep has no evidence -- meaning "none in the story," which is a pretty good stand-in here for "none, period" -- that there's an "eager public" waiting to read stuff they've been reading since ... gee, was it really January? Fully six months before Jon Stewart put the Residence Inn in Madison Heights on the National Love Map?

OK, I'm feeling like I want another night like the most recent Saturday at the Residence Inn!


Edge of my seat, I'm telling you.

And Fox? Almost the same. The story has some bodies and some sunbathers, but there's no way to tell if any Italians were shocked in the making of this story, because nobody bothers to ask any of them. But like the Freep's eager public, it tends to slide right past the slot, because -- well, no doubt some of them are shocked, or would be if we asked them, but mainly because shocked is what Real People should be at the very thought of such a thing.

Things like "Critics blast Obama's West Bank trip," noted here earlier, get more attention because of the blatant ideological thumb on the scales. But they're not that different in substance. Fox doesn't have a critic to present, but there has to be one out there; the Freep doesn't have an "eager public," but if the ungrateful swine aren't eager to hear about the text messages, they should be.

Welcome to the Wonderful World of News, where everything we tell you is true -- or, if not, is the way things ought to be.