Teen sex, iPods, rock 'n' roll
This looks like a Bad Science entry, but it really isn't. It's about Pretty Good Science and what happens when it collides with Bad Journalism. Most of what follows isn't the copydesk's fault, but some of it is, and plenty of chances for copyeds to step up and save the world appear to have gone begging.
The usual lessons are going to apply: Featurizing is inherently suspect. Stories that get sexier usually get worse. Study Says is a very different animal from AP Says. Inserting new facts in a study is the same thing as "making stuff up." And so on. Exercises are at the end.
Here's the budget line from the Sunday midafternoon AP News Digest, embargoed for Monday ayems. You can see why it got some attention around newsrooms:
SEX LYRICS-TEENS
CHICAGO -- Teens whose iPods are full of music with raunchy sexual lyrics start having sex sooner than those who prefer other songs, a study found. Its influence on behavior appears to depend on how sex is portrayed, the researchers found.
And here's the hed play from a nonrandom sample of front pages at the Newseum site. We'll start with the worst and work through levels of sin toward the best. If your paper didn't front the story, good. If you spiked it or toned it down, even better. If you wrote one of these heds, well ...
The Stupid Science Grand Champion? Step forward the Yakima Herald-Republic and its top-left hed:
Teenagers are jumping to the jive
Sexually explicit lyrics another reason for makin' whoopee
Runner-up, for not bothering to attribute the causal relationship the study doesn't claim to find:
Lyrics incite teens to sex
Wyoming Tribune-Eagle (downpage left)
Second runners-up, for buying the aforesaid causal relationship:
Study: Raunchy lyrics prompt teens to start having sex
Anniston, Ala. (downpage right)
Raunchy lyrics prompt teens to start having sex earlier, new research says
Pensacola (bottom strip)
The judges are particularly impressed by the mug of an iPod that illustrates this effort, since the study has nothing whatsofreakingever to do with iPods.
Raunchy lyrics lead to early sex, study says
Des Moines Register (downpage right)
Raunchy lyrics trigger teens' sexual behavior, study finds
Las Vegas Review-Journal (downpage left)
Raunchy lyrics spur teen sex, study concludes
Akron (downpage strip
The illustration of the teenager with earphones is especially compelling, since they aren't even the iPod the study has nothing to do with.
Faulty causal relationship, but important distinction:
Degrading lyrics lead to early sex, study finds
Pittsburgh Post-Gazette (midpage right)
Missed the distinction but appropriately skeptical about cause and effect:
Study: Teens who listen to explicit lyrics begin sexual behavior earlier
The Tennesseean (downpage left)
Study: Raunchy lyrics influence teenage sex
Modesto, Calif. (downpage left)
Meaningless "may" hed, which at least avoids some damage in this case:
Explicit songs may tempt teens
Study links racy lyrics to sex at young age
Columbus Dispatch (midpage left)
Not content with the AP's featurization, Ohio's Greatest Home Newspaper goes out and finds its own teens. Things get worse from there, and the heds miss almost all the relevant points.
And, at last, a properly skeptical, specific, nondirectional, long-enough-to-get-it-right 1A hed. Take a bow, the St. Louis Post-Dispatch:
Study finds link between degrading sexual lyrics, early sexual behavior
OK. As you might gather, quite a few copydesks made a bad decision much worse by being careless with the hed (and the urge to dress up the page with a silhouetted iPod isn't exactly smiled upon either). But there's more to editing than avoiding harm. An aggressive look at the story finds plenty of reason to hold it or kill it.
Job One in editing a story about a piece of research is to find the study itself (given the AP's penchant for providing links, that shouldn't be too hard). Look first for the abstract, in which trained researchers try to turn a couple months' worth of work into a news brief, and be sure you can find the methods and results sections.
Let's compare the AP's lede with the real thing and see what we get.
CHICAGO (AP) -- Teens whose iPods are full of music with raunchy, sexual lyrics start having sex sooner than those who prefer other songs, a study found.
Hmm. Problem One, as noted above, is the annoying pop-culture reference. The study isn't about iPods or what they're full of. Then the writer takes another leap: Is the study really about "raunchy, sexual lyrics"? No, according to the abstract, it's about "degrading and hostile" sexual content. Is it about what kind of songs the little darlings "prefer"? No, it's about their levels of exposure to different kinds of sexual content. So we've taken three featurization steps in the lede and we've already misstated the boundaries, the purpose and the findings of the study. Not bad for 24 words.
Now, whence the cause-and-effect idea that so many heds fell for? The AP writer doesn't use it, and it's not in the budget line, but it is in the AP suggested hed (reminder: Never use an AP suggested hed until you've verified every assertion in it). It's not in the abstract. Look for a section called "limitations" or "future research" (sentences like "This underscores an important limitation to our study" are a big honking clue). If they aren't there, try "discussion." You'll note that the authors don't claim a causal link either:
Our results suggest that the relationship between exposure and behavior may be causal in nature, because we controlled for teens’ previous sexual experience, as well as factors like parental monitoring, religiosity, and deviance; however, our correlational data do not allow us to make causal inferences with certainty. (Martino et al, 2006, p. 437; emphasis added)
Indeed, one of the things that makes this a good study is that the researchers address and try to control for the other explanations. One is the chicken-and-egg issue (sexual leanings predict musical tastes); another is the they're-both-eggs problem (something else predicts both sexual behavior and musical tastes). They think they have enough evidence to be worried about, but they don't pretend it's something it ain't.
How do a bunch of researchers decide what's "degrading"? Look at the methods section; you get a good idea of why "raunchy" is a poor choice for the lede. "Just one night with you could set
me free. ... I’m dreamin’ day and night of making love." On the study's terms, that's explicit but not degrading; whether it's "raunchy" or not is up to the listener.
How good are the data as a whole? The music variable is based on coding lyrics from albums by 16 artists (for a total of 193 songs), then asking respondents how much they listen to those artists. All the data about sexual behavior and the covariates -- the other things (socioeconomic status, race, sex, previous sexual experience, performance in school, risk-taking and the like) brought into the equation as controls -- are self-reported in telephone interviews. Is that perfect? Well, research is about trade-offs. Get used to it.
End of sermon, and if you're saying "about time," you're probably right. But there is a moral. Lots of the routines that go into making "good stories" -- reaching for the pop-culture tie-in, making a relationship sound stronger than it is, asking Real People to react to a reporter's misperception of the research at hand -- end up making for very bad journalism. If we really think the reading public is too dumb to understand what scientists do, maybe we're underestimating the skills of the reading public. And, worse, overestimating our own virtues.
Exercises
1) See how long it takes you to find a copy of the study as published (Martino, S.C., Collins, R.L., Elliott, M.N., Strachman, A., Kanouse, D., & Berrry, S. H. (2006). Exposure to degrading versus nondegrading music lyrics and sexual behavior among youth. Pediatrics, 118, 430-441.)
2) How would you try to improve on the definition of "degrading"?
3) Find the "interrater reliability" scores for how "degrading" was measured. What's the consensus among social science researchers on how good those results are?
4) Here's a lyric that's mentioned in the study but not categorized. Would you code it as degrading or nondegrading? Compare your decision with those of some friends.
When it comes to sex don’t test my skills/Cause my head game will have you head over heels.
The usual lessons are going to apply: Featurizing is inherently suspect. Stories that get sexier usually get worse. Study Says is a very different animal from AP Says. Inserting new facts in a study is the same thing as "making stuff up." And so on. Exercises are at the end.
Here's the budget line from the Sunday midafternoon AP News Digest, embargoed for Monday ayems. You can see why it got some attention around newsrooms:
SEX LYRICS-TEENS
CHICAGO -- Teens whose iPods are full of music with raunchy sexual lyrics start having sex sooner than those who prefer other songs, a study found. Its influence on behavior appears to depend on how sex is portrayed, the researchers found.
And here's the hed play from a nonrandom sample of front pages at the Newseum site. We'll start with the worst and work through levels of sin toward the best. If your paper didn't front the story, good. If you spiked it or toned it down, even better. If you wrote one of these heds, well ...
The Stupid Science Grand Champion? Step forward the Yakima Herald-Republic and its top-left hed:
Teenagers are jumping to the jive
Sexually explicit lyrics another reason for makin' whoopee
Runner-up, for not bothering to attribute the causal relationship the study doesn't claim to find:
Lyrics incite teens to sex
Wyoming Tribune-Eagle (downpage left)
Second runners-up, for buying the aforesaid causal relationship:
Study: Raunchy lyrics prompt teens to start having sex
Anniston, Ala. (downpage right)
Raunchy lyrics prompt teens to start having sex earlier, new research says
Pensacola (bottom strip)
The judges are particularly impressed by the mug of an iPod that illustrates this effort, since the study has nothing whatsofreakingever to do with iPods.
Raunchy lyrics lead to early sex, study says
Des Moines Register (downpage right)
Raunchy lyrics trigger teens' sexual behavior, study finds
Las Vegas Review-Journal (downpage left)
Raunchy lyrics spur teen sex, study concludes
Akron (downpage strip
The illustration of the teenager with earphones is especially compelling, since they aren't even the iPod the study has nothing to do with.
Faulty causal relationship, but important distinction:
Degrading lyrics lead to early sex, study finds
Pittsburgh Post-Gazette (midpage right)
Missed the distinction but appropriately skeptical about cause and effect:
Study: Teens who listen to explicit lyrics begin sexual behavior earlier
The Tennesseean (downpage left)
Study: Raunchy lyrics influence teenage sex
Modesto, Calif. (downpage left)
Meaningless "may" hed, which at least avoids some damage in this case:
Explicit songs may tempt teens
Study links racy lyrics to sex at young age
Columbus Dispatch (midpage left)
Not content with the AP's featurization, Ohio's Greatest Home Newspaper goes out and finds its own teens. Things get worse from there, and the heds miss almost all the relevant points.
And, at last, a properly skeptical, specific, nondirectional, long-enough-to-get-it-right 1A hed. Take a bow, the St. Louis Post-Dispatch:
Study finds link between degrading sexual lyrics, early sexual behavior
OK. As you might gather, quite a few copydesks made a bad decision much worse by being careless with the hed (and the urge to dress up the page with a silhouetted iPod isn't exactly smiled upon either). But there's more to editing than avoiding harm. An aggressive look at the story finds plenty of reason to hold it or kill it.
Job One in editing a story about a piece of research is to find the study itself (given the AP's penchant for providing links, that shouldn't be too hard). Look first for the abstract, in which trained researchers try to turn a couple months' worth of work into a news brief, and be sure you can find the methods and results sections.
Let's compare the AP's lede with the real thing and see what we get.
CHICAGO (AP) -- Teens whose iPods are full of music with raunchy, sexual lyrics start having sex sooner than those who prefer other songs, a study found.
Hmm. Problem One, as noted above, is the annoying pop-culture reference. The study isn't about iPods or what they're full of. Then the writer takes another leap: Is the study really about "raunchy, sexual lyrics"? No, according to the abstract, it's about "degrading and hostile" sexual content. Is it about what kind of songs the little darlings "prefer"? No, it's about their levels of exposure to different kinds of sexual content. So we've taken three featurization steps in the lede and we've already misstated the boundaries, the purpose and the findings of the study. Not bad for 24 words.
Now, whence the cause-and-effect idea that so many heds fell for? The AP writer doesn't use it, and it's not in the budget line, but it is in the AP suggested hed (reminder: Never use an AP suggested hed until you've verified every assertion in it). It's not in the abstract. Look for a section called "limitations" or "future research" (sentences like "This underscores an important limitation to our study" are a big honking clue). If they aren't there, try "discussion." You'll note that the authors don't claim a causal link either:
Our results suggest that the relationship between exposure and behavior may be causal in nature, because we controlled for teens’ previous sexual experience, as well as factors like parental monitoring, religiosity, and deviance; however, our correlational data do not allow us to make causal inferences with certainty. (Martino et al, 2006, p. 437; emphasis added)
Indeed, one of the things that makes this a good study is that the researchers address and try to control for the other explanations. One is the chicken-and-egg issue (sexual leanings predict musical tastes); another is the they're-both-eggs problem (something else predicts both sexual behavior and musical tastes). They think they have enough evidence to be worried about, but they don't pretend it's something it ain't.
How do a bunch of researchers decide what's "degrading"? Look at the methods section; you get a good idea of why "raunchy" is a poor choice for the lede. "Just one night with you could set
me free. ... I’m dreamin’ day and night of making love." On the study's terms, that's explicit but not degrading; whether it's "raunchy" or not is up to the listener.
How good are the data as a whole? The music variable is based on coding lyrics from albums by 16 artists (for a total of 193 songs), then asking respondents how much they listen to those artists. All the data about sexual behavior and the covariates -- the other things (socioeconomic status, race, sex, previous sexual experience, performance in school, risk-taking and the like) brought into the equation as controls -- are self-reported in telephone interviews. Is that perfect? Well, research is about trade-offs. Get used to it.
End of sermon, and if you're saying "about time," you're probably right. But there is a moral. Lots of the routines that go into making "good stories" -- reaching for the pop-culture tie-in, making a relationship sound stronger than it is, asking Real People to react to a reporter's misperception of the research at hand -- end up making for very bad journalism. If we really think the reading public is too dumb to understand what scientists do, maybe we're underestimating the skills of the reading public. And, worse, overestimating our own virtues.
Exercises
1) See how long it takes you to find a copy of the study as published (Martino, S.C., Collins, R.L., Elliott, M.N., Strachman, A., Kanouse, D., & Berrry, S. H. (2006). Exposure to degrading versus nondegrading music lyrics and sexual behavior among youth. Pediatrics, 118, 430-441.)
2) How would you try to improve on the definition of "degrading"?
3) Find the "interrater reliability" scores for how "degrading" was measured. What's the consensus among social science researchers on how good those results are?
4) Here's a lyric that's mentioned in the study but not categorized. Would you code it as degrading or nondegrading? Compare your decision with those of some friends.
When it comes to sex don’t test my skills/Cause my head game will have you head over heels.
posted by fev at 8:19 PM
1 Comments:
I'm glad the Post-Dispatch wrote a favorable head, but I did hear some people questioning why we put it on the front page in the first place — the first question I had when I saw it in the morning, followed by the "why does it talk about iPods?" query.
Post a Comment
<< Home