Scary study of the week
Here's a nice example of how to report on a piece of social science, and if it leaves your faith in the all-conquering power of TRVTH a little shaken ... well, good. The author is Ben Goldacre, and "Bad Science" is his column in the Grauniad. Take it away, Dr. Ben:
Elections are a time for smearing, and the Daily Mail's desperate story about Nick Clegg and the Nazis is my favourite so far. Generally the truth comes out, in time. But how much damage can smears do?
An experiment published this month in the journal Political Behaviour sets out to examine the impact of corrections, and what they found was more disturbing than expected: far from changing people's minds, if you are deeply entrenched in your views, a correction will only reinforce them.
Several things to like so far: He tells you what sort of study it is, provides a link to the study itself (in the 2nd graf; the one in the lede is tweaked to take you more quickly to a live sample of Mail journalism*) and talks about what it found without trying to personalize it. And he's not afraid to call a stunningly dishonest bit of eight-year-old quote-mining a "smear."
Goldacre does a nice job of condensing the methods section without distorting it (another place where most reporting on social science does a poor job). To make a shortened story even shorter, the researchers prepared a simulated AP article (containing a genuine Bush quote) about Iraq's purported WMD programs. Participants (n = 130) were randomly assigned to two conditions: one group got a version mentioning a report that knocked a hole in the whole idea, the other got the story without that mention.
... They had expected the correction would become less effective in more conservative participants, and this was true, up to a point: so for very liberal participants, the correction worked as expected, making them more likely to disagree with the statement that Iraq had WMD when compared with those who were very liberal but received no correction.
For those who described themselves as left of centre, or centrist, the correction had no effect either way. But for people who placed themselves ideologically to the right of centre, the correction wasn't just ineffective, it backfired: conservatives who received a correction telling them that Iraq did not have WMD were more likely to believe that Iraq had WMD than people given no correction. Where you might have expected people to dismiss a correction that was incongruous with their pre-existing view, or regard it as having no credibility, it seems that such information actively reinforced their false beliefs.
One thing it's nice not to see is that tabloid touch of SCIENTISTS SAVE WORLD! This is a nice piece of research that answers a cool question, but it's a small part of the huge puzzle of political information: not just what it looks like and where and how people get it, but what they do with it and how they use it to make sense of the world. This is the broad category of stuff like "reactive devaluation" (people respond differently to the same peace plan if they're told it came from their side, rather than the other side**) and the "hostile media effect" (shown an identical article, State fans and Tech fans will both find it biased against their side). It doesn't answer a big-ticket question: say, does Fox make you stupid or does stupid make you Fox? But it does fill in the picture a bit, and Goldacre does a fine job of demonstrating that it's possible to write sensibly about this stuff for the daily press.
Just for comparison, here's one from the pile of Bad Stuff That Came In During Finals Week:
For CEOs, beauty is a boon, Duke study says
Becoming a corporate CEO is supposed to involve hard work, long hours and business acumen.
It also often requires a solid jaw line and small, piercing eyes, according to a new research study from three finance professors at Duke University.
The hed's flat wrong. The researcher quoted in the story specifically says it didn't find the so-called "beauty premium" as much as a gosh-he-looks-competent premium. And there's nothing about what being a CEO "requires"; the writer just made that up. And does the study hit the newspaper because of the light it sheds on current events? On appearances, it seems more likely to have gotten there because it's about Local Professors. Oh well.
* Is that the same "Tim Shipman" who was making stuff up for the Torygraph from Washington in 2008? Inquiring minds want to know.
** According to his bio, Goldacre (a doctor) works for the NHS, Britain's Kenyan Muslim Communist system of death panels. Surely that doesn'teffectaffect your opinion of his analytic skills.
Elections are a time for smearing, and the Daily Mail's desperate story about Nick Clegg and the Nazis is my favourite so far. Generally the truth comes out, in time. But how much damage can smears do?
An experiment published this month in the journal Political Behaviour sets out to examine the impact of corrections, and what they found was more disturbing than expected: far from changing people's minds, if you are deeply entrenched in your views, a correction will only reinforce them.
Several things to like so far: He tells you what sort of study it is, provides a link to the study itself (in the 2nd graf; the one in the lede is tweaked to take you more quickly to a live sample of Mail journalism*) and talks about what it found without trying to personalize it. And he's not afraid to call a stunningly dishonest bit of eight-year-old quote-mining a "smear."
Goldacre does a nice job of condensing the methods section without distorting it (another place where most reporting on social science does a poor job). To make a shortened story even shorter, the researchers prepared a simulated AP article (containing a genuine Bush quote) about Iraq's purported WMD programs. Participants (n = 130) were randomly assigned to two conditions: one group got a version mentioning a report that knocked a hole in the whole idea, the other got the story without that mention.
... They had expected the correction would become less effective in more conservative participants, and this was true, up to a point: so for very liberal participants, the correction worked as expected, making them more likely to disagree with the statement that Iraq had WMD when compared with those who were very liberal but received no correction.
For those who described themselves as left of centre, or centrist, the correction had no effect either way. But for people who placed themselves ideologically to the right of centre, the correction wasn't just ineffective, it backfired: conservatives who received a correction telling them that Iraq did not have WMD were more likely to believe that Iraq had WMD than people given no correction. Where you might have expected people to dismiss a correction that was incongruous with their pre-existing view, or regard it as having no credibility, it seems that such information actively reinforced their false beliefs.
One thing it's nice not to see is that tabloid touch of SCIENTISTS SAVE WORLD! This is a nice piece of research that answers a cool question, but it's a small part of the huge puzzle of political information: not just what it looks like and where and how people get it, but what they do with it and how they use it to make sense of the world. This is the broad category of stuff like "reactive devaluation" (people respond differently to the same peace plan if they're told it came from their side, rather than the other side**) and the "hostile media effect" (shown an identical article, State fans and Tech fans will both find it biased against their side). It doesn't answer a big-ticket question: say, does Fox make you stupid or does stupid make you Fox? But it does fill in the picture a bit, and Goldacre does a fine job of demonstrating that it's possible to write sensibly about this stuff for the daily press.
Just for comparison, here's one from the pile of Bad Stuff That Came In During Finals Week:
For CEOs, beauty is a boon, Duke study says
Becoming a corporate CEO is supposed to involve hard work, long hours and business acumen.
It also often requires a solid jaw line and small, piercing eyes, according to a new research study from three finance professors at Duke University.
The hed's flat wrong. The researcher quoted in the story specifically says it didn't find the so-called "beauty premium" as much as a gosh-he-looks-competent premium. And there's nothing about what being a CEO "requires"; the writer just made that up. And does the study hit the newspaper because of the light it sheds on current events? On appearances, it seems more likely to have gotten there because it's about Local Professors. Oh well.
* Is that the same "Tim Shipman" who was making stuff up for the Torygraph from Washington in 2008? Inquiring minds want to know.
** According to his bio, Goldacre (a doctor) works for the NHS, Britain's Kenyan Muslim Communist system of death panels. Surely that doesn't
posted by fev at 5:24 PM
2 Comments:
"effect", Fred?
Bother. Never try to be your own copy editor.
Fixed, tnx.
Post a Comment
<< Home