Spotting the bogus story
Yet another in the Occasional Series series, this time devoted to how the busy copy editor can spot a bogus story and call in enough firepower to stop it before it becomes an embarrassment to the paper.
To repeat a previous theme: Since a newspaper copy desk doesn't have folks detailed to look up every single assertion that crosses its screens, editors need to rely on well-informed instincts and a checklist of practiced suspicions. Here's a look at how you can apply those habits to a real-life story -- and, conveniently, to any other story of its type. Because (here's the punch line) you will see another story that fibs with numbers. Maybe not today and maybe not tomorrow, but soon and for the rest of your career.
Our candidate is "Missouri is No. 1 in wasted time" (1A Tuesday). It's easy to see from the second and third grafs why it made the cut in budget meeting: Workers waste time on the job! A lot! It costs business billions! And Missourians are the worst fork-offs of all! Your job as a news cop is to follow this one long enough to see if it starts weaving. It does in the second graf, in that sentence ending with "... according to a new Web survey by America Online and Salary.com."
"Web survey" is a polite way of saying "b*lls**t." Whenever you see it, assume the content of the story to be false until you see a graf explaining the methodology. There are some rigid ways of collecting good data on the Net, as there are ways of collecting good data on the phone. Any story that's done the work will explain it. Any story that doesn't is a fake until proven otherwise.
In this case, the story doesn't, but if you poke around a bit at the originating sites, you get an idea of how the data were collected: Visitors to an AOL job site and Salary.com could fill out the survey if they found it. And that means the third graf -- "The No. 1 state for wasting time was Missouri, where workers who responded to the survey resported slacking off 3 hours and 12 minutes a day" -- is crap by definition. The numbers themselves might be true, but we have no reason whatever to believe them. Any numbers pulled out of the air have an equal chance of being true.
"Visitors to AOL's job-search site can click on the survey" is, in short, the same thing as going to the mall and asking the first 10 non-scary people you meet. It's a convenience sample, and you can't draw conclusions about the population from it. It doesn't make any difference that 10,000 people allegedly responded. No generalization from that sample or any part of it -- the national average, the Missouri average, anything -- is worth reporting, because there's no reason to believe it's true.
End of screed, technically. The whole premise of the story is false. We have no way of knowing how much time workers waste anywhere. Any sub-conclusions are false too ("fruit of the poisoned tree," it's called). But many of the other dishonest things the story does might show up in other stories, so they're worth mention.
Here's one: Is Internet access equally distributed in the target population? Not if we're talking about "the average worker." It's lower in black and Latino populations, and it's lower in lower-socioeconomic-status populations. So even a random sample of the Internet population wouldn't allow you to say anything about the population as a whole. (Rule: You can only generalize from a random sample, and you can only generalize to the population you sampled. No exceptions.)
Here's another: What's the most common form of wasting time at work? Personal Internet use! So any group that doesn't have Internet access at work wouldn't count either. (You're getting the idea? There's more to the working world than upper-middle-class white yuppies?)
And that leads to another, which I'll admit I came to because Tuesday is trash pickup day at HEADSUP-L Manor, and I was tempted to run this one out and show it to the guys on the garbage truck, because I bet they weren't surfing over to ThighNoon.Com between houses. Lots of folks out there put in a pretty full workday, and some of them put in more: What in this "survey" accounts for time past 8 hours? Where does the time Wal-Mart steals from the hourly worker figure in?
Which gets us back to the second graf and that big, scary figure: Companies lose $759 billion a year to worker slacking-off! Again, when you see a figure that precise in a story of this nature, you should assume it's a lie until it's proven innocent.
How do we get that figure? An hourly number derived from a national salary average,* times the amount of screwing-off. And what's wrong with that? Again, is Internet access evenly distributed? No; it's skewed (particularly broadband) toward younger groups. Funny, they're the same groups that -- if we could trust the numbers in this "survey" -- report the most time-wasting. And we multiply that by a number that's usually skewed toward older populations? And pretend the result is some sort of national average? Excuse my calculator; it wants to hork on the carpet.
Should you happen to be working for a particular flavor of reader-friendly twinkie, there's a 48.2 percent chance the next comment will be something like: Oh, lighten up. It's a talker. I bet it's the most-read story in the paper. So who cares if it isn't run like a Gallup poll?
There are three answers to that.
One: I'm sure readers will love it. I'm sure they'd love
HITLER REINCARNATED AS MOBERLY TOT
In Austrian lisp, 2-year-old demands 'lebenthraum'
too. But we're a newspaper. We don't run stories that we don't have some reason to believe are true. Do we?
Two: This particular stacking of the deck is racist, classist, and biased in favor of the powerful against the powerless. It comforts the comfortable and afflicts the afflicted -- all those longstanding press traits that make serious readers guffaw when they hear about "liberal" "bias" in "the" "media." Do we want to take diversity seriously, or is it expendable for a good piece of 1A fiction?
Three: Won't be too long before there's another election around here. Do we want to have some credibility when we run poll stories, or would we rather have readers just assume we're too dumb to know a real poll from a bogus one?
The AP should have known better than this one. So should the Missourian. (Normally I'd bitch about an ayem paper's using an 11 a.m. story when a 3 p.m. version was available, but in this case we managed not to get in the governor's response -- an irrelevant actor offering a boilerplate comment about a story that wasn't true to begin with -- so I'd score that as a net gain.)
That's a lot of noise for a small story. Go forth and whack a dragon or two with it.
* We aren't told, of course, whether it's a mean or a median. For extra credit, work out for yourself why that would make a difference in this calculation.
To repeat a previous theme: Since a newspaper copy desk doesn't have folks detailed to look up every single assertion that crosses its screens, editors need to rely on well-informed instincts and a checklist of practiced suspicions. Here's a look at how you can apply those habits to a real-life story -- and, conveniently, to any other story of its type. Because (here's the punch line) you will see another story that fibs with numbers. Maybe not today and maybe not tomorrow, but soon and for the rest of your career.
Our candidate is "Missouri is No. 1 in wasted time" (1A Tuesday). It's easy to see from the second and third grafs why it made the cut in budget meeting: Workers waste time on the job! A lot! It costs business billions! And Missourians are the worst fork-offs of all! Your job as a news cop is to follow this one long enough to see if it starts weaving. It does in the second graf, in that sentence ending with "... according to a new Web survey by America Online and Salary.com."
"Web survey" is a polite way of saying "b*lls**t." Whenever you see it, assume the content of the story to be false until you see a graf explaining the methodology. There are some rigid ways of collecting good data on the Net, as there are ways of collecting good data on the phone. Any story that's done the work will explain it. Any story that doesn't is a fake until proven otherwise.
In this case, the story doesn't, but if you poke around a bit at the originating sites, you get an idea of how the data were collected: Visitors to an AOL job site and Salary.com could fill out the survey if they found it. And that means the third graf -- "The No. 1 state for wasting time was Missouri, where workers who responded to the survey resported slacking off 3 hours and 12 minutes a day" -- is crap by definition. The numbers themselves might be true, but we have no reason whatever to believe them. Any numbers pulled out of the air have an equal chance of being true.
"Visitors to AOL's job-search site can click on the survey" is, in short, the same thing as going to the mall and asking the first 10 non-scary people you meet. It's a convenience sample, and you can't draw conclusions about the population from it. It doesn't make any difference that 10,000 people allegedly responded. No generalization from that sample or any part of it -- the national average, the Missouri average, anything -- is worth reporting, because there's no reason to believe it's true.
End of screed, technically. The whole premise of the story is false. We have no way of knowing how much time workers waste anywhere. Any sub-conclusions are false too ("fruit of the poisoned tree," it's called). But many of the other dishonest things the story does might show up in other stories, so they're worth mention.
Here's one: Is Internet access equally distributed in the target population? Not if we're talking about "the average worker." It's lower in black and Latino populations, and it's lower in lower-socioeconomic-status populations. So even a random sample of the Internet population wouldn't allow you to say anything about the population as a whole. (Rule: You can only generalize from a random sample, and you can only generalize to the population you sampled. No exceptions.)
Here's another: What's the most common form of wasting time at work? Personal Internet use! So any group that doesn't have Internet access at work wouldn't count either. (You're getting the idea? There's more to the working world than upper-middle-class white yuppies?)
And that leads to another, which I'll admit I came to because Tuesday is trash pickup day at HEADSUP-L Manor, and I was tempted to run this one out and show it to the guys on the garbage truck, because I bet they weren't surfing over to ThighNoon.Com between houses. Lots of folks out there put in a pretty full workday, and some of them put in more: What in this "survey" accounts for time past 8 hours? Where does the time Wal-Mart steals from the hourly worker figure in?
Which gets us back to the second graf and that big, scary figure: Companies lose $759 billion a year to worker slacking-off! Again, when you see a figure that precise in a story of this nature, you should assume it's a lie until it's proven innocent.
How do we get that figure? An hourly number derived from a national salary average,* times the amount of screwing-off. And what's wrong with that? Again, is Internet access evenly distributed? No; it's skewed (particularly broadband) toward younger groups. Funny, they're the same groups that -- if we could trust the numbers in this "survey" -- report the most time-wasting. And we multiply that by a number that's usually skewed toward older populations? And pretend the result is some sort of national average? Excuse my calculator; it wants to hork on the carpet.
Should you happen to be working for a particular flavor of reader-friendly twinkie, there's a 48.2 percent chance the next comment will be something like: Oh, lighten up. It's a talker. I bet it's the most-read story in the paper. So who cares if it isn't run like a Gallup poll?
There are three answers to that.
One: I'm sure readers will love it. I'm sure they'd love
HITLER REINCARNATED AS MOBERLY TOT
In Austrian lisp, 2-year-old demands 'lebenthraum'
too. But we're a newspaper. We don't run stories that we don't have some reason to believe are true. Do we?
Two: This particular stacking of the deck is racist, classist, and biased in favor of the powerful against the powerless. It comforts the comfortable and afflicts the afflicted -- all those longstanding press traits that make serious readers guffaw when they hear about "liberal" "bias" in "the" "media." Do we want to take diversity seriously, or is it expendable for a good piece of 1A fiction?
Three: Won't be too long before there's another election around here. Do we want to have some credibility when we run poll stories, or would we rather have readers just assume we're too dumb to know a real poll from a bogus one?
The AP should have known better than this one. So should the Missourian. (Normally I'd bitch about an ayem paper's using an 11 a.m. story when a 3 p.m. version was available, but in this case we managed not to get in the governor's response -- an irrelevant actor offering a boilerplate comment about a story that wasn't true to begin with -- so I'd score that as a net gain.)
That's a lot of noise for a small story. Go forth and whack a dragon or two with it.
* We aren't told, of course, whether it's a mean or a median. For extra credit, work out for yourself why that would make a difference in this calculation.
posted by fev at 11:02 PM
0 Comments:
Post a Comment
<< Home