Democrats eat babies, too

What are our late-night-news-hungry friends finding over at the Web site of the Fair 'n' Balanced Network? Glad you asked!

Certainly seems to be a particularly interesting story there, doesn't it? Those pesky Democrats want us to lose the war! And it's a real poll! So it has to be true!

Well, let's cavil about execution a bit before we get to substance. Give Fox a point for posting a PDF of the questions -- well, at least all the questions starting with No. 40. You have to wonder what might have lurked, say, at 27:
Yur leg it has a flavr:
[] Strongly agree
[] agree
[] neutral
[] disagree
[] strongly disagree

And methodologists are divided on the mythical Question 39:
[] Ya! Cthulhu fthagn!
[] No! Cthulhu no fthagn!
[] Neither ya nor no!
[] Not sure!
[]Other!

But let's talk about what we can actually see here in the empirical world, not what might have primed the respondents during the first nine-tenths of the survey or anything.

Fox reports the poll's margin of sampling error* as 3 points, which is understated; it's about 3.3 points, or 10 percent larger than we're told (that's a maximum, for the entire sample on a question with a 50-50 split; for Democrats on the question at hand, which is split about 4-1, the margin is 4.1 points). The poll was conducted over two days, in the evenings, which is a good way to lose points in your beginning methods class. This claims to be a national sample of registered voters, but it's a national sample of registered voters who are home in the evening. In this zip code at least, such a sample is a long way from being representative.

In fairness, you might respond here: Who cares? One major party is defending America and one wants us to start speaking German (oops! Arabic!), and you're bitching about stuff like basic standards in sampling and reporting public opinion? Well, yeah. Standards are how we know that research results are valid and reliable. And another couple are worth mentioning.

The reason you always want to introduce the concept described in graf 4 as "margin of sampling error," rather than "margin of error," is that "sampling" is only one of the many kinds of error that bedevil research. It's an annoying cliche to say "there are no stupid questions," because in fact there are rather a lot of them. Stupid questions are a leading source of error too, as in these samples from other recent Fox/Opinion Dynamics surveys:

Which of the following two descriptions of the way things are today do you agree with more?...The nation's economy is not improving, Osama bin Laden has not been captured and the United States is in an unnecessary war with Iraq. The nation's economy is strong, there have been no new terrorist attacks in the United States and the world is safer with Saddam Hussein out of power and an elected government in Iraq. (May 2006)
Each of the options has three components. Which one is the poor respondent responding to? What other ways are there of describing "the way things are today"?

How do you think President Bush should decide how to proceed in Iraq--should he follow his gut instincts about what is right or follow public opinion polls? (July 2007)
False dichotomies are almost by definition Stupid Questions. If you were writing this question, can you think of a few other options open to the nation's chief executive?

Who do you think is more to blame for the way things are going in Iraq?...The Bush administration for starting the war, Congress for approving the war and continuing to fund it, the Iraqi people themselves for failing to take control, Al Qaeda and insurgents for attacking the region ... (July 2007)
Check out the embedded presumptions in the available responses. Choice 2 assumes that there was a vote in Congress to "approve" "the war," which there wasn't, and it double-barrels the initial vote with an unspecified sequence of later votes that may or may not have specifically been about "the war." Choice 3 assumes both that the "Iraqi people" could have done something called "taking control" and that they tried and failed to do it. Choice 4 drags a large red herring across the trail by dragging Qaida into the mix.

So "question error" broadly is the set of stuff that asks: Given that N% (+/- 3.1) of subjects said "yes" or "1" or "2," what does "yes" or "1" or "2" really mean? Which, with all due respect, is how rational grownups need to approach Fox's Question 40:

Do you personally think the world would be better off if the United States loses the war in Iraq?

OK, how would you have responded? "Gee, FOX News, I thought we'd already won that war!" Or: "You 'win' a war when you plant your flag in the smoking ruins of the enemy capital. Do you mean 'would the world be better off if the United States pulled up its flag and left Baghdad,' or do you mean 'would the world be better off if Iraq planted its flag in Washington'? Because those are two radically unrelated questions that have somewhere between 'lots' and 'nothing' to do with the political choices I might decide on?"

Polling isn't inherently dishonest, but dishonest polling is inherently dishonest (and, yeah, ineptitude doesn't exactly help the situation, either). So I suppose this is a reminder that the great battle isn't between "left" and "right" wings of American journalism. It's between people who try pretty hard to get stuff right (and often, for whatever reasons, fail -- sometimes spectacularly) and paid professional liars, which would be the side Fox is on. Do bear that in mind as the season wears on.

Still wondering? Check out the last item under "Latest News," above: "Barack Obama Stops Wearing American Flag Lapel Pin." Q-E-sodding-D.

* I'm getting tired of saying this, but what the heck: "Margin of error" is meaningless unless you report the confidence level it's calculated at. Report your confidence levels or be branded a goober for life.

Parade of stupid questions

Quick, what's a good argument against running a hed like this:

Are remains truly Joan of Arc's?
Experts hope to prove whether skin, bone fragments are saint's
PARIS - Nearly 700 years after the death of Joan of Arc, a French forensic team hopes a series of tests will prove whether charred fragments of skin and bones might be those of the 15th-century heroine

How about ... because we ran the same hed on the AP version three weeks ago!
Could remains come from Joan of Arc?
A team of scientists hopes to crack one of the layers of mystery surrounding 15th-century French heroine Joan of Arc: Could a rib and other fragments recovered after she was burned at the stake be hers?

Notice that although the heds ask different Stupid Questions, the ledes ask effectively the same one. And the answer is: Of course they could! And they could be Fideaux the dog!

If we'd bothered to read our own AP tale, we might have noticed something relevant. Given that there's no sample to compare the relics with, there's no way on God's green earth we can know whether they're hers. Best we can do, apparently, is determine whether they're actually a human female from around the 15th century (it's hard to tell whether the doc or the AP is responsible for saying tests can determine the stuff's "exact age," but if that didn't strike you as patent nonsense from the outset, please post your credit card number here for a chance to win your own bridge across the Rhone). We might be able to rule out Joan of Arc, or John of Arc, or Fido of Arc, but we aren't going to be able to rule anybody in. And we probably could have figured that out without running the story twice.

HEADSUP-L would really appreciate it if we exercised a little more judgment over the stuff that goes on the religion pages. There's no reason the Faith section has to be the Stupid section. Come to think of it, there's no reason the A section should be either, but one battle at a time.

And sit up straight when I'm talking to you

If you wandered into the Fair 'n' Balanced homepage Monday afternoon from somewhere in the adult world, the presuppositions in the No. 4 story might have seemed a little unusual:

Former "View" co-host Meghan McCain scolded Vice President Kamala Harris Monday for laughing when asked about the ongoing crisis in Afghanistan, comparing the "craven" veep to Joaquin Phoenix’s mentally troubled character in the 2019 film "Joker."

Harris initially laughed when asked about the U.S. withdrawal from Afghanistan and the controversial evacuation process as Americans try to exit the country due to the Taliban’s swift takeover. Harris has a long history of laughing when confronted with tough questions and famously giggled on multiple occasions when she was asked about visiting the southern border. 

Long history, famously giggled -- but wait! There's more:

... Phoenix’s "Joker" also suffered from uncontrollable laughter, sometimes at inappropriate times. He was eventually driven insane and became a psychopathic murderer. McCain made the comparison after Harris burst into laughter when a reporter shouted a question seeking her response to reports of Americans being trapped in Afghanistan.

"Hold on, slow down everybody," Harris said while laughing. She eventually answered the question but critics slammed her for appearing to find humor in the questions.

If you're new here, let's flash back to March and the best-known case of Fox policing Harris's inability to sit up straight and act like a young lady. Here's the No. 2 story from March 22:

Vice President Kamala Harris laughed while responding to a question from a reporter who asked Monday if she would be visiting the border amid the growing migrant crisis.

While taking questions from reporters outside of Air Force One, Harris was asked if she had "plans to visit" the southern border as the immigration crisis continues to develop.

The vice president responded to the query with a "not today" before laughing. She continued on to say that she had visited "before" and that she probably would go back.

The White House did not immediately respond to Fox News’ question as to whether they thought Harris’ response was appropriate, given the situation at the border that the Biden administration refuses to call a "crisis."

Things got worse the next day, because backlash:

Vice President Kamala Harris is facing a backlash after laughing when a reporter asked if she planned to visit the southern border amid the migrant crisis.

"Kamala Harris laughs at a reporter who asked her if she has plans to visit the border. Don’t believe this administration when they say they are serious about solving this crisis," Rep. Steve Scalise, R-La., wrote on Twitter on Tuesday.

And by the time we got to reporting on the afternoon talk shows March 30, Fox was in full behavior-cop mode:

The Biden administration's border policies are creating a catastrophe for everyone, Fox News contributor Katie Pavlich said Tuesday on "Outnumbered," in reaction to migrants receiving in-person schooling in San Diego while American students are continuing to learn from home. Pavlich blasted Vice President Kamala Harris for laughing off a question about the subject.

KATIE PAVLICH: "I don’t understand what she thinks is funny. American parents are getting screwed. Their kids are falling behind and the very people Democrats claim to be standing up for, minority students, aren’t getting an education and aren’t going to be able to compete with their peers in the United States. Not to mention across the world. And just on a basic staffer level, has nobody sat down with the vice president and said, you have to stop laughing when you’re asked serious questions. It makes you look out of touch, it makes you look incompetent. That’s a basic thing that they should’ve fixed by now.

Questions don't become "tough" or "serious" by themselves, of course. They become "tough" through being asked by Fox reporters. Here's a Peter Doocy example:

Doocy also asked about news that copies of Harris’ children’s book "Superheroes Are Everywhere," published in 2019, were given to migrant children arriving in the U.S.

Even Fox eventually had to admit that the "Superheroes" story was bogus:

At least one copy of Harris’ children’s book, "Superheroes Are Everywhere," was made available for migrant children arriving at the Long Beach Convention & Entertainment Center, a recently converted influx facility, along with basic hygiene supplies and clothing. The book was part of a citywide book and toy drive to support the migrant children staying there.  

... EDITOR'S NOTE: In a previous version of this story, it was implied there were multiple copies of Harris' book available for children in welcome packs. 

By July 3 (yeah, slow holiday weekend), her uppity impudent cackle had become the lead story:*

Jesse Watters said Friday on ‘Tucker Carlson Tonight’ that Vice President Kamala Harris' signature 'cackle' is a "defense mechanism" when she is faced with a question she can't answer or doesn't want to respond to, adding that it often is employed to lighten the interview – which he said notably failed during a recent back-and-forth with NBC News anchor Lester Holt.

JESSE WATTERS: as someone who made his career doing man-on-the-street interviews, I feel like I am uniquely qualified to recognize when someone is uncomfortable in public -- and yes, this is a sign of discomfort when she's presented with the topic or question, when she's unsure or uncomfortable, she results to cackling.

In our industry, we refer to this as a defense mechanism. In layman's terms, it perhaps would be a glitch in the system, a tick, or a 'tell' in poker and she does it for several reasons: One, to soothe her anxiety. Two, to buy her time if she's unsure, she can use that time to formulate a response. Three, to kind of guide the vibe of the interview from a serious to a less serious one, and then finally, it is kind of a lame attempt to form a bond with the person asking her a tough question. 

Going back to last fall, one could point out that the laugh is often a response to a "stupid" question: 

Sen. Kamala Harris D-Calif., laughed when asked during an interview Sunday if she would advocate for a “socialist or progressive perspective” if elected.

CBS News anchor Norah O’Donnell asked Harris the question during a "60 Minutes" interview with her and the Democratic presidential nominee Joe Biden.

... and even then can quickly spin off a second story:

Podcast host and author Ben Shapiro on Monday ripped Sen. Kamala Harris, D-Calif., for laughing at an interviewer who asked if she had socialist views.

“Kamala Harris -- I would love to play poker with Kamala Harris because she has the most obvious tell in the history of politics which is if she is asked a question that she does not want to answer, she breaks into that insane Joker laugh and it is pretty wild,” the host of "The Ben Shapiro Show" told “Fox & Friends.”

Shapiro said that “there is nothing funny about that question” and the follow-up question about her being a “liberal senator.”

It's almost as if ... I don't know, there were some immutable demographic characteristics that make Kamala Harris's response to stupid questions especially terrifying at Fox News. What could those characteristics be?

* It's a bit odd to have the "Fox News Staff" byline, rather than a named staffer, atop the lead story, but Fox talk shows have a habit of producing MAJOR BREAKING NEWS like this.

No, but thanks for asking

What a delightful illustration of the difference between question-begging and Stupid Questions! What would we do without The Fox Nation?

OK, the embedded question isn't actually question-begging; it's the interrogative form of question-begging called the "fallacy of multiple questions." You can't answer a question about what "Obama's class warfare rhetoric" did without acknowledging that there is such a thing as "Obama's class warfare rhetoric." That's an informal logical fallacy, as opposed to the Stupid Question, which is evidence of brain-dead cluelessness. Here, we can answer the Stupid Question by asking how many times the hated name of the Kenyan Muslim socialist usurper is mentioned in the story -- either the 390-word version at Fox Nation or the glorioous 1,300 words of AP prose at Fox proper.

The answer in both cases is, um, none. Not only is there no known "class warfare rhetoric," there's no Obama. Whatever the Trotskyite rascal is up to, he isn't up to it here.

One of the fun things about teaching a methods course is reminding the younguns that screwing up isn't necessarily an ethical failure. Lying, on the other hand, is. We all screw up sometimes, but lying is a matter of choice. Or, at certain news outlets, a matter of policy. Please don't hesitate to point out the differences.

Dunno

Two possibilities suggest themselves:

1) The story contains the answer, in which case it'd be nice to be somewhat more direct.

2) The story doesn't contain the answer, in which case the appropriate reply is: Why are you asking me?

OK, maybe (3): Somebody bet that the desk couldn't spell the guy's name right on 1A.

On the Stupid Questions scale, this is distilled essence of stupid, matured in charred stupid barrels and served neat. You should charge more than a buck a copy for it.

March of the Stupid Questions

Dear friends at the Daily Herald: If you want to demonstrate the paper's value, shouldn't you use the space above the fold to tell the audience more things than you ask the audience?

As a reminder: The question mark is not a form of attribution. If a story isn't well enough sourced to support a headline that tells me something, you should be asking questions of the story -- not of the audience. 

Tricksy whereaboutses!

Dear AP: There are so many, many answers:

1) Toss a coin. Everybody else does
2) What does it sound like it should be?
3) Well, what if you looked it up?

Door No. 3 might lead you to a reference like the Merriam-Webster Concise Dictionary of English Usage, which would tell you that (a) the "s" has nothing to do with number, (b) singular and plural verbs are both standard with "whereabouts," and (c) plural verbs appear to be more common,

In other words, if you want a rule, knock yourself out. Go ahead and write one; you won't be wrong, whatever you decide. A sensible usage rule might be: OK, usually plural, but go ahead and make it singular if you're really feeling it. The world won't end either way.

What you should never -- as in, please stop wasting the time of hard-working editors -- do is invent a rule based on a nonsensical reading of the matter at hand. Why would noun-verb agreement be affected by the number of a possessive pronoun that precedes the noun?

My house is big
Our house are big

Your attitude sucks
Their attitude suck


This would seem less absurd if we weren't in the middle of the War on Editing. Time wasted on stupid questions is time diverted from the main effort. Pronouncements about bogus rules only underscore the glassholes' contention that editors are a waste of time. If this is the impression we give of editing as a craft, we should not expect editing to survive as a craft.

The AP shouldn't be wasting time on questions for which the first answer is "did you look it up?" If it must, though, it should at least avoid doing harm. Like most people in my line of work, I teach AP style as part of the basic professional toolkit. If "AP style" continues to manifest itself as a set of unintelligible, unfounded claims about some mystical grammar world, I'm going to revisit that decision in favor of some style guide that makes sense.

The road to hell is ...

... due for resurfacing in Summer 2008 or thereabouts, until which it's paved with well-intentioned stories like unto the following. It seems harsh to call it bad journalism, because its heart is in the right place and ultimately it sort of seems to get somewhere near what the researchers might have been close to finding, and the conclusions are probably good for you even if they're reached on specious grounds. Still, you're left wondering -- what if we actually tried to report accurately on this stuff?

BODY IMAGE
Articles on diet bad for teens?
Study links reading on weight loss to eating disorders

The big type is the fault of the member paper, not the AP (with an implied exception or two we're going to get to in a moment), so to the extent this is "bad journalism," the copydesk deserves a chunk of the blame. From the top: The relevant results of this study aren't about body image, they're about behavior. Stories about assertions should never have question heds, and this study doesn't examine what is or isn't "bad for teens" anyway. And it also "links reading on weight loss" to ... exercise! and eating vegetables! And other stuff like that.

Setting aside the inept treatment in the big type, you can get a sense of why this was waved through. Here's the lede:

Magazine headlines entice teenage girls with promises: "Get the body you want" and "Hit your dream weight now!" But a new study suggests reading articles on diet and weight loss could have unhealthy consequences.

I'll take your word for what the headlines sound like. Trouble is, this study doesn't have anything to do with what sorts of headlines are used to entice teenage girls (or, as the article acknowledges later, with how articles about weight loss are illustrated, or what they say, or anything like that). Articles that say "be happy with your body" are the same as "get the body you want" for these purposes.

Teenage girls who frequently read magazine articles about dieting were more likely five years later to practice extreme weight-loss measures, such as vomiting, than girls who never read such articles, the University of Minnesota study found.

Yep. That's one of the things the study found.

... The study, in January's issue of the journal Pediatrics, adds to evidence that girls' attitudes toward their bodies are shaped by popular culture.

Really? As the authors write: "It is somewhat surprising that associations were found with weight-control behaviors but not with body satisfaction" (emphasis mine). The same can be said for self-esteem, only more so; the p value there is .949.

The new findings were based on surveys and weight-height measurements of 2,516 middle school students in 1999 and again in 2004.

That's not true. The 2004 part of the study (the "panel" part, which at least allows a bit of thinking about cause and effect) involved surveys only. The weight-height measurements were done in 1999. Let's not hear any complaints that that's a quibble. This is journalism. You aren't supposed to make any of it up.

About 45 percent of the students were boys.

Only 14 percent of boys reported reading diet articles frequently, compared with 44 percent of girls. For those boys who did read about weight loss, there was no similar lasting effect on behavior. (Which underscores the importance of not raising Stupid Questions about what is or isn't "bad for teens" in the heds.)

In the new study, it was unclear whether it was the diet articles themselves or accompanying photographs of thin models that made a difference. The study didn't ask teenagers which magazines they read, only how frequently they read magazine articles "in which dieting or weight loss are discussed."

It wasn't "unclear." It didn't even come up. The study doesn't ask about illustrations, or which magazines are read, or what the heds look like, or whether the articles encourage positive body images or the "thin ideal" that the study is justifiably concerned with. It asked one question -- "How often do you read magazine articles in which dieting or weight loss are discussed?" -- and allowed four responses: Never, hardly ever, sometimes, or often. Then, five years later, it asked some behavior and attitude questions.

Now, those aren't irrelevant or uninteresting results. But the point is they say only what they say. (One of the things they say is that -- at least up through "sometimes" -- reading articles about weight loss is associated with the weight-loss behaviors the study calls "healthy," like exercising and eating more fruits and vegetables.)

The study was based on students' self-reports about their behavior and, like all surveys, could be skewed by teenagers telling researchers what they think the researchers want to hear, said study co-author Patricia van den Berg.

Time out to complain about inept writing. A survey can't be skewed by "teenagers telling researchers what they think the researchers want to hear" unless it uses teenagers. But it is subject to social desirability bias like any other survey. Meaning you can state the possibility of error caused by self-reporting as a fact, independent of whether the researcher admits it or not. It's good of van den Berg to point it out, in case the AP had overlooked the possibility, but it doesn't need attribution.

Nathan Christopher, a spokesman for Seventeen magazine, said health is important to the magazine's readers. He wouldn't comment specifically on the study because it was unclear which magazines the teenagers read.

Fair enough. Why should he? More to the point, why did we ask him?Again, it's not "unclear" what magazines are involved. It's never even come up.

Thus, much as this one has good intentions and could end up having good outcomes, somebody needs to throw the flag on it. Is it too much to ask for good execution along with the good intentions?

Asking simple questions

Honest, I'm not trying to pick on this particular survey, or all stories depending from it, or anything like that. But we do seem, once again, to have a story that's eating up a bunch of front-page space without, apparently, adding anything to the conversation. Let's walk through it step by step and see if we can raise the questions an editor should have thought of earlier.

As the immigration debate heats up across the country, a new study shows Latinos in Charlotte-Mecklenburg are as divided over immigration reform as any other group — and possibly more so.

That's a lot of assertions and implications for one lede, so let's skip the bit about whether we measured the degree to which the immigration debate is heating up across the country. Is there a new study, is there a measurable thing called "divided over immigration reform," does the study measure it, and are Latinos higher on it? Let's see.

The Crossroads Social Capital study, which measured social ties in the community, found almost six out of 10 Latinos (58 percent) in Charlotte-Mecklenburg feel immigrants are “too demanding in their push for equal rights.”

Oh. Would this be the Crossroads Charlotte Social Capital Benchmark Community Survey, reported back in the first week of June? In which the "too demanding" question was the 17th graf? Can someone explain why "too demanding in their push for equal rights" is a measure of support for "immigration reform"? What does it mean for a greater proportion of a subgroup to agree with this question: is a 50-50 split more or less divided than a 60-40 split?

“I'm upset at some of the demands I hear some parts of the illegal community making,” said Ricardo Mata, a Venezuelan native who has lived in the country for two decades. “Sometimes, I get fed up at the double standards I see.”

So when we shift to real people, it becomes a little clearer that the survey and the lede are talking about different things.

... Critics of the study's findings say they reflect only a small segment of the community and not the majority of Latinos who do support immigration reform.

Really? Do we have comparable data on how this majority feels? Local or national? How are the assertions in this graf going to be supported?

“I think the people who were surveyed were mostly established Latinos who are not having to face this issue,” said Angeles Ortega-Moore, executive director of the Latin-American Coalition.

Here we're getting data confused with opinions again. The survey either did or didn't draw a representative sample; if you don't like the results, that's fine, but this isn't the sort of question that's addressed by asking people what they think.

One hundred seven people who identified themselves as Hispanic or Latino participated in the Crossroads study. The full study's margin of error was plus or minus 3.24 percentage points.
Didn't we discuss something like this before? Why report the confidence interval for the full sample* when we're interested in a subgroup? For which, on this question, the margin of sampling error is 9.4 percentage points?**

While the findings don't appear to track national trends, they do seem to follow economic and generational lines. The longer and more successful Latinos have been in Charlotte-Mecklenburg, the more likely they are to think newly arrived immigrants are too pushy.

What trends don't they "appear to track," and who(m) don't they appear to track it too? Are we comparing questions about immigration reform or questions about whether immigrants are too pushy? Should we note along here somewhere that not all Latinos are immigrants and not all immigrants are Latinos?

The study also shows that Latinos are not monolithic thinkers and that some disagree with parts of the immigrant rights movement.

Uh ... does anybody outside the Observer newsroom really need to be reminded of those points?

Anyway, then we're off into Random Interviewland, talking with people who do or don't espouse the views that the study doesn't address. None of it gets near addressing the points in the hed or the lede, which -- presumably -- were the selling points for putting the story on the front.

The hed, of course, is abysmal; it's a Stupid Question, for one thing, and it's an incorrectly formed question at that. The deck raises issues that the story at least touches on in passing, but if the study's reported correctly here, they're factors in how people view the "demanding" question -- nothing, to bang on the main point again, that has anything to do with views on immigration reform.

And that's really the problem. If the story was sold as a bit of survey data that shed light on how people view immigration reform, it was mis-sold. It doesn't. It rehashes a finding that was a bullet point in a story two and a half weeks ago and dresses it up with quotations that sometimes address the main point and sometimes don't. It's hard to see how the sum of human knowledge would have been hurt by holding this story up until a few fundamental questions were answered.

* Just report it to one decimal place, OK? Two is spurious precision (though it's not as bad rounding to a whole number, as the NYT does).
** At 95% confidence (kids, don't forget those confidence levels!)

Bad hed, ray of hope, bad story

Time to complain about one of the Worst of All Hed Adjectives for a moment. That word is "faithful." Lots of writers use it in the sense of "having religious faith" -- not wrong, though Webster suggests it's obsolete. The biggest trouble with it in heds like the one shown here is other meanings are equally, if not more, accessible. Are the cousins down at the Dispatch talking about people who are conscientious about voting or religious people who vote?

Turns out it's the latter, which leads to another problem. The survey described under the hed doesn't attribute the toe-keeping characteristic to all religious voters, which suggests that the paper is adopting some of the pollster's baggage. That'd be the Barna Group, which calls for a few comments on the art of polling that may be worth noting over the next few months.

There's a common misperception out there that pollsters can wave a secret magic wand and make polls say anything they (or their paymasters) want. That's not true -- or at least not very true. Given a properly drawn random sample and mildly professional execution, a question like "If the election were held today, would you vote for A, B or someone else?" is going to produce a (more or less) valid and reliable result, no matter who asks it.

That's not to say there aren't evils done in the name of polling. One form arises from questions that are badly written or tendentious. Here's a genuinely stupid one from a Fox poll last month:

When talking about United States citizens of different ethnic or national descents, do you think the word American should come before or after the descent? That is, should Americans of Irish descent be called Irish-American or American-Irish, and should Americans of African descent be called African-American or American-African?

Does this mean Fox's results on the "If the election were held today ..." question aren't an accurate representation of what registered voters think?* No; it means Fox's polling firm will throw in anything the bottom-feeders at Fox think needs to be asked.

The other big offense that "pollsters" (as opposed to the journalists who invent portentous trends based on perfectly innocent results) commit is not showing their work -- in this case, describing outcomes without demonstrating how thoroughly those outcomes are supported. Such is the case with the poll the Dispatch falls for here:

If the presidential election were today, Barack Obama likely would become the first Democrat in 32 years to win among born-again Christians.

In fact, of the 18 faith segments tracked by pollster George Barna, the Illinois senator currently leads in all but three. Only two groups of evangelical Christians and those with an "active faith" favor Republican John McCain. (The writer means two groups, evangelicals and actives, not two groups of each. Yoo hoo! Editors!)

But Barna finds that the political landscape among the faithful is churning rapidly, with support for Obama dropping almost across the board.

Time to stop the tape and unpack a few things. Yes, your first questions at this point should be about sample size: How big is the poll, how big are those subgroups? Without that information, you can't make a reliable judgment about whether the "political landscape" is "churning." (That's just a mixed metaphor; "churning rapidly," on the other hand, is an invention on the reporter's part. "Rapid" isn't the sort of thing you can measure with two polls two months apart.)

There's where Barna loads up the old spitball -- which, given that it's a free country and all, he's welcome to do. But it's the paper's job to ask the questions, and it fails. For one, according to the graphics, the overall sample is 1,003 adults, but the conclusions Barna reports are among "likely voters." In the June poll, that was a substantially smaller chunk, or 561 people. That makes for a noticeable difference in the confidence interval. Does the August poll make the same distinction? Gee. We don't seem to say.

Now, how big are those 18 "faith segments"? No idea. How many of them overlap? A bunch. How can we assess that "across the board" claim? Apparently through faith alone and not by works, and that isn't going to get you into methodology heaven.

The Barna Group doesn't make any secret of its biases, at least. And the biases don't mean the raw data are bad. They do, though, strongly suggest that any conclusions about "evangelicals" should be taken with a barrel of salt. Broadly, one of the ways Barna sorts Christianity is into three categories (look up the full definitions toward the end here): born-again, evangelical and "notional." So when the pollster himself offers assertions on the order of "While there is still a decided preference for Senator Obama, the more conservative element of the Christian population is slowly coming to grips with what an Obama presidency might be like," the writer needs to be aware -- and to make the audience aware -- that things like "core" and "conservative" might mean very different things in Barna World than they mean in the corporeal one in which political campaigns are fought.

You can say one good thing on the hed's behalf. It doesn't make any proclamations about who's ahead or behind. More survey heds should be so inconclusive.

* From the list of questions (give Fox credit for linking to the whole thing), it also appears that the preference question comes first -- meaning, at least, that the results aren't misprimed by questions like "How likely do you think it is that Obama is secretly a radical terrorist? Verry likely, somewhat likely, or only a little likely?"

Gollum gollum gollum

While you're enjoying Friday morning's Fair 'n' Balanced homepage (in order, Brave Massster! Wicked, tricksy Soros! Hobbitses are mean to Massster! Massster makes us happy!), ponder what it might look like when different arms of the Murdoch empire start to disagree with each other. Here's the Journal, for example, from the beginning of the week:

The report concludes “with high confidence” that Mr. Putin “ordered an influence campaign in 2016 aimed at the US presidential election” to “undermine public faith in the US democratic process, denigrate Secretary [Hillary] Clinton, and harm her electability and potential presidency.” It also concludes that Mr. Putin “developed a clear preference for President-elect Trump.”

Yet the report offers no evidence or judgment that the hacking influenced the election result. The leaks from Clinton aide John Podesta’s email and the Democratic National Committee were embarrassing in their candid views of individuals, but they included no bombshells. The emails that really hurt Mrs. Clinton’s electability were those she kept on a private server while Secretary of State.

My, my, my. That's certainly not what Massster said at his news conference!

But remember this — we talk about the hacking, and hacking's bad, and it shouldn't be done, but look at the things that were hacked. Look at what was learned from that hacking, that Hillary Clinton got the questions to the debate and didn't report it? That's a horrible thing. That's a horrible thing. Can you imagine that if Donald Trump got the questions to the debate, it would have been the biggest story in the history of stories, and they would have said immediately, "You have to get out of the race." Nobody even talked about it. It's a very terrible thing.

Nor, of course, is it what Fox thought as those bombshells kept landing:

Should you have actually read the stories, you probably agree with the Journal: Even the ones that Fox fabricated (no, the Clinton Foundation didn't pay for Chelsea's wedding; no, "a longtime Clinton confidant" didn't "express regret that the terrorist wasn't a white man") fall apart in the sad light of day. That doesn't mean that the emails on the Sekret Server were the ones that "really hurt Mrs. Clinton's electability"; the Journal is making that up, because that's what editorial writers do with fact claims about public opinion. It means that "the emails" worked as a Fox story because real issues and fake issues became indistinguishable.

Read more »

Forget flood. Interview ... no, don't

Often, the best answer to a Stupid Question is "no, but thanks for asking." That won't work in this case, because the problem isn't that the hed is asking a question that the story answers; it's asking a question before which journalism must either stand mute or, you know, break out the tools of the supernatural. And that way lies witch-burning.

Earlier this week, the Christian Broadcasting Network's chief political analyst teed up a question for White House Press Secretary Sarah Sanders.

"Does it kind of blow your mind that someone like Donald Trump, who is sitting in the Oval Office," said CBN's David Brody, "I know you can list the accomplishments, but at the same time just from a spiritual perspective, there are a lot of Christians who believe that for such a time as this ..."

"For such a time as this" is the key phrase in that sentence. It's a quotation from the Bible's Book of Esther, in which an unlikely savior delivers the Jews from persecution.

Sanders picked up on Brody's biblical implication and ran with it.

"I think God calls all of us to fill different roles at different times," said Sanders, an evangelical Christian herself. "And I think he wanted Donald Trump to become president and that's why he's there."

While Sanders' statement may have raised secular Americans' eyebrows, many white evangelicals likely agree with her. According to a 2017 survey by Public Religion Research Institute, more than half (57%) say God played a "major role" in the 2016 presidential election.

If your eyebrows haven't already been raised by Axis Sarah, it's hard to see why this would get their attention. But you might have also gathered that you're not going to get the interview that the headline promised.

That view is particularly pronounced among charismatic and Pentecostal Christians, a subset of evangelicalism that puts special emphasis on prophecies, believing that God is omnipotent, immanent and extremely active. That is, all-powerful and present in all areas of existence.

The fullest accounting of this view comes in Stephen Strang's book "God and Donald Trump," in which the Pentecostal publisher writes that evangelicals had been praying for deliverance from an overbearing, hostile (and Democratic) federal government.

Trump, Strang says, was the answer to their prayers.
 
... But if a majority of Trump's white evangelical base believes that God wanted him to be president, many other Christians do not agree.

Less than half of non-white Protestants (47%) and fewer than a quarter of white mainline Protestants (21%) and Catholics (22%) say God played a major role in the 2016 election, according to the PRRI survey.

Now we're getting somewhere (not, of course, into answering the Stupid Question), but we're straying off into guesswork to do so. These are questions about what people think God did, not what people think God wanted. Specifically,  about what English-speaking Protestants (with a side of Catholics) think, and that seems a rather narrow view of the interaction of religion and politics.

To some extent, the question of God's role in the 2016 election is impossible to answer. After all, who among us can claim to know the mind of God? 

If you need your religion editor to point out that -- with some qualifications, mind -- we can't entirely answer what God was up to in the 2016 election, you're doing it wrong. (Hell, we can't entirely answer what the Russians were up to, and Russia's on the map.) And quite a few among us feel that life is but a joke claim to know the mind of God. Traditionally, we ask them to buy an ad.

If you want to analyze public opinion, fine. Just keep the emphasis on questions that were actually asked -- and entities that actually pick up the phone.

Hat trick of Stupid Questions!

The print edn of the Fort Worth Star-Telegram certainly sets the bar high with the hed at right (hint: No). Can the newly redesigned ("To our readers: Tell us what you think") Web site live up to this standard?

It's hard to top a photo that shows "what could be a UFO,"* but you have to admit, the early signs are promising. Here's the hed from the new! improved! Buzz (hint: Light show).**

And what could top an offer like this? Here's the story you get to if you manage to click in the right place:

His UFO sighting is captured on cellphone
Is it a UFO? (Hint: No)
Can't say for sure. (Hint: Can too!)
But truck driver Sean Kiel said he photographed an unidentified flying object with his cellphone as he drove along Interstate 20 near Cisco on Jan. 8. (Breaker one-nine there, teddy bear: How about keeping your mind on your driving and your hands on the wheel?)
... Kiel, of New Haven, Ky., contacted Sherry Webb of the Texas Dairy Review in Stephenville. She put him in touch with the Star-Telegram. (Thanks, Sherry! Let us know if we can do you a favor sometime.)

OK, enough of that. Dear friends at Fort Worth, the newspaper is the home of the mundane and empirical. Save the supernatural for the horoscopes. And take a journalism tip from Texas Dairy Review: When somebody calls in with pictures of space creatures, the right answer is: "Thanks! Here's the number of someone who will be really interested!"

* On the other hand, it could be one of these Daleks from the rift at Canary Wharf. Or an emanation of Giant Albion.
** As a general redesign matter, elements that cut words in half in display type are usually not a good idea.

Stupid Questions: The archive

Fun things you run across while tuning up the hed collection for a new season of lecturing and whatnot:

Would Jesus eat Easter ham?

To be fair to the hed writer, the hed does reflect a question raised in the column itself. Still, you have to admit it's a benchmark in question hed inanity.

Dawn of the Stupid Question

Once more with feeling: No question heds on stories that make assertions. Ever. Under any circumstances. Sunday's fronts produce two distinct and distinctly awful examples.

First to Wichita, and there's a simple prescription for this: We don't do supernatural. We do mundane. People can say what they wish, and the Vatican can investigate what it wants, and it's fine to report all those things, but we don't do it by raising questions that are not answerable by earthly tools. (Yes, in a happier world, journalism would be a little slower to deify the medical profession, but we don't have to get there by belittling their work.) We don't do miracles and the pope doesn't slot 1A Mideast stories; sounds like a deal to me.

On to Detroit and a different flavor of Stupid Question: the question mark as attribution. Pause for a moment and think of how many ways this 1A lead story could be answered:
1) No, cops groped 11 men in Toledo!
2) No, cops groped 9 men in Detroit!
3) No, cops groped 97 men in Detroit!
4) No, cops didn't grope any men!

There's a fairly aggressive bit of reporting here (one of the few things left that the Freep still does beyond entry-level standards), and it deserves a grownup headline.

Please join the fight. Help stamp out the Stupid Question.

The AP's new clues

Here's an interesting item from this week's Ask The Editor collection at the AP Stylebook site:

I have a question about reporting poll results and the margin of error. The stylebook address a poll taken when two candidates are facing off, but I'm not exactly sure how to apply that to a ballot measure that requires a certain percentage to pass. Specifically, a poll found 51 percent of voters plan to vote against a measure, 42 percent are for it and 7 percent are unsure, with a margin of erro

And the response (the nut of it, at least; most of the response is taken up with noting that the questioner seems to have overlooked the character limit on questions):

I'll try to find out, but rpt just this sentence with the missing words ...

I don't think it's really a trick question,* or something unique to the AP. It's broader than that, and that makes it interesting, because it says something about how journalism interprets concepts like expertise and authority. "Margin of error" isn't something you can own, like Yastrzemski's batting average in 1967;** it's something you have to take an expert's word for (once you find an expert). And if you're interested in social science reporting, word choice, audience perceptions and other stuff that comes up around here at times, that's a way of understanding what things like "precise" and "credible" and "objective" mean when we apply them to news decisions.

What's the answer? Well, start with the question. The stylebook doesn't say anything about how to handle a poll "taken when two candidates are facing off" (or putting the gloves on, or taking the gloves off, or doing the horizontal hokey-pokey). It addresses how to discuss a difference between two candidates.*** There's no mention of whether that means two of two candidates or two of five candidates -- or if it's a primary with a 40% threshold, or a ballot issue ("yes" and "no" being about as two-candidate as you can get) with a 60% threshold, or anything like that, because none of those things affect what the margin of sampling error measures. So the answer is simple: Apply the same guidelines.

That doesn't mean the question is a stupid question. It suggests that the questioner doesn't know much about the domain involved (which is vastly different from being stupid). The problem is that a straightforward question ought to get a straightforward answer, and what it gets is a misleading (I'm trying hard not to say "stupid") one.

Misleading how? Because it pushes a mathematical formula that's only slightly more complicated than "earned-run average" into the domain of wizardly expertise. Journalists can't live there; we can only knock on the door, accept what we're told, and go home to see what the Times is saying. And the whole point of inferential stats is that -- as in baseball -- we all own them. We might use a batting average to support different arguments (you say a .367 average proves that power is irrelevant, and I say it fails to show that the clown doesn't hit with runners on second or third). But we're both using the same proportion, and in both cases it represents the same thing. If someone asks how to calculate batting averages if a player needs 350 plate appearances to qualify for the batting title, we have the same answer: No difference. Do all the same calculations, then go to "select cases" and include 'em out**** if PA is less than 350.

If we want to cure what ails poll reporting, and science reporting in general, that'd be a good place to start. The discussion section -- where you talk about what your statistics mean, and why they do or don't support your theory-driven predictions -- is where experts disagree (and ought to be consulted; there's nothing wrong with talking to smart people, as long as you're talking to the right people about the right stuff). The results section is a slightly more complicated version of what you've been doing since the first time you picked up a Sunday sports section and started pondering the AL Top Ten. You can play there. Any copyed, on any desk, anywhere, can own "sampling error" as thoroughly as the sports desk owns "earned-run average." And once you do, you can challenge any faulty conclusions that rely on your being too ill-informed -- or too cowed by magic -- to challenge.

Is the AP serious about "accountability journalism"? A little more aggression with undergraduate statistics would be a good place to start.


* As in "Hey, Stylebook! If a plane crashed on the border of two countries, where would you bury the survivors?" Though it's close.
** Though since you're reading this on a computer, you probably have a calculator a few clicks away, and the margin of sampling error needs maybe one more keystroke to calculate.
*** To its everlasting credit, the stylebook also notes that there is no such thing as a "statistical tie." Please heed it.
**** Should have been Berra, was probably Goldwyn. Alas.

Objectivity failure

Time for another brief rant about the public interpretation of survey research. (If you're growing tired of those, may we recommend the excellent Comics section downpage?)* As we mentioned at the weekend, there's a widely held -- and generally wrong -- belief that some people called "pollsters" do evil magic stuff to numbers while we aren't looking and thus distort public opinion to reflect (a) their own Maoist leanings, (b) the fascist tendencies of their paymasters, or (c) either of the above, as long as they can embarrass innocent citizens in the process.**

That'd actually be a really stupid way to do business. Polls can, and do, ask vacant, loaded, double-headed, misnegated or openly racist questions. But poll questionnaires tend to sound alike on the relevant stuff:

If the election were held today, would you ...
Would you say that's very likely, somewhat likely, or ...
On a scale of 1 to 5, with 1 being "very happy" and 5 being "very unhappy," how would you ...

because that's how you get reliable answers. You could draw a random sample of 1,004 people and ask if they're planning to vote for the terrorist or the war hero, but if that's your goal, it's a lot cheaper to just make up a few percentages. Properly sampled and professionally arranged, a poll from Fox is going to be every bit as reliable on the big-league questions as one from -- well, if you can imagine a left-wing U.S. news organization, a poll run by a left-wing news organization.

Which gets us to "objectivity" -- that happy state in which we base our results on things as they work in the real world. What "objective" should mean is that everything's in the open: questions, order, results, subgroup sizes, all that. If you want to run all the tests again and see if anybody's cheating, you can. We don't have to make dark inferences about motives; we can talk about facts.

Does that give you the idea that "objectivity" isn't, or shouldn't be, a requirement to let people tell lies? Hope so, because that's the point of today's example. McClatchy is reporting (it's a bit hyperbolic, as usual) on a survey of registered female voters that -- surprise, surprise -- finds a pretty massive lead for Obama among the so-called GenY crowd. Calling forth this response:

EMILY's List overwhelmingly supports Democratic candidates, and that slant is going to figure into its numbers, said Wendy Wright, the president of Concerned Women For America, a political action group based in Washington that opposes legal abortion.

Erk. This is a genuine failure of objectivity. Wendy Wright is welcome to say or think what she wants, but the reporter (failing that, the desk) is justified here in telling her to sit down and shut up. She can say she doesn't like the numbers, or that she expects American womanhood to come to its senses soon, but when she says people who make their living by running polls are stacking the deck on their results, she's out of bounds. (Try that with accountants or lawyers and they're likely to sue you for libel.) "Objectivity" gives her the information to run a survey of a random sample of 1,400 female voters and see whether the results differ significantly from the ones reported here. It doesn't create a right to lie.

But don't the numbers need some sort of comment from people who don't like Emily's List? That's actually a better question. If the numbers are accurate, why does it? And if they aren't, why is the story running?

* And yes, this is an editing issue. Editors -- until some front-office genius decides to put us out of the sport altogether -- are still the folks who are empowered to stand up and ask things like "Why are we putting that crap in the paper?"
** Response biases are really interesting. Lying about voting behavior tends to go in only one direction: People don't say "I didn't vote" if they did vote, but they're happy to say "I voted" when they didn't.

Stupid Questions of the (still-young) week

1) Dunno. When exactly was it that sex with robots became illegal? (Yeah, yeah. The answer Fox wants is: "Massa- chusetts: Who else?" But I'd bet on South Carolina if I were the betting sort.)
2) No. But thanks for asking.

Just another busy day of saving the republic over at the Fair-n-Balanced Network. Long as we aren't talkin' any wide-stance android blue-on-blue in the little robots' room, I suppose.

Dawn of the Stupid Questions

You do have to wonder what The Daily Herald thought would be a likely answer:

Yes!
No!
Compared to what?

The story, at least judging from the deck, has nothing to do with whether guns and pot "mix." It's about whether a medical marijuana law will require potential patients to "voluntarily surrender" their CCW permits and remind caregivers that they shouldn't have guns, subject to, you know, something.

Please. Given the choice between telling me something and asking me something, err on the side of providing information.

No, but thanks for asking

Today in Stupid Questions:

NORFOLK, Va. — Do you love Lucky Charms so much that you would like the marshmallowy cereal taste in your beer?

Smartmouth Beer is introducing its Saturday Morning Marshmallow IPA on March 2 in an all-day event that will also showcase trivia and food vendors.

Sad to note that this isn't technically an Elongated Yellow Fruit, in that "taste" isn't a variation on Lucky Charms. But ledewise, I hope we can agree that it fails the Wheaties test.