Why are Reporters So Bad at Math?

Lisa Takeuchi Cullen of Time is just the latest in a long string of print and broadcast journalists and editors to misread and misreport the results of a presidential approval poll. Just look at her opening paragraph:

A spate of good news at home and abroad has so far failed to boost how Americans feel about President Bush's job performance. Bush's approval rating slipped to 35% in a TIME poll taken this week, down from 37% in March (and 53% in early 2005). Only 33% of Americans in the survey said they approved of Bush's handling of the situation in Iraq, vs. 35% in March, and 47% in March 2005. His management of the U.S. economy lost supporters, too, as 36% approved, compared with 39% three months earlier. Bush's handling of the war on terror saw a slight gain in support, from 44% to 45%.

Now, I am the first person to scream at “the media” when they report something like a one percentage point increase in Bush’s approval rating as a “bounce,” so I want to be among the first to say that a two-point change in the opposite direction is not a “slip.”

Time has not published the raw data for each question, but I think I can say that a two-point move one way or the other in a poll of about 1,000 adults is not statistically significant. Moving to 35% from 37% is not really slipping, and a change from 44% to 45% in the approval of the President’s handling of the war on terror is not any kind of a “gain”—not even a “slight” one.

To the credit of Time and Cullen, the headline strikes the right tone (Poll: Good News Fails to Boost Bush’s Job Approval), as does the first sentence of the second paragraph (“Bush’s poll numbers remain stuck in a rut. . . .”). But, in general, though it may run against some J-school rule requiring language to be active and energetic, stories on polls need to avoid adjectives when the numbers are statistically flat (he was at “X” then, and he’s at “Y” now), and they need to provide some rudimentary explanation of how polls are analyzed and how numbers can fluctuate from week to week. Simply stating the “margin of error” (which, by the way, this story does not do) is neither academically accurate nor enough.


Now for the Real News

Of course, I, like Time, am guilty of burying the lead. From the third paragraph:

But continued pessimism about the situation in Iraq and a broad sense of unease about America's direction may be undermining Bush's popularity. In the TIME survey, 66% said the country is on the wrong track, vs. 28% who said it's going in the right direction. Those numbers have worsened since March, when the poll recorded a 60% to 34% split.

Putting aside the first sentence of this paragraph and its obfuscatory use of the word “may” (are the stated reasons born out in the poll’s findings, or just journalistic/artistic license?), I am guessing a six-point shift in this result is statistically significant, and a President ushering in a two-thirds “wrong track” number is historically so.

That 66% of those surveyed now think that Bush’s America is on the “wrong track”—even though we supposedly saw improvements in Iraq and in the domestic economy—makes me think that the public is out ahead of the journalists, headline writers, and, yes, Congress, when it comes to understanding the “facts on the ground.” That this right track/wrong track number is trending negative more significantly than the stagnant job approval numbers makes me think that this is the statistic that might be out ahead of others in measuring Americans’ increasing dissatisfaction with Bush and his rubberstamp Republican Congress and/or despair that we are only in for more of the same. . . and, thus, should be the lead in the story.


(cross-posted in a slightly different form at Daily Kos)