Many political observers groaned at the candidacy of disgraced former Rep. Anthony Weiner in the New York City mayoral race to replace outgoing Mayor Michael Bloomberg.
Of all the mayoral jobs in the nation, it’s the most prestigious. So naturally, the media’s pushing for the latest research in the race to see if this infamous “selfie” Tweeting character ends up running the Big Apple.
However, recent conflicting survey results tripped up the media who either did not dig deeply enough out of sheer laziness or due to an information gap in understanding the nature of polling.
One poll conducted by the Wall Street Journal/ NBC New York/ Marist showed Weiner snagging 25 percent to City Council speaker Christine Quinn’s 20 percent, followed by City Comptroller William C. Thompson, Jr. with 13 percent.
The next day, Quinnipiac University released a poll showing a virtual tie: Quinn at 19 percent, Weiner at 17 and Thompson at 16 percent. One minute Weiner is up; the next he’s down, 8 ticks in this case.
Try to spot the logical and data dead spots in this excerpt from the New York Time’s “Explaining 2 Polls with Conflicting Results:”
“…The timing of the two polls may also have affected the results.
The Wall Street Journal/NBC New York/Marist poll was conducted June 17 to 21, and the Quinnipiac Poll was done June 19 to 25. The United Federation of Teachers, which represents the city’s public-school teachers, endorsed Mr. Thompson on June 19, giving him extra media exposure during the full period when the Quinnipiac poll was conducted. Compared with polls conducted in May, Mr. Thompson gained two percentage points — a statistically insignificant change — in the Marist poll, but he jumped by six percentage points in the Quinnipiac poll from a previous Quinnipiac poll.”
The reporter theorizes – without explicitly saying this — that the union endorsement gave Thompson a boost which showed up in the Quinnipiac poll. Whoa! That’s attributing a lot to one endorsement.
Aside from the fact that a 3% difference between Thompson’s Wall Street Journal and Quinnipiac numbers is almost certainly not statistically significant, the problem with the reporter’s theory is, well, you don’t need one. The Wall Street Journal researchers easily could have broken down their results into pre- and post-endorsement time periods for the reporter. Applying basic statistical tests would then allow real conclusions to be drawn. As presented, the story doesn’t really “explain” the conflicting polls as much as it assumes, leaving the public to wonder about the validity of the research.
Pollsters should seize every opportunity they get to put theories to the test. As was the case in this instance, the data is often sitting right there in their datasets. All that’s needed is some smart analysis.