The Edge: Competitive Edge Blog

Public Opinion Research Insights: Yep, Phone Surveys Still Work

Print Friendly, PDF & Email

The increase in cell phone ownership during the last decade has led some to conclude that the traditional phone survey must be a defunct public opinion research tool incapable of yielding accurate data on issues that matter. As the reasoning goes, doesn’t everybody just have cell phones now (with which they screen calls)? Who answers a landline to talk to a stranger around dinner time?

Although the answer to the first question is “mostly” – meaning, yes, a lot of people are cell phone-only now — the trend’s impact on data quality has been minimal. Furthermore, the people answering telephone surveys, sometimes on landlines and other times on cell phones, are more or less just like you and me, with a few minor exceptions that statistical tools easily correct. This may sound surprising until you understand how phone surveys work.

It is true that, according to the Pew Research Center, 20 years ago 36% of people contacted responded to the survey and that’s dropped to 9% now. However, this 9% response rate has been steady for the last four years and has stabilized. Now, one might suppose, doesn’t this mean that whoever those 9% of people are they probably are biased on a whole range of issues; maybe they’re politically affiliated a certain way, or elderly, or uneducated, or skewed in some other way? Such a low response rate means that the potential scope of phone survey research is severely limited, right?

The answers here are, yes, there is just a little bias (but a lot less than you would think) and nope, it shouldn’t make hardly a whit of difference for phone polling.

Let’s start with response rate. Scholarly paper after scholarly paper has shown time and again that low response rates rarely lead to much bias when the surveys themselves are high quality. One big reason we know this is because we compare response rates from this 9% to very high response rate general surveys – like the General Social Survey – conducted in person. Such high response general surveys are a benchmark against which phone surveys can be tracked. And once those comparisons are made, there’s very little difference on the important measures of party affiliation, political ideology or religious beliefs between the two.

Now, regarding demographics, the difference between people who answer phone surveys and overall U.S. demographic profile is relatively small. As more phone survey calls are made to cell phones, the representation in surveys of young people and Hispanics, for example, has increased. As long as cell phone-only respondents are included in a sample, the bias is minimal and, regardless, statistical weighting easily corrects for minor demographic variable imbalances. Is this simple to do? No. Is it cheap and easy to get a reliable sample? No. But good research never has been simple, cheap or easy. To generate sound research conclusions, the pollster must be up to the challenges posed by lower response rates.

Telephone polls do tend to consistently over-represent one key variable – civic engagement. People who participate in phone surveys are more likely to be “joiners” and more like to be aware of what’s going on in their community. Fortunately, however, this doesn’t correlate much with partisan political attitudes or most other things pollsters are trying to determine. Where this does make slight a difference is that those predisposed to “join” are also more politically engaged. Yet even here, sound data collection and weighting methods will conform samples to the general population.

There’s a more complex story to be told here, of course, in the technical language of statistics. Our expert researchers at Competitive Edge Research & Communication speak this language. They would be happy to answer any general or technical questions you may have. Please feel free to give us a call if you’ve got an issue into which you need insight and analysis. We’ve served San Diego for 30 years, and have done it well by listening closely to our client’s needs. We offer a full range of public opinion and market research methodologies – including on-line, and we’re proud to offer a cutting-edge, professionally staffed call center as a key component of our research mix.

This entry was posted in Competitive Edge News. Bookmark the permalink.

Related Articles

  • The Edgy Interview: Jason Roe

    I first got to know Jason Roe over lunch I’d invited him to at the old-school Bully’s North in Del Mar.  I had a salad, he had some red meat dish. I pitched him on CERC’s abilities and how we help clients win.  He listened and talked about how he wanted to build Revolvis in San Diego.  We both put ourselves out there without pretense – something you don’t always get in the political game – and I found it refreshing.  The result is that, in a tough local environment, we’ve won a lot more than we’ve lost on efforts we’ve partnered on.  I expect that to continue with his new firm Roe Strategic.  Here’s Jason at his unvarnished, revealing and direct best.  – John Nienstedt Read more

  • The Edgy Interview: Rachel Laing

    Rachel has worked with Competitive Edge on a number of local projects here in San Diego. She’s also worked with John Nienstedt on the not-so-pleasant divorce of the local Alzehimer’s Association chapter from the Chicago-based national organization. Aside from being among San Diego’s most active and witty twitterati, Rachel notably stars (along with Jason Roe of Roe Strategic) in the Voice of San Diego banner ads you keep complementing us on. ~JN Read more

  • The Edgy Interview: Jerry Kern

    n Competitive Edge’s three decades we’ve had many successes that make us proud. We love helping clients win in whatever arena they find themselves. One of our most gratifying efforts was helping Oceanside City Councilman Jerry Kern beat back an attempted recall. Not only did we assist his campaign in crafting the message that would take him to victory and vindication, our poll was spot on. Yeah, we know that frequently happens; it’s what we’re paid to do. But in Kern’s case we conducted the poll five months before the election. ~ JN Read more

Leave a Reply

Your email address will not be published. Required fields are marked *

Archives

Featured Research Download

Are you targeting the "persuadables" of your campaign? Download our seven step process to winning an election.

Download