The Edge: Competitive Edge Blog

Public Opinion Research Insights: Yep, Phone Surveys Still Work

Print Friendly, PDF & Email

The increase in cell phone ownership during the last decade has led some to conclude that the traditional phone survey must be a defunct public opinion research tool incapable of yielding accurate data on issues that matter. As the reasoning goes, doesn’t everybody just have cell phones now (with which they screen calls)? Who answers a landline to talk to a stranger around dinner time?

Although the answer to the first question is “mostly” – meaning, yes, a lot of people are cell phone-only now — the trend’s impact on data quality has been minimal. Furthermore, the people answering telephone surveys, sometimes on landlines and other times on cell phones, are more or less just like you and me, with a few minor exceptions that statistical tools easily correct. This may sound surprising until you understand how phone surveys work.

It is true that, according to the Pew Research Center, 20 years ago 36% of people contacted responded to the survey and that’s dropped to 9% now. However, this 9% response rate has been steady for the last four years and has stabilized. Now, one might suppose, doesn’t this mean that whoever those 9% of people are they probably are biased on a whole range of issues; maybe they’re politically affiliated a certain way, or elderly, or uneducated, or skewed in some other way? Such a low response rate means that the potential scope of phone survey research is severely limited, right?

The answers here are, yes, there is just a little bias (but a lot less than you would think) and nope, it shouldn’t make hardly a whit of difference for phone polling.

Let’s start with response rate. Scholarly paper after scholarly paper has shown time and again that low response rates rarely lead to much bias when the surveys themselves are high quality. One big reason we know this is because we compare response rates from this 9% to very high response rate general surveys – like the General Social Survey – conducted in person. Such high response general surveys are a benchmark against which phone surveys can be tracked. And once those comparisons are made, there’s very little difference on the important measures of party affiliation, political ideology or religious beliefs between the two.

Now, regarding demographics, the difference between people who answer phone surveys and overall U.S. demographic profile is relatively small. As more phone survey calls are made to cell phones, the representation in surveys of young people and Hispanics, for example, has increased. As long as cell phone-only respondents are included in a sample, the bias is minimal and, regardless, statistical weighting easily corrects for minor demographic variable imbalances. Is this simple to do? No. Is it cheap and easy to get a reliable sample? No. But good research never has been simple, cheap or easy. To generate sound research conclusions, the pollster must be up to the challenges posed by lower response rates.

Telephone polls do tend to consistently over-represent one key variable – civic engagement. People who participate in phone surveys are more likely to be “joiners” and more like to be aware of what’s going on in their community. Fortunately, however, this doesn’t correlate much with partisan political attitudes or most other things pollsters are trying to determine. Where this does make slight a difference is that those predisposed to “join” are also more politically engaged. Yet even here, sound data collection and weighting methods will conform samples to the general population.

There’s a more complex story to be told here, of course, in the technical language of statistics. Our expert researchers at Competitive Edge Research & Communication speak this language. They would be happy to answer any general or technical questions you may have. Please feel free to give us a call if you’ve got an issue into which you need insight and analysis. We’ve served San Diego for 30 years, and have done it well by listening closely to our client’s needs. We offer a full range of public opinion and market research methodologies – including on-line, and we’re proud to offer a cutting-edge, professionally staffed call center as a key component of our research mix.

This entry was posted in Competitive Edge News. Bookmark the permalink.

Related Articles

  • The Edgy Interview: John Hoy

    Working from his Del Cerro lair, John Hoy is a San Diego political consultant who takes a thoughtfully methodical approach to winning elections. There is no bluster. There is no BS. There is no flash of bravado. John just thinks through every situation he’s faced with and guides his clients to make the best decision given the circumstances. I’ve had the pleasure of working with him on big wins like Lorie Zapf for San Diego City Council and Jan Goldsmith for San Diego City Attorney. John’s also the master of the signature-gathering process, recently helping Soccer City’s backers submit and qualify petition signatures in record time. As he mentions, his wife Cheri’s pretty great too. ~JN Read more

  • The Edgy Interview: Karolyn Dorsee

    Everybody loves Karolyn, because when you’re a great fundraiser, you’re a people person and you have fun along the way, people love you. I call Karolyn my political Godmother for the start she gave me. She’s had a hand in starting hundreds of political careers here in San Diego. And, 33 years after meeting Karolyn, through all the ups and downs politics can throw at her, she’s still enjoying her work and working for people she believes in. Here’s how she began the interview… ~JN Read more

  • The Edgy Interview: John Kern

    The year was 1987. Reagan was president, Ron Roberts was just elected to the San Diego City Council and Competitive Edge was a newborn baby. Having no idea what to do to keep our clients happy and build a business, we went to lunch with news reporter and political consultant.... READ MORE ►

Leave a Reply

Your email address will not be published. Required fields are marked *


Featured Research Download

Are you targeting the "persuadables" of your campaign? Download our seven step process to winning an election.