Cell Phone Polling Gets A Big Push

I've written in the past about the problems associated with polling young voters and how these are exacerbated by our habits of not owning a land-line. This is a growing problem for pollsters as more and more people abandon their landline and go cell-phone only.

This week, Gallup took a huge step forward in addressing these problems when they announced plans to begin including cell phones numbers in their surveys.

Still, Gallup has been studying and investigating the implications of cell phone only households for well over a year now. And, as of Jan. 1, 2008, Gallup has made the decision to include cell phone interviewing as part of the sample used for its general population studies.

This is a complex and costly modification in methodology. Our statisticians and methodologists have spent a great deal of time reviewing the procedures and implications of the change. Essentially, in addition to sampling from the traditional database of all landline telephone exchanges, Gallup now also adds in sampling from a new database of all cell phone telephone exchanges in the country. We screen for those individuals using cell phones who report not having a landline, and then interview a random sample thereof. We then weigh into the sample a proportionate percentage of these interviews conducted via cell phone.

We’re continually monitoring the methodology of our interviewing, and revise on a regular basis as appropriate. We'll be analyzing the implications of this shift in methods particularly carefully.

Mark Blumenthal has more on the consequences and significance of this change in methodology:

For now, at least, this change is not likely to produce dramatic differences in the results. The ongoing cell phone surveys conducted by the Pew Research Center have shown that the missing cell-phone-only population rarely makes a difference of more than a point or two. But that point or two may sometimes make a difference, especially in a close race. Consider last week's Gallup poll in New Hampshire. USA Today polling editor Jim Norman let us know, via email, that they included a cell-phone sample on that survey:

[I]t added a point to Obama's total and took one away from Clinton. In other words, without the cell-phone-only respondents, Obama's lead among likely voters was 11, not 13.

The bigger significance in this change is symbolic. Gallup is the granddaddy of all polling firms. Their polling "time series" goes back to the 1930s. As such, they are typically the most cautious about changes in methodology, so their move to regular cell-phone sampling is likely to have a big ripple effect on the polling industry. At very least, this most closely watched poll will provide a regular source of data on the potential impact of the cell-phone-only households that will be missing from other surveys.

If Gallup continues to adopt this as their standard, other polling firms will follow, and the "added costs" of collecting a large enough sample of younger voters will instead become the normal costs of doing business. Maybe then we'll see more polling that can reliably break out the opinions of young voters, which currently are subject to wide margins of error or ignored altogether.

If Gallup begins to show statistically different findings from other polls, you can be sure that this will happen.