What Polls Can And Can’t Tell Us

What if, instead of focusing on what political polling is getting wrong, we adjust our expectations for what polling can and should be getting right?

That is a central premise of G. Elliott Morris’ new book “Strength in Numbers: How Polls Work and Why We Need Them.” Morris, a data journalist at the Economist, isn’t suggesting that we simply accept the shortcomings of survey research. In fact, one of the ways he thinks political polling can be improved is for pollsters to “abandon polls fielded entirely by phone, and incorporate samples drawn by other methods” like online surveys and SMS texts. Many pollsters — especially those doing private campaign polling — have been doing this for years. Many public polls are still phone-only. 

But, while he doesn’t think we should abandon polling, he does think we’ve come to expect more from them than they can — or ever have — delivered. “Over the ninety-year history of polling, we have learned public opinion surveys are less like pulse oximeters and more like a cracked mirror — a tool that reveals a portrait of the gazer that is roughly correct but with notable imperfections.” From the first presidential straw polls of military veterans in the 1820’s to the introduction of more scientific methods of sampling and surveying by pioneers like George Gallup and Elmo Roper, polling has always struggled to get a truly representative sample of the voting population.

Today, those challenges have been compounded by the fact that Americans are more accessible than ever but harder to connect with. There are technological blockers, like caller ID, and sociological blockers, like a trust deficit and fears about privacy violations. Pollsters are also vying with lots and lots of other individuals and groups for our attention. The advent of polling aggregators like FiveThirtyEight and RealClearPolitics made polling more accessible to those who don’t work in the professional political class. This ‘democratization’ of data, however, came without the deeper appreciation for polling’s inherent shortcomings. Just as most people see a weather forecast predicting a 75 percent chance of rain to mean it will most certainly rain, instead of appreciating the 25 percent chance that things stay dry, they also look at a forecast for a 70 percent chance for a candidate to win as a 100 percent chance of victory, instead of a 30 percent chance that that leading candidate will lose. 

So, where does that leave those of us who are in the business of predicting and evaluating campaigns? Or, for the consumers who’ve become disillusioned by the shortcomings of survey research in 2016 and 2020?

“The forecasters of the future will not improve on their record by endlessly massaging the polls to find their signal,” writes Morris, “but in allowing more room for us to accept their inherent noisiness fully.” 

Or, another way to think about it is to see polling as one tool in the tool box, instead of the only tool out there. 

I’ve become a big proponent of listening to as many focus groups as possible. Technological innovations like Zoom have made that more accessible than ever. Not all that long ago, if I wanted to listen into a group of voters, I’d have to pony up the money to travel to that swing state. Now, I can log on to my computer from the comfort of my own home. The goal of focus groups isn’t to figure out who is ‘winning’ but to help us sift through “the inherent noisiness.” It helps me understand how voters are prioritizing the many different issues in front of them. It also helps you appreciate how few of the things Washington and political Twitter obsess and argue about register with regular voters. It is an essential reality check. 

Polling can help us quantify who is and isn’t enthusiastic to vote. But, we can also check that against actual turnout data. John Couvillon is a political analyst based in Baton Rouge, Louisiana. Since 2010, his firm JMC Insights, has been tracking primary election turnout. In 2010 and 2014 (GOP wave years), JMC found that Republicans got 55 percent of the primary vote. In 2018, a Democratic landslide election in the House, the primary electorate was 54 percent Democratic. In other words, writes Couvillon, “partisan primary party vote does have some predictive value.”

Couvillon’s analysis of 28 primaries held between March and June of this year, “have shown signs of elevated Republican enthusiasm.” The Republican share of the primary electorate is 53 percent to 47 percent Democratic. The one silver lining for Democrats is that they are — thus far — slightly ahead of their 2010 (3 points) and 2014 (4 points) primary turnout showing. 

Even the best polls and projection models, writes Morris, are not a science “and will likely never be.” However, they remain the best way to get the most representative sample possible of our very diverse electorate. Don’t throw out polling, but supplement it with other data and information that can pick up the nuance and the cross-currents that are critical in understanding and appreciating the true complexity of our politics.