Every time a new 2024 poll is released — the ones that have consistently shown former President Donald Trump performing well against President Joe Biden — high-profile Democrats will, seemingly in unison, list everything that is wrong with the state of polling.
“I think there should be humility, especially with turnout and the election,” Rep. Ro Khanna, a prominent Biden surrogate, told NOTUS. “That whole polling profession needs humility in how accurate these things are.”
The criticisms range from picking apart the published methodology to dismissing poll results as nothing more than a collection of responses from people so old or weird that they still have landline phones — and still answer them when strangers call.
The question of how to look at polling remains open after the polling industry conducted a very public autopsy of the 2016 public polling results — and largely cleared itself of any significant errors while acknowledging a big loss in public trust. Pollsters say that a lot of denialism comes from a fundamental misreading of what polls should do.
For example, Democrats complain there’s too much coverage of a head-to-head poll in March as a predictor of an election result in November. And pollsters say, well, yes.
“If you think of the polls today as a forecast of how the election is going to turn out, that’s just wrong,” said Charles Franklin, director of the Marquette Law School poll. “That’s just a fact, but it’s been a fact in every election that there’s ever been.”
Sen. Chris Coons grumbled to NOTUS when asked about polling.
“Voting is far more predictive than polls,” he said. “In the special election in Long Island, half-dozen other special elections going back to the midterms, Democrats have overperformed over and over and I think that’s what matters more.”
“Polling to me is just a snapshot in time, and I know what these polls are saying, but also know what these polls said last time around,” agreed Rep. Jim Clyburn, another close Biden ally. “According to every poll, Joe Biden shouldn’t be president, but he is.”
Pollsters again say, sure, why not? We’ll all find out in November. There has long been an uneasy relationship between the scientists who conduct public polling and the media that turns it into headlines and narratives. The maxims are known to everyone in the world of politics: Don’t base a broad opinion on one poll, don’t dig too far into the crosstabs of a single poll for wacky stories, carefully consider the exact questions a poll asks before making big conclusions about the answers it reports back and always check who specifically was asked the questions in the first place.
But inevitably, media organizations spend a fortune on exclusive polling to create a lot of coverage. Jumpy partisans following a campaign second by second can’t help but trumpet when a fresh public poll shows something good for their side or bad for the other.
Sign up for the latest from NOTUS.
The people who conduct public polls will repeat the age-old cautions against these bad habits to anyone who asks. But they are not happy about the recent Democratic attacks on their industry as a whole.
“Certainly, the use of polls is just a selective tool for partisan campaigning on both sides of the aisle,” said Jennifer Benz, deputy director of the Associated Press-NORC Center for Public Affairs Research, based at the University of Chicago. Her goal, as a scientist, “is to go into this making sure that we are asking questions that we feel are capturing the true underlying issues at heart.”
Pollsters say they are doing a lot of things differently than they did in 2016.
Franklin has made big changes to the way the Marquette poll gathers a universe of prospective respondents in the years since 2016 when the final poll of that cycle in Wisconsin showed Hillary Clinton winning by six points before she lost by less than one on Election Day. In that cycle, Marquette segmented the state into five regions, sourcing respondents so they had a good sample from each. Now, there are 90 segments to try to reach people in the most Democratic and most Republican parts of the state with “a lot more granularity,” Franklin said.
About a quarter of their respondents now are from a recruited opinion panel. Unlike before, Marquette’s also reaching out to potential respondents via email and text, calling a couple of days later if they don’t hear back. Franklin said around 75% of the respondents are coming in online.
This transformation is not unique. Public pollsters are striving to find new ways to find respondents, finding those who reflect the population and putting those new methods into their polling data. Many of these schemes will get their first serious test in November when Trump — whose support has been hard to pin down accurately — is on the ballot.
Benz said pollsters across the industry have been adjusting weighting to combat the disproportionately low response rates from Trump voters. They’ve started asking survey respondents if they voted in 2020, and, if so, for whom. This way, they can try to adjust “by correcting for the past electorate.”
Pollsters have noted that most of the public polling out there shows similar results, which makes them feel confident they’re doing something right.
“If you believe the polling is pointing to some real weaknesses of Joe Biden right now, the campaign surely needs to recognize those issues and address them in the eight months remaining,” Franklin said, “rather than to deny the validity of the results and campaign as if it isn’t true.”
The Biden campaign has repeatedly said it’s built around running a close race and that close public polling does not surprise them. Even as Biden supporters online loudly dismiss polling as broken, veterans of Democratic politics say public polling can distract from the task at hand.
“I have a good gut. Joe Biden is doing well out there. I’m not somebody that listens to polls, that lets them guide me,” said Rep. Debbie Dingell. “My pollster always wants to know if I’m gonna bad-mouth him. Fred will text me and say, ‘Thank you for not bad-mouthing me.’ So, you know, it may give you some thoughts about what the key issues are, but I’m an old-fashioned politician, I believe what’s in my gut.”
—
Evan McMorris-Santoro is a reporter at NOTUS. Nuha Dolby is a NOTUS reporter and an Allbritton Journalism Institute fellow.