Are Americans Tuning Out The 2024 Election?
38:52 Share

You're a podcast listener, and this is a podcast ad. Reach great listeners like yourself with podcast advertising from Lipson Ads. Choose from hundreds of top podcasts offering host endorsements, or run a reproduced ad like this one across thousands of shows to reach your target audience with Lipson Ads. Go to LipsonAds.com now. That's L-I-B-S-Y-N-Ads.com. What was the most scandalous thing that happened at APOR last week?

There was one gentleman. He didn't interrupt because it was during the Q&A session, but he used the Q&A session to kind of get on his hobby horse about margin of error. And nobody talks about margin of error anymore. And it wasn't really germane to the conversation. And so it was a little... And the moderator had to kind of politely... Whoa! Oh my God. That sounds scandalous. It was about as big of a deal as Watergate, to be honest. ♪

Hello and welcome to the FiveThirtyEight Politics Podcast. I'm Galen Druk. It's not a secret that we are headed towards a presidential election that many Americans said they didn't want. Somewhere in the range of 20% of Americans have an unfavorable view of both former President Trump and President Biden. And if ratings are any indication, a lot of folks seem to be tuning campaign news out.

So what does this portend for turnout? We've been in an era of high turnout elections since 2016, but will fatigue or even disdain keep people home this fall? It's an important question for our democracy. It's also an important question for polling. Figuring out who will actually show up can be tricky,

And so far, likely voter models show a more optimistic picture for Biden than polls of registered voters overall. So today we're going to dig into it. We are also going to talk about, yes, the debates.

Last week, President Biden and former President Donald Trump agreed to two dates, one in June, one in September. That June date will be the earliest presidential debate ever by nearly three months. So is that an attempt to shift the dynamics that I just mentioned and get people to actually pay attention to the race? Here with me to discuss it all are my colleague, senior elections analyst Nathaniel Rakich. Hello. Welcome, Nathaniel.

Hey, Galen, how are you? You know, doing pretty well. Also here with us is senior elections analyst Jeffrey Skelly. Hey, Jeff. Good morning, Galen. And I should also say that you were both just recently back from the annual APOR conference, which is the American Association of Public Opinion Research. A little later on, I'm going to ask you about the best and worst uses of polling at that conference. But let's begin with the debates for now before we get quite that nerdy. So as I mentioned earlier,

Biden and Trump agreed to two debates, one on June 27th, hosted by CNN, and one on September 10th, hosted by ABC. And these two debates are notable for a couple of reasons. One, the candidates jointly bypassed the Commission on Presidential Debates, a nonpartisan group that has planned the debates since 1988.

I mentioned that the June debate is unprecedentedly early before both candidates have actually even been formally nominated. And then according to stipulations from the Biden campaign, both debates are going to be held without an audience. So first of all, Nathaniel, my question to you, is it time to eat crow? When we played buy, sell, hold a couple of weeks ago, the

The odds that a debate would happen were in the 70s on Polly Market, and you scoffed at that high of an odd that debates would happen. You said you would be selling in the 70s. It now looks like the debates will happen, although, you know, this isn't really over, I guess, until the fat lady sings. But what do you have to say for yourself?

Yeah, in fact, it's worse than that, Galen. On, I think it was Wednesday morning when all this went down and there were first rumblings about it, I tweeted basically like, the debates are not going to happen, folks. This is all just a kabuki dance. And then like literally three hours later, they had agreed to a debate. So yeah, I was wrong.

I overestimated the degree of norm breaking that would happen between these two candidates in this election, although, as I'm sure we'll discuss, we still had some norm breaking in terms of we're, you know, I mean, yeah, we're outside the normal calendar range. They bypassed the commission on presidential debates. But yes, it seems like the debates will happen now. So I'm happy to be wrong.

Well, OK, there's still some stuff going on here, right? Which is, for example, over the weekend, Donald Trump said that he wanted Biden to take a drug test before the debate, for example. I mean, the likelihood that these debates happen is not all that high.

100%. It's probably greater now than 70%. But how are, Jeffrey, you thinking about the likelihood that we go for it? As we just started taping, you were saying that you were looking up whether or not RFK Jr. was going to qualify for the CNN debate. And my response to that was, well, if RFK Jr. qualifies, will the other two agree to do the debate? Yeah.

I mean, that's a huge question. And it looks like, to be clear, we don't actually have, or at least I have not yet seen, and I've been reaching out about this, the start and end dates of the qualification poll period for CNN. We know which pollsters they will count surveys from.

But we don't know what the start point for what would count in the end point would be just ahead of the debate. But based on just like a rough look at things like right now, if it was from sometime in March to now, there would be two polls, a CNN, SSRS poll and a Quinnipiac University poll.

That would meet the qualification standard of 15% for RFK Jr. So that would be two of the, I believe, four that he would need. Now, you also need to be on the ballot in enough states to equal at least 270 electoral votes.

Now, that's going to be an interesting question mark because of the timeline for even getting confirmation of that information from various states differs a fair bit. That's going to be perhaps the murkiest part of this. And so kind of a huge wildcard as well. But you have to think if if.

you're going to have all this kind of back and forth between the two campaigns, maybe sniping at each other about different things that they want. Trump saying things like Biden needs to take a drug test. And then if you throw in RFK actually qualifying, at least based on the polls, and then maybe getting some sort of allowance that he's like on track to qualify, because CNN, I'm sure, would love to have RFK Jr. in the debate for the eyeballs. Does that end up

making Biden go, eh, maybe I don't want to participate in this. So we'll keep an eye on all of this. But for the time being, you know, Biden was the one who proposed this schedule of one debate in June and one in September, only doing two debates, doing them outside of the commission. Why did he propose what he proposed from a tactical perspective?

So I think the reason why it is smart-ish of Biden to debate early is we talked on this podcast. What was it a week ago, two weeks ago? I don't know. What is time about how Biden?

Biden needs to consolidate support among his base. And a thing that could help him do that is to basically point at Donald Trump and be like, remember this guy? He has a very real chance of being president. And this is like a live issue right now. And the kind of the closer we get to the quote unquote general election campaign and that

realization for folks. I think the sooner Biden's support will come home, assuming that it does, that is still also an open issue. But we'll get kind of an indication of whether that will end up saving Biden or at least kind of making him close the gap with Trump in the polls. And I think that having a general election debate in June is a good way to do that earlier than it might otherwise happen, which would be like, I don't know, September or whatever.

I also think that having two debates early, the September 10th debate, that is still quite early for a presidential debate. You're going to have almost two full months of campaigning between that last debate and Election Day. I think that also means that if Biden doesn't perform well in the debates, which, of course, he may not do, there are still, I think, very real concerns about his age.

probably even within the Biden campaign itself. It puts a lot of distance between a potentially bad Biden performance and the actual voting. And as we know, during September and October of an election year, a bazillion things happen. The news cycle moves so quickly and people probably will not remember the debates by the time they go to the polls.

Jeffrey, why did Trump accept, given that this schedule seems quite advantageous, or at least more of a tactical move on Biden's part? It's a great question. I am not 100% sure why he accepted. To me... Oh, come on. You don't have, like, a phone to the former president? No. Jeffrey. No, no, well...

Basically, my attitude was that Biden thinks he's losing. So, of course, he wanted to debate. I mean, it's not like an absolute truism in politics, but the person looking for debates is usually the person who feels like they need something out of the debates.

At least to some extent, that's kind of the feeling I have here. And to Nathaniel's point, I think Biden is looking for something to shake things up, to remind people that he's competent together, that, yeah, I'm old, but I've still got a fastball or something. And so a June debate could serve as that. It could also serve as a, well, if it goes really badly and all the different crazy scenarios we've discussed previously on this podcast about people replacing Joe Biden,

Yeah, you do. I wasn't going to say it. I'm glad you did, Jeffrey. You leave the door at least slightly ajar for those kinds of things. Not that I think that's at all likely. Contested convention. That's very unlikely, but I'm just saying it leaves the door ajar. You get some of these questions out of the way earlier in a sense.

Trump accepting is curious. At the same time, we're looking at polls where we have Biden and Trump in the low 40s together. It's not like Trump's got some sort of lock on this election. He may have the attitude that

Well, I can just get up there and like kick Joe Biden's ass and make him look like the old guy is and like the Democrats will be even more in disarray or whatever you want to say about it. Right. So and to be clear, we know that further stipulations could come along here that might make Trump accepting maybe in the end he doesn't even want to debate. And we just don't know how things are going to pan out yet. And he'll find some way out of it. So I think it's hard to say exactly. I think the Biden situation is much more easy to read.

Yeah, I think my read on the situation is Biden had to say he wanted to debate, maybe actually does want to debate for the reasons that Jeffrey said, although I think there is still a high risk for him, given his, shall we say, sluggishness. High risk, high reward, you know, kind of. Right. But I think that maybe Trump's calculus was.

We're not going to participate the regular way here. Biden is making an offer of two debates. I do want to debate him because I think the debate will be advantageous for me. Therefore, I'm going to take what's on the table instead of rejecting it and then giving Biden an excuse to be like, OK, well, we tried no debates then, which I think would be a worse outcome for Trump. So that's my read.

Yeah, that's an interesting point, Nathaniel. I mean, you said that Biden had to at least say that he wanted to debate part of his identity and his pitch for becoming president in the first place was a return to normalcy. And, you know, debating during a presidential campaign is a norm at this point.

But he did go outside the norms to some extent, which is bypassing the Commission on Presidential Debates. You know, the calendar is a bit weird, saying he didn't want an audience, etc. I mean, in the grand scheme of things, I don't think you would compare this norm breaking to some other norm breaking that we've seen in the Trump era. But it is still not just the usual, all right, we're going to do the three commission debates, and they're going to be at this university and that university, and we're going to have the audience and all that stuff.

Why did he feel, I guess, empowered to break that norm? So I actually think it was Republicans that broke this norm because in 2022, the RNC voted to withdraw from the commission for presidential debates. So it was already basically not going to happen. The Republicans weren't going to participate. So I think that's probably why Biden went outside of it. You don't think Trump would have done...

done the debates put on by the commission for presidential debates if Biden had agreed to those three? I don't know. The RNC had, yeah, I mean, the RNC, they were in a big public fight with the commission on this. So yeah, that'd be hard to back down from.

We've touched on this a little bit, but let's talk about impact, right? Looking at studies of presidential debates, both in the U.S. and abroad, suggests that the best predictor of performance post-debate is standing in the polls pre-debate. So not actually debate performance. Is there any reason to expect this year to be different?

So what's interesting to think about with the debates is that they are usually in the fall. September 21st was the earliest in 1980, but that was actually a debate between Ronald Reagan and John Anderson, and Jimmy Carter refused to participate. Basically, the earliest head-to-head debate involving a Democrat and Republican was September 23rd, it looks like, in 1976.

So this is clearly a lot earlier in the campaign. And so on the one hand, people aren't as tuned in. But on the other hand, people – by the fall, a lot of people have already sort of made their decision about what they're going to do to vote. And so I kind of wonder if you think about another important event that happens over the course of the summer, at least historically, is the convention.

And we talk about like the convention bounce, the extent of convention bounce in this day and age is debatable. It's not nearly what it once was. And I think that has a lot to do with how polarized the country is. So you don't have people responding in polls saying they're going to support the caucus and then they end up voting for Bush or something. Or you have like an effect in terms of who is answering polls that maybe was influencing that. But nonetheless –

I do wonder about the potential for something akin to a convention bounce out of a debate that is this early. So that's I think those things are still more. Yeah. So that to me is maybe the interesting wrinkle here with having such an early debate.

But I think, again, much like a convention bounce, it would be fleeting. And going back to the point that I said earlier about there probably being a million things that will happen in the news cycle between the September 10th debate and Election Day, I don't ultimately think they will make a difference in the end, no.

And for what it's worth, even if there isn't much of a bounce or something, if Biden's sort of strategic aim here is to shore up his – shore up concerned voices in his party and maybe pull in some people who aren't happy with him at the moment by reminding them of the stakes and he's facing Trump.

If anything, it's just like purely like a campaign thing. And maybe it doesn't show up that much in the polls, but it's just like reassuring his party in a way so that they can move forward with maybe less of the concern that you've heard, less stories about people being like, I don't know if Biden can still do this kind of kind of narrative that we've heard over and over again.

Another interesting thing kind of coming out of the shift away from the commission, right, is that like, what are the ratings going to be like? Like CNN is going to be carrying this debate exclusively in June. And like, not everybody has CNN. In fact, 53% of American households have access to CNN. Right. And they might make it streamable though, you know? Sure. But like,

It's going to be June. People are going to be on vacation. People aren't going to be tuned in. Like, I think it will bring in some people who weren't tuned in previously. They'll tune in because of the debate. But also there are going to be a lot of people who are like, I'm just not paying attention until October. Yeah, that's that's when you get into the whole like it's it's about like the post debate coverage and analysis of it and like what were key events in the debate and how does that get filtered through the media in like the day or two following the debate? And that will be important as well.

All right, we will see how this all turns out. Let's move on and talk about turnout. But first, a break. Today's podcast is brought to you by GiveWell. You're a details person. You want to understand how things really work. So when you're giving to charity, you should look at GiveWell, an independent resource for rigorous, transparent research about great giving opportunities whose website will leave even the most detail-oriented reader stunned.

GiveWell has now spent over 17 years researching charitable organizations and only directs funding to a few of the highest impact opportunities they've found. Over 100,000 donors have used GiveWell to donate more than 2 billion dollars.

Rigorous evidence suggests that these donations will save over 200,000 lives and improve the lives of millions more. GiveWell wants as many donors as possible to make informed decisions about high-impact giving. You can find all their research and recommendations on their site for free. And you can make tax-deductible donations to their recommended funds or charities. And GiveWell doesn't take a cut.

Today's podcast is brought to you by Shopify. Ready to make the smartest choice for your business? Say hello to Shopify, the global commerce platform that makes selling a breeze.

Whether you're starting your online shop, opening your first physical store, or hitting a million orders, Shopify is your growth partner. Sell everywhere with Shopify's all-in-one e-commerce platform and in-person POS system. Turn browsers into buyers with Shopify's best converting checkout, 36% better than other platforms. Effortlessly sell more with Shopify Magic, your AI-powered all-star.

Did you know Shopify powers 10% of all e-commerce in the U.S. and supports global brands like Allbirds, Rothy's, and Brooklinen? Join millions of successful entrepreneurs across 175 countries, backed by Shopify's extensive support and help resources.

Because businesses that grow, grow with Shopify. Start your success story today. Sign up for a $1 per month trial period at shopify.com slash 538. That's the numbers, not the letters. Shopify.com slash 538.

You're a podcast listener, and this is a podcast ad. Reach great listeners like yourself with podcast advertising from Lipson Ads. Choose from hundreds of top podcasts offering host endorsements, or run a reproduced ad like this one across thousands of shows to reach your target audience with Lipson Ads. Go to LipsonAds.com now. That's L-I-B-S-Y-N-Ads.com.

The question of turnout is an interesting one this cycle. On one hand, it seems like Americans are somewhat down on their options or even bored by them. But on the other hand, the 2020 election saw record turnout and midterms and off-year elections have also attracted high participation in the Trump era.

We already have this in mind as a topic. And then when I was talking to Nathaniel about the hot topics at APOR, the Conference on American Public Opinion Research, he said that pollsters were wringing their hands a bit about modeling turnout this cycle. So let's talk about it. First off, Nathaniel, what were folks at APOR worried about when it comes to turnout?

So this is based on one particular panel that I thought was one of the most interesting things I saw between campaign pollsters. And, you know, basically, you know, there were a couple of questions about what keeps them up at night about 2024. And, you know, obviously, there's a lot of uncertainty about how the polls are going to do after 2020 and 2016. But I think that, yeah, like modeling turnout and who is a likely voter was a big part of that just because, well, first of all, we have this.

issue where it seems like the more stronger propensity voters are more democratic now and it's basically like okay so like how

high turnout might actually benefit Republicans this year. But like, how high is the turnout going to be? Is it really going to be like sweeping in all these really disaffected voters or for the reasons you said, Galen, or people who are disaffected just going to stay home? So that was one of many things I think that pollsters are a little worried about. And just for the sake of clarification here, I

different pollsters model likely voter turnout differently. But one of the most straightforward ways is you just ask folks, you know, do you plan on voting? How likely are you to vote? And you can include people who just say that they're absolutely certain to vote. You can include people who say they're absolutely certain or very likely to vote. And so there are decisions that pollsters have to make that will ultimately affect the results. And as Nathaniel alluded to, even when we were talking about the

the New York Times-Siena College poll last week, we were talking about registered voters. But if you look at the likely voters that are reported also in those results, things do look a little bit better for Biden. Were pollsters saying that they were sort of trying anything or doing anything in particular this cycle to try to help with the turnout quandary?

Yeah, I don't recall anything specific about turnout, but in terms of trying to make sure they had sort of a good sample from the start, there was obviously a lot of talk about the modes that you reached out to people with in order to make sure that you're not missing parts of the electorate when you are sampling. Because look, pollsters take a sample and then they weight it to various aspects of the population and their expectations.

But if the initial sample tends to miss a certain part of the population more than others, you can't necessarily wait your way out of that. But making sure that their initial sample is at least like mostly getting most parts of the electorate so that the actual results that they turn out after they wait them are closer to what the actual mood is. And that's a huge – of course, a huge challenge and a thing that's been a challenge forever. But particularly in recent years, it seems to be a huge challenge.

Okay, now for the real debate at hand. Do you all think that this is going to be a high turnout election, even a record-breaking turnout election?

It will not be record-breaking. 2020 was absolutely bananas. The interest in that election was extremely high. You also had a bunch of states making it easier to vote because of the pandemic and voting by mail and stuff like that. And that is hopefully not going to be replicated again this year. So I do not believe that it will be higher than 2020, but I do think that it will be

high by overall historical standards. I think actually the really interesting question, something that you and I were kind of debating before the podcast, Galen, was like, will it be higher or lower than like the 2016, 2012, even 2008, which is not an unbreakable

metric, I think. 2008 obviously had high turnout because people were excited about Barack Obama. So I think that's the interesting question. But even all of those years, even 2012 and 2016, which we don't think of as particularly like, you know, golden eras for voter participation, were still higher than a lot of the elections that preceded them.

Well, OK, if you're just saying that, like all elections that we've had for the past 20 years are high turnout elections and it will be in line with the past 20 years, you're not saying very much. I mean, not not to like poke the bear, but right. Are we talking about.

I think you've made the argument to me that we should expect very high turnout because people ultimately are very interested, feel like the stakes are high, that we should use 2020 as more of a benchmark. No, not that last part. Not the 2020 benchmark part. Not that last part. But yes. I have argued that we should maybe think of 2016 more as a benchmark because we have two very unpopular candidates that people are just a little turned off by. And that's exactly what happened in 2016. See, my instinct is that it will be...

It will not be as high as 2020 because it's just not very smart to go out and say that something's going to match the record setting like circumstances of something. But at the same time, I can easily imagine it being somewhat higher than 2016. So just to put some numbers to this, 66% of the voting eligible population. So that's like anyone who was 18 or older, who was a citizen who was eligible to vote and couldn't theory register and vote. 66%.

showed up and voted in the 2020 presidential election, which is basically a modern record. And in 2016, you're talking about 60%.

So to me, I sort of would maybe fall somewhere between those two numbers because – Were you the middle child, Jeffrey? No, no. I'm the oldest. No, you're not. I'm the oldest. So I'm superior in every way. So we do have some polling that has shown that people have given a lot of thought to the election process.

Gallup recently released their polling on how much thought people have given to the election, and actually 71% of Americans said they have given quite a lot of thought. This is like their spring-summer reading of things, and this was roughly on par to 2008 and 2020 and higher than 2000, 2004, and 2012.

This has tended to correlate somewhat to turnout over the years. It's not a given, but I think the other piece of the puzzle is to look at the 2022 midterm and see that, okay, look, 2018 was like the highest midterm turnout

Essentially ever. I mean you have to go back to like the 19th century practically or the early 20th century to find something higher and like women couldn't vote. So it's like a different ballgame. In 2018, 50% of the voting eligible population showed up for the midterm, which was absurd, like unparalleled in modern times.

46% of the voting eligible population showed up in the 2022 midterms. So to me, that says people are still highly engaged to a degree that is different from what came before. So to me, like sort of taking those data points, like

I've had it in my head that turnout will be pretty high by historical standards in 2024. Will it be 2020? I don't think it'll be quite that, but I still think people are going to be very engaged. And another wrinkle here is having a high profile third party candidate. It turns out more low propensity voters to have more candidates on the ballot. So like the pool of potential people who could show up will expand. You know, at the end of the day, I don't think RFK is going to do that great, but I'm just saying, you know,

He's going to be in the news, and I do think that that will have some impact on turnout. Yeah, that's an interesting point. And I'll say that Nathaniel, as much as...

some of the provisions that were in place for the 2020 election were rolled back with the ending of the pandemic, it did create some habits that are likely to endure, which help with higher turnout. I mean, voting by mail is just easier. And so in states that have, for example, universal vote by mail, you see pretty high turnout, even in

if they're in states like Nevada, which has other demographic markers that would suggest a lower turnout state. Right. And we have more state and some states did permanently switch to vote by mail after the pandemic. So not all of those things have been rolled back. That's a good point. But here's another question I have. I mean, you look at the ratings data, both for TV, also for, you know, political sites, and also probably people just have a sense from their own personal lives that, you

People don't really want to talk about politics. People aren't really engaged with the ins and outs of this election. You know, just one example from actual polling and not anecdotes is a majority of Americans say they're paying either no attention or just a little attention to Trump's criminal trial right now. If this were happening in 2016 or 2020, it's hard for me to imagine that being the case. So what do we make of that? How does that fit into this turnout conversation?

I think I'd say two things. One is that all of the ratings data about like, you know, cable news ratings or fall have fallen off or whatever that I've seen is compared to 2020, which again, I don't think anybody is arguing that turnout will be as high or higher than in 2020. And then the second thing is just like, people don't have to be like thrilled with their choices or like interested to, to turn out a vote. And cause in fact, like hatred of the other side is a big motivator as we've talked about on this podcast. So like,

Yeah, they don't need to be thrilled, but they need to be like anti-thrilled to go out and vote against Biden or against Trump. And that'll do the job. So it's almost like voters don't need any more information. Like they have all the information they need. In many ways, they've made up their minds. And even if they're not tuning in, they're going to vote.

Yeah, I just like traditionally like this has been framed as like a question of quote unquote enthusiasm to vote. And like, I just don't think that enthusiasm is the right word in a scenario where people most people don't like at least one of the two candidates, if not both. But it doesn't mean that there aren't reasons other than being enthusiastic that they might vote. I think the other thing and something I wrote about recently is that people perceive very large differences between the two parties now.

Yeah.

And so it's no wonder that they begin to hold more negative attitudes toward the opposition because they find them – they're sort of even farther apart from them in a way. And so I think that's another thing that's going to spur people to turn out is at the end of the day, it's not Tweedledee and Tweedledum as was once said. Like it's – these are very significant differences in terms of the candidates and what they stand for and what they're going to – how they're going to carry themselves as president.

And I think, you know, you got to respect voters. They recognize that. And so I think that's the thing that's going to get people to the polls. Even if they're not enthusiastic about their choices, they will be engaged enough because of the contrast in those choices. Right. And in fact, I think that's probably the reason why turnout like this century has generally been higher than it was in the 80s and 90s. So turning back to the nervier portion of the conversation. Yeah.

When can we expect sort of the likely voter model to kick in? Is it post Labor Day that pollsters sort of switch to using the likely voter numbers to

Instead, because I know our averages convert registered voter polls to likely voter polls. So we are not going to see a big change between now and post Labor Day in our averages. But for the pollsters who are trying to sort of solve this question, what are they going to do? It's different for every pollster. But yeah, historically, Labor Day has kind of been the switching over point.

All right. Well, I imagine that this is a conversation we are going to come back to because it is a very interesting one. And it's one that none of us really know the answers to, even if there are lots of good arguments in one direction or another. Before we wrap up for the day, though, I do want to do something of a good or bad use of polling segment, which is to ask questions.

you guys what the best and worst uses of polling were that you saw at APOR. All right, let's start with bad just so we can end on a happy note. OK, so, Jeffrey, what was your bad slash contested use of polling? So there was there was a panel where there was some debate about waiting by past presidential vote.

in polling. So this is when pollsters take the sample they have. One of the weightings that they do is not only to like the population, but to an election in this case, usually the 2020 election. It's a way of maybe trying to avoid oversampling Democrats, for instance, once you once you

account for the fact that the kinds of people who are responding to polls may be more democratically named because they're more educated or what have you. This is like a thing that a number of pollsters are doing now. There was a gentleman from SSRS who sort of, I thought, put it best, which was like, I don't love that we're doing this. It seems like it could be potentially problematic because electorates look different election to election, right? And so just assuming that like

A big part of the electorate is going to – should be modeled in that way is maybe dicey. But at the same time, it might be sort of the best choice you have under the circumstances to make sure you sort of avoid –

a sample that's sort of completely out of kilter with reality. So it was just like a very interesting debate. And there are a number of pollsters that are doing this now. There are others that don't do it. I thought one of the other interesting things was that some pollsters have also been waiting by not 2020 vote, but waiting by the method that people used to vote because we know that Democrats are more likely to vote by mail, for instance, and Republicans are more likely to vote in person on election day. So this was like –

It was an interesting discussion about how to do that. And it doesn't seem like there's a perfect answer. So it was really interesting. All right, Nathaniel, what was your bad use of polling? So my bad use of polling was from Sean Patterson at the University of Pennsylvania. It's not a bad call on him out by now. No, it's good. Now, even he would agree like this. This experiment did not work. It was a noble, a noble effort. But it was really interesting. So basically, he's trying to solve a lot of people are trying to solve this issue of Republicans aren't

responding to surveys. They're not joining like panel surveys at as high rates, for example. And one of the things that he tried was to like really just like ratchet the patriotism up to 11. So like the like letters that he sent to voters to get them to join their panel survey, they like put a big American flag on like the top of it. And instead of saying like, dear voter, they said like, dear fellow American,

and stuff like that. So they're thinking Republicans are patriots, right? So they'll join our patriotic panel. It didn't really work. The panel did end up being more Republican by share. But the issue is that this messaging turned off everybody. It just turned off Republicans by less. So Republicans responded to the poll at a slightly lower rate than the control sample. But Democrats and independents really responded at a much lower rate. So not exactly the best practice going forward, which Sean rightly pointed out.

Very interesting. Okay, let's end on a good use of polling. Nathaniel, what do you got? So my good use of polling came from Courtney Kennedy at the Pew Research Center. She did a really interesting and compelling presentation on basically on why margin of error is not sufficient. It's an outdated statistic. So something that we've talked about before, but frankly should be talking about more is that margin of error only accounts for sampling error, but there are lots of other types of error that can happen in polling like measurement error, non-response error, things like that.

And so when you say a poll has a plus or minus margin of error of like three points or whatever, theoretically, it's supposed to be within that range 95% of the time. But we know that isn't true. We know, for example, that an average polling error for a presidential election is like four points and for Senate and House elections, it's even higher than that.

And so basically, she was advocating for creating a new statistic called like margin of total error. And she kind of went through a couple of different ways in which we could go about making that. And they all kind of have their problems. But I think it's a really important pursuit, given that

Obviously, there is a crisis in kind of trust in polling and institutions more broadly. And that when we tell people this poll has a margin of error of only three points, don't worry about it. That's actually misleading because in reality, an average polling error is larger than that. It's generally about twice as large as the published margin of error for election polls. So that's important in terms of people's understanding or interpretation of the polls, knowing that there is more uncertainty than is usually advertised.

Yeah, it sounds like there's a lot of good information that hopefully we'll try to get on the podcast down the line at April. Jeffrey, wrap it up for us. What was your good use of polling?

I would say just broadly the sense that I got from a number of presentations about the ways that different pollsters are trying to make sure they get a good initial sample and are reaching hard-to-reach populations. And this involved, for instance, the mode of sampling. So pollsters sort of expanding the ways that they would get in touch with people. So they might send out a letter to respondents asking them to join a panel, right?

And it was like, well, we can have, you know, phone just basically different ways to take the poll once you've joined the panel. Like it's,

Is it easier for some people to text? Sure. Younger people were like somewhat more likely to do that. But interestingly, there were like fascinating demographic breakdowns on like black voters were more likely to answer by phone. So making sure you had a phone option for people to actually call in and respond was like important for making sure that you had like a good initial sample of Americans for this panel or poll or whatever you were working on. And so just kind of.

The different information out there about how people were trying to reach individuals. And in fact, this phrase was used multiple times by different presenters. Meet the voters where they are was a really interesting thing that was reassuring. Pollsters are out there working really hard to try to get the best data they can. So I thought that was a good use for sure. The non-response devil works hard, but pollsters work harder. Good to know. Good to know. We're going to leave things there for today. Thank you so much, Nathaniel and Jeff.

Thanks, Galen. Thank you, Galen. My name is Galen Druk. Our producers are Shane McKeon and Cameron Chortavian, and our intern is Jayla Everett. You can get in touch by emailing us at podcast.538.com. You can also, of course, tweet at us with any questions or comments. If you're a fan of the show, leave us a rating or review in the Apple Podcast Store or tell someone about us. Thanks for listening, and we will see you soon. Bye.