Calling elections is a high-stakes algebra problem. Here’s how it works.
On Tuesday, November 3, we will start getting results for the crucial presidential, Senate, House, gubernatorial, and other elections taking place throughout the country. We won’t have full results that night, but the media will still probably be able to tell you who won a few (if not many) of them, due to the subtle but important art of election calling.
For decades now, news agencies like the Associated Press and broadcast services like NBC News and CNN have been projecting presidential election winners on a state-by-state basis, before all the votes are in. It’s an important service that greatly speeds up our knowledge of who’s won what, without having to wait for the final tally from state authorities, which in some cases can take a while to count. Their track record for accuracy has been excellent, but in the rare cases it goes wrong, it can go very wrong — as in 2000, where every major service called Florida for Al Gore early on, only for it to become clear the race was a squeaker.
This year could be especially tricky for election callers because of the unprecedented number of mail-in/absentee ballots, owing to the practice’s vast expansion during the Covid-19 pandemic. Furthermore, President Trump and his allies have pushed aggressively for networks to call the election the night of, dubiously portraying late-arriving votes as fraudulent when they should count just like every other vote. All these factors mean additional pressure on networks to call early.
The news outlets, however, emphasize that the “decision desks” making calls at networks and the AP are staffed by professionals, often PhDs in political science or related fields, who are well-insulated from the more opinionated sides of their news operations. For instance, Arnon Mishkin, Fox News’s decision desk leader, is known as a straight shooter by observers across the political spectrum, not beholden to the right-wing figures at his network. “He’s good. He’s been doing this a long time,” Scott Tranter, who is running the race calls team at Decision Desk HQ, a private company providing vote tabulation and race call data, told me. “He — how do I put this — he could give a fuck what Sean Hannity says at 8 pm. And that’s good, and Fox still hires him for it.”
The decision desks at other networks are similarly professionally staffed, apart from any anchor interference. If you don’t like Tucker Carlson or Rachel Maddow or Wolf Blitzer, don’t worry, they’re not making the calls.
Vox is doing election calls this year, too, with the help of Tranter and the team at Decision Desk HQ. They provide trustworthy, timely information to contracting media firms; besides Vox, their clients have included BuzzFeed, the Economist, the Atlantic, Huffington Post, FiveThirtyEight, Axios, Reuters, and Business Insider. They’re good, in other words. Here’s how their election-calling procedure, and that of their rivals at the AP and on various TV networks, works.
Election calling, explained
Arguably the most famous, and perhaps the most labor-intensive, election-calling operation is run by the Associated Press. Per the AP, it will employ more than 4,000 stringers (freelance reporters hired for this assignment specifically) and send them to county election centers, where they’ll call in raw vote totals to AP’s decision desk as they come in. Some 800 vote-entry clerks will screen the data for abnormalities and then enter them into software that also raises flags if the numbers look “inconsistent or statistically unlikely.”
“AP’s race callers are staff who are deeply familiar with the states where they declare winners. Most have called races in a state for many years,” the agency explains. The calls they make depend heavily on incoming vote totals being reported from counties, analyzed in conjunction with knowledge of how those counties typically vote, their demographics compared to those of the state as a whole, etc.
But sometimes the AP and other news agencies make calls as soon as the polls close. In the past, agencies have relied heavily on exit polls to make those projections. The AP has moved away from exit polling in favor of a system it calls AP VoteCast, which is “expected to complete about 140,000 interviews with registered voters between Oct. 28 and Nov. 3, concluding as polls close on Election Day.”
Conducted with the National Opinion Research Council (NORC), which also runs the highly regarded General Social Survey, AP VoteCast is meant to replace exit polls with an alternative method that accounts for the large chunk of voters who vote before election day, either in early voting or via absentee/mail. (Fox News will also use AP VoteCast, but calls it “Fox News Voter Analysis.”)
Either way, exit polls/VoteCast can make it possible to call easy, not-close races early. Take, for instance, the Senate election in Rhode Island in 2020. Jack Reed, the Democratic incumbent, has won each of his past three reelection battles with over 70 percent of the vote. If AP VoteCast’s polling suggests this time will be no different, it’s very reasonable for the AP to call the election for Reed shortly after polls close.
Decision Desk HQ, Vox’s partner, calls these “insta-calls,” and relies on a house-made forecasting model to make them rather than exit polls or a VoteCast-style poll. Tranter explained that the forecast model uses public polling but also economic data, demographics, turnout estimates, and so on.
The race call team will also rely on private polls it is conducting in North Carolina, Georgia, Florida, and elsewhere DDHQ is in the field. They’re doing a high-sample survey in Florida, for instance, which Tranter says, “will inform us on how to think about those early vote returns at 8 pm.”
ABC News, CBS News, CNN, and NBC News have a different method on election night. They rely on a partnership called the National Election Pool, which provides exit polling and other data to the member organizations. (Fox News and AP used to be a part of it but left the group after the 2016 race.)
NBC News explains that NEP will have about 4,000 staffers in 2020, both for exit polling and for collecting county- and precinct-level votes (which AP used to do for all these groups before it left the pool). Like AP, NBC claims, “Vote results are rigorously checked and verified. Part of quality control involves checking that vote data is consistent across sources, and we also compare the vote to past election results to see whether the turnout looks extremely different across multiple past races.”
NBC clarifies that while it relies on that vote and exit poll data to make its projections, its projections are independent of the NEP itself and made for NBC alone. Other NEP members like CNN have explained their process similarly. Once there’s raw vote data to look at, Jennifer Agiesta, CNN’s director of polling and election analytics, says, “we’re looking at how much we know about all the different types of vote that are out there, where in the state those votes have come from, how they compare to what we know about votes there in the past, and what we know about what’s left to count.”
The unusually high level of absentee voting/vote-by-mail in 2020, amid the Covid-19 pandemic, could complicate calling efforts. To get a sense of what the vote that’s in at, say, 8:30 pm means, you have to have a sense of how many people voted in person that day versus by mail, how much of the initial vote total is by mail versus in person, where the in-person votes come from, how in-person versus by-mail voters lean in key races, and so forth.
With voting methods changing dramatically this year, that all could be harder to predict. “If a state is somewhat slower to count its mail-in ballots, which we find often does happen, then we need to adjust our race calling to make sure that we have an accurate view of all the outstanding votes before we call a race,” Sally Buzbee, a senior vice president at the AP who runs their election-calling operation, explained at a roundtable event of election-calling experts.
CNN’s Agiesta agreed with that sentiment, saying, “the amount of vote left to count is critical, and may be a more difficult piece of information to track down in 2020,” because of vote-by-mail and “the decreased value in knowing the number of precincts reporting. There are fewer people voting on Election Day in most places and some states are consolidating precincts, so comparisons of the number of people voting in a particular precinct now to the past are less valuable, and it may be harder to get a good read on Election Day turnout before a county or town is fully reported.”
The result might be more cautious network calls this year than in the past. But even the presidential election not being called until the next day would not be unprecedented. In 2000, of course, the process was slow, but in 2004 as well, networks declined to call the election for George W. Bush until Wednesday morning.
Decision Desk, Vox’s partner, explained
Vox has worked with Decision Desk since 2017. Here’s how its process works: Officials responsible for reporting elections generally report them on a public website. But if you’ve ever tried to access one of these sites, you’ve probably noticed they are mostly terrible. They often do not post the most useful information front and center — like where to vote and election results. They also aren’t necessarily easy to find through a Google search.
This is where firms like Decision Desk come in. Media outlets pay them to do the extra work necessary to pull results together. Decision Desk uses an API, or application programming interface, that essentially allows the firm to get the information at the same time as it’s published on those state and county websites, provided by election officials. It also scrapes information directly from other public sites. And it uses old-fashioned methods, like phone calls and faxes, though to a far lesser extent than the AP does, to communicate with county officials.
The result, it argues, is a data collection operation that’s much faster than other organizations’, which in turn translates into faster race calls. Tranter explains that ultimately, for both his team at Decision Desk HQ and other call teams, calling elections is an algebra problem. The equation is super-simple: Each candidate’s votes (vote-by-mail plus early voting in person plus Election Day) summed equal total turnout:
Candidate A (VBM + EIPV + EDV) + Candidate B (VBM + EIPV + EDV) + … + Candidate Z (VBM + EIPV + EDV) = TT
Where VBM = Vote By Mail; EIPV = Early In-Person Voting; EDV = Election Day Vote; and TT = Total Turnout.
Going into the night, the teams have a decent sense of how many people have voted by mail and in person, and might be able to use polling to figure out whom those people have voted for. These team’s turnout models provide an estimate of total turnout. But the rest is in flux throughout the night.
“The key variable in all this as tabulation nears completion is, how many votes are left?” Tranter says. “If you know how many votes are left you can determine if there are enough votes to move a second-place candidate into first place, and if there is not, then you can assume the current vote leader will win.” On Tranter’s team, there’s a unanimity rule: All three race call team members assigned to a particular race have to agree that it’s impossible for the trailing candidate to make up the difference in order to call that race.
Is it possible they’ll miscall some races? Sure: They expect to call 500 to 600 close races on election night, and even a team with 99.8 percent accuracy would miss a race in that situation.
But so far, Decision Desk has had a strong track record. It’s only had to go back on one general election call since it started offering its services to clients like Vox, which was in California’s 21st Congressional District, where Republican David Valadao lost to Democrat TJ Cox in 2018. It was an extraordinarily close race, and though at first it looked like Valadao had a lead that wouldn’t be affected by mail-in ballots, it ultimately flipped on the last mail-in ballots. On this, Decision Desk wasn’t alone. The Associated Press had to recall that election too, plus a race in North Carolina that DDHQ didn’t erroneously call.
Decision Desk also made one incorrect projection in the 2020 primaries, predicting that Rep. Lacy Clay (D-MO) would win his primary when challenger Cori Bush beat him in an upset. “The challenge we had in that race was an unexpected split between the early mail-in votes and the day-of vote,” Tranter says. “While that has been expected in the general election, we had not until that point seen such a drastic split within a Democratic primary challenge. When the Election Day vote went so strongly to Bush, it became clear she would win and that our call for Clay had to be publicly withdrawn.”
But DDHQ called the vast majority of races correctly, and the AP has had two retractions in Georgia’s congressional primaries this year, putting DDHQ still ahead by a bit in terms of accuracy. DDHQ is often quite fast, relative to competitors. It beat its main competitor, AP, by 47 minutes in New Hampshire and by 15 minutes in Nevada, and it called the race almost immediately in South Carolina.
In short, you can trust Decision Desk HQ. You can also trust its competitors like AP, NBC News, ABC News, Fox News, and so forth. We picked DDHQ because we think they’re the best, but no one is running a shoddy operation here.
The key, though, is patience. Part of being a good race caller is declining to call races when there’s legitimate uncertainty. That means you might not get definitive answers as soon as you want to on election night, or even election week. Be patient, and we’ll all figure this out together.
The United States is in the middle of one of the most consequential presidential elections of our lifetimes. It’s essential that all Americans are able to access clear, concise information on what the outcome of the election could mean for their lives, and the lives of their families and communities. That is our mission at Vox. But our distinctive brand of explanatory journalism takes resources. Even when the economy and the news advertising market recovers, your support will be a critical part of sustaining our resource-intensive work. If you have already contributed, thank you. If you haven’t, please consider helping everyone understand this presidential election: Contribute today from as little as $3.
Author: Dylan Matthews