States are inflating their coronavirus testing numbers with the wrong kind of test

America’s coronavirus testing numbers appear to be improving — but the data may be getting less reliable.

Throughout May, America has made a lot of progress in scaling up its coronavirus testing abilities. But as they dug deeper over the month, experts and journalists have increasingly questioned whether some of the testing data can really be trusted.

First, the good news: America’s testing numbers continued trending up this week. There were about 380,000 tests a day over the week of May 14, up from roughly 320,000 tests a day over the week of May 7 and around 150,000 a day during much of April, based on data from the Covid Tracking Project.

That’s still not the 500,000 a day that many experts say is necessary — much less the tens of millions a day that some have called for — but the country is making noticeable progress nonetheless.

But then there’s the bad news: Several states are inflating, deliberately or not, their testing numbers by including two different types of tests that experts say should be separated out. That includes the numbers reported by the Covid Tracking Project and subsequently used in Vox’s chart, which rely on the data that states report.

Only a handful of states are confirmed to be doing this, and a few have already backed off, based on reporting from Alexis Madrigal and Robinson Meyer at the Atlantic. So it’s not all of the data that’s now under question.

But it’s a worrying trend that presents a somewhat rosier picture of the Covid-19 testing numbers than is perhaps warranted. And that makes it hard to gauge whether these states are really ready to start reopening their economies, even as some have already begun to do so.

Some states are muddying their testing numbers

Traditionally, Covid-19 testing numbers have been measured based on the number of diagnostic tests on a daily basis. Diagnostic tests gauge whether a person has the virus in her system and is, therefore, sick right at the moment of the test.

Some states’ counts, however, now include antibody tests, which check if someone ever developed antibodies to the virus to see if she has ever been sick in the past. It’s a way for states to pad the numbers — but at the cost of accuracy, since antibody tests are generally less precise, and timeliness.

Since antibody tests can’t tell you if someone is sick at the moment of the test, they can’t tell you, like diagnostic tests can, what the current state of Covid-19 in a community is. “We need to understand that there is a new case of a new disease happening in our community,” Pia MacDonald, an epidemiologist at the research institute RTI International, told me. “There are public health interventions that need to happen around that.”

Given these different purposes, experts say that states should separate out diagnostic and antibody tests in their counts. Only then can they truly isolate their Covid-19 preparedness in the present by tallying their diagnostic tests and metrics attached to those, such as the positive rate, while separately gauging the overall impact of Covid-19 on the population via antibody tests.

“Both of [the tests] are useful, but you can’t put them together,” MacDonald said.

Yet some states are doing just that. Madrigal and Meyer reported in the Atlantic on May 14 that Virginia was “blending the results of two different types of coronavirus test in order to report a more favorable result to the public.” Then on Thursday, May 21, they reported that Georgia, Pennsylvania, Texas, and Vermont were doing the same, as well as the Centers for Disease Control and Prevention at the federal level. Maine was also mixing its test results, but reversed course on Wednesday, May 20. Virginia also stopped, following reporting from the Atlantic and the Richmond Times-Dispatch.

Apparently, not all of these states did this deliberately. At a press conference with the governor, Vermont officials claimed that “they didn’t know they were — they were just publishing numbers from testing sites that were adding em in,” VTDigger reporter Erin Petenko tweeted.

Regardless of intent, the practice muddies the waters — making it harder to know if states really are in a better place to start reopening their economies.

This makes it hard to tell if states are truly ready to reopen

Testing is a crucial component to getting control over the pandemic. When paired with contact tracing, testing lets officials track the scale of the Covid-19 outbreak, isolate the sick, quarantine those whom the sick came into contact with, and deploy community-wide efforts as necessary. Testing and tracing are how other countries, like South Korea and Germany, have managed to control their outbreaks and started to reopen their economies.

“The whole point of this social distancing is to buy us time to build up capacity to do the types of public health interventions we know work,” Natalie Dean, a biostatistics professor at the University of Florida, previously told me. “If we’re not using this time to scale up testing to the level that we need it to be … we don’t have an exit strategy. And then when we lift things, we’re no better equipped than we were before.”

Some states have explicitly built testing into their reopening plans, calling for a certain amount of tests or positive rates. But those goals were built with advice from experts that by and large assumed the metrics would be based on diagnostic tests, not antibody tests.

So we’re left with a situation in which America’s testing numbers do seem to be improving (finally), but how much of that improvement is real, and how much of it suggests that any particular state is ready to reopen, could come down to how honestly places are reporting their numbers.

Support Vox’s explanatory journalism

Every day at Vox, we aim to answer your most important questions and provide you, and our audience around the world, with information that has the power to save lives. Our mission has never been more vital than it is in this moment: to empower you through understanding. Vox’s work is reaching more people than ever, but our distinctive brand of explanatory journalism takes resources — particularly during a pandemic and an economic downturn. Your financial contribution will not constitute a donation, but it will enable our staff to continue to offer free articles, videos, and podcasts at the quality and volume that this moment requires. Please consider making a contribution to Vox today.