Which Covid-19 restrictions really worked — and which ones really didn’t?
After a partial lockdown in the spring of 2020, Florida and Texas were among the states that most aggressively reopened their economies. Most businesses were allowed to resume operations in May or June of last year. Florida never instituted a public mask mandate; Texas was the first state to revoke its mask mandate this spring.
California and New York, on the other hand, have been more cautious. They didn’t let some businesses, like movie theaters and gyms, reopen until months after their more conservative counterparts had already done so. Their state mask mandates are still at least partially in effect.
And yet, looking at the case and death numbers since the coronavirus pandemic began, it’s not obvious which states were cautious and which were not. New York, the original epicenter of the outbreak, has endured the second most deaths per capita behind New Jersey (271 per 100,000). Florida and Texas, despite much criticism of their laissez-faire approaches, rank right in the middle among states (26th and 24th, respectively) in the number of deaths per 100,000 people. California fared only marginally better, sitting at 30th.
After a year of debates over mask mandates, lockdowns, and school closures, that mixed evidence might suggest a certain fatalism: Did none of these state policies really matter? Or was the virus going to spread no matter what states did? Was it all for nothing?
Which interventions actually work is one of the most important lessons the US could learn from the Covid-19 pandemic. There will be more outbreaks in the future and, after the current response was marked by indecision and inconsistency, this is an opportunity to separate what policies and interventions are most effective from those that are a waste of time.
But it’s also a maddeningly difficult question to answer.
“State policies mattered,” Jen Kates, director of global health policy at the Kaiser Family Foundation, told me. “But it is hard to know and we may never know whether policies made at the state level were able to overcome all the other challenges of this pandemic.”
Once the US lost control of the virus, mitigation was the only realistic path forward; fully eradicating Covid-19 was out of the question. According to the available research and interviews with experts, one policy, the requirement to wear masks indoors, appears to have successfully slowed transmission. But others, namely school closures, don’t appear to have had nearly as much of an effect on case rates.
The pandemic was constantly evolving, which adds to the difficulty in figuring out which policies worked and what didn’t. And while early on, some lockdown measures — especially stay-at-home orders and closing restaurants and bars — seemed to limit Covid-19’s spread, they may have become less effective over time, in part because states abandoned them and in part because Americans’ behavior had become more politically polarized and some people stopped following those rules.
For all of these reasons and more — the unpredictable nature of the virus’s spread, the structural differences between states, and the American tradition of federalism that delegated most policy decisions to state governments — the nation’s response was a policy morass.
Still, the variation has one potential upside: With the benefit of hindsight, experts told me we can begin to deduce whether certain interventions were more effective than others. It’s a start.
It’s hard to figure out how effective social distancing policies actually were
If you look at a list of states by their number of Covid-19 deaths per capita, it’s hard to discern much of a pattern. States in the Northeast, where the virus first arrived and where states were most aggressive in restricting people’s activities, rank at the top: New Jersey, New York, Massachusetts, and Rhode Island go first, second, third, and fourth.
But after that, the picture becomes more confused: Mississippi, Arizona, Alabama, and South Dakota — where the virus didn’t land until later and which were some of the most relaxed states in their policy responses — are also in the top 10.
The coronavirus arrived in different places at different times. The first wave of cases in New York peaked in mid-April. California and Florida didn’t see their first peak until the back half of July. States also have immutable characteristics — climate, population density, population health, and so on — that affected their experiences during the pandemic.
“A surface-level analysis of Covid-19 outcomes and state-level policies misses many other factors that differed across boundaries and over time that also had an influence,” Joshua Michaud, associate director of global health policy at the Kaiser Family Foundation, said. “It’s so hard to disentangle effects. We’re often left trying to draw firm conclusions about whether certain policies worked or not from data that can’t tell us that.”
The different timelines are one of the biggest reasons it’s hard to decipher which policies were effective.
Most of the country locked down in March and April 2020; 40 states issued stay-at-home orders in those first two months. But while Covid-19 was raging in New York at the time, it had hardly touched the Midwest or the South.
That gave those regions a false sense of security. Many states lifted their stay-at-home orders and other business restrictions starting in April, before their own outbreaks really got going — and some of them never reestablished those policies once cases did take off.
Florida, for example, rescinded its stay-at-home order and reopened most businesses starting on May 18. When cases started to accelerate there in late June, Gov. Ron DeSantis closed bars specifically but otherwise declined to put any restrictions back in place.
Most states had also implemented a whole slate of closures and restrictions at the same time during the initial lockdown period, making it even harder to isolate the effect of a single policy. How do you distinguish whether it was school closures, restaurant closures, both, or neither affecting caseloads if they were all enacted at once?
“Given that most states enacted multiple policies to encourage social distancing over a short time period, it is not possible to estimate the independent effects of individual policies,” the authors wrote in a study published this month in Nature, which did conclude that social distancing policies collectively reduced spread.
Complicating matters further, different places within a state may have implemented different strategies if the local governments were more aggressive than the state. (And then some states, including Florida and Texas, sought to block those local restrictions. The policy landscape was a mess.)
Compliance with mandates and restrictions also waned. From the beginning, there was some geographic variation in how much people’s behavior (as measured by mobility data) changed after state stay-at-home orders. And over time, the response to the pandemic became even more politically polarized. In spring 2020, Democrats and Republicans were about equally likely to say that the coronavirus was a serious threat. But by late summer, opinions had sharply diverged. That same polarization showed up in polls about wearing masks and, eventually, in a person’s willingness to take the Covid-19 vaccines.
Policy doesn’t matter as much if people don’t adhere to the government’s requirements.
The one Covid-19 intervention that definitely worked was mask mandates
The evidence on lockdowns may be dicey, but the science on masks is clear: They work. Even experts I spoke with who think harsh lockdowns may have been counterproductive say indoor mask mandates were clearly effective.
“Indoor masking guidance was proven to be effective,” Amesh Adalja, senior scholar at the John Hopkins Center for Health Security, told me. “When you look at it all, I think that is probably going to be the one that shows the most effect. … Most things can be done safely if people socially distance and wear a mask indoors in an unvaccinated setting.”
The available research supports that conclusion. In a study published in March 2021, CDC researchers examined case and death rates at the county level after mask mandates were put into place and found the mandates were associated with slower transmission.
“Mandating masks was associated with a decrease in daily Covid-19 case and death growth rates within 20 days of implementation,” they concluded, and the effect grew the longer the mandates were in place.
An earlier study, published in June 2020 in Health Affairs, had reached the same conclusion. Its authors estimated that mask mandates had averted some 200,000 Covid-19 cases by mid-May; at the time, the US had counted less than 2 million cases, indicating that the mask mandates had a meaningful effect in slowing the virus down early in the pandemic.
Some commentators have questioned why dire warnings about what would happen when Texas lifted its mask mandate for good in March 2021 never materialized. But the mandate’s rollback took place in a very different context from the spring of 2020.
For one, many more people now have protection from the virus, between vaccinations and prior infections. More widespread immunity was already an obstacle for the virus.
But on top of that, because the pandemic has become so politicized, people have already sorted themselves into their different camps, experts indicated — and so a state mandate might not have changed behavior. By now, you are already either a mask-wearer or you’re not. A government mandate probably isn’t going to affect someone’s behavior in June 2021 as much as it would have a year ago, especially after enforcement has been nonexistent.
Other lockdown measures were effective early but lost value over time
The Texas story is a good example of one clear takeaway on state policies: Early interventions did help slow Covid-19 down, but the longer the pandemic dragged on, the less effective they were.
Some states were victims of bad timing, with the virus spreading rapidly before any of these policies could have an effect; I would put New York and New Jersey in that bucket. But other states that would also eventually see among the most cases and deaths per capita — some of those Southern and Midwestern states — may have suffered because compliance fell off, a pattern potentially driven by their states’ lax policies as well as their political make-up.
Researchers have been trying to figure out for a year whether the lockdowns actually worked. Aaron Yelowitz, a University of Kentucky economist, and his colleagues were some of the first to publish such a study in the July 2020 edition of Health Affairs. They concluded that stay-at-home orders and closing bars and restaurants did slow down Covid-19’s spread. The data on school closures and bans on large gatherings was less convincing.
I asked Yelowitz whether he thought that conclusion still held up. He told me he believed it did — but that the value of those mitigation measures had started to deteriorate in the later stages of the pandemic.
“During the initial phase of the pandemic … there was large overall compliance and buy-in with public health initiatives,” he said. “Mobility fell dramatically, and that slowed the spread of the virus.”
But then states relaxed their policies and local outbreaks spiraled from there. Almost every state relaxed some restrictions by May or June of 2020; the differences were a matter of degree. Republican-led states tended to drop all or most of their policies; Democratic-led states were more gradual. But the true lockdown period in the US was actually quite brief and, as states were reopening in the summer, the virus had more opportunities to spread.
At the same time, Donald Trump was casting doubt on the seriousness of the situation and the summer protests were driving controversy about whether the lockdown measures were being fairly applied. Some public officials were also discovered to not be living by the same rules they were setting for everyone else.
Public support weakened and in many places the restrictions of the spring never came back. Even if they had, they might not have been as effective as they were at first. Policies lose their power when a lot of people won’t abide by them.
“During the remainder of 2020, had such measures remained in place, they would have impacted the spread of Covid-19,” Yelowitz said. “But not nearly as much due to lower ‘buy-in’ from the public and increased polarization.”
More recent analysis, notably the new study in Nature from researchers at Google and Boston University, also supports the conclusion that social distancing measures did affect Covid-19’s spread early on.
States started to see an effect when they declared a state of emergency; the amount of time people spent away from home dropped by nearly 10 percent. That effect grew with one additional social distancing policy or more (another 25 percent drop in mobility) and a shelter-in-place or stay-at-home order (a 30 percent decline).
Less mobility correlated with fewer Covid-19 cases, the authors wrote: “A 10% reduction in mobility was associated with a 17.5% reduction in case growth two weeks later.”
They cautioned about the difficulty in deducing which specific policies had the most impact, given that most places implemented multiple restrictions at once. However, they made an attempt by focusing on states that initially put only one social distancing measure into place.
That secondary analysis indicated closing bars and restaurants was more effective than school closures or bans on large gatherings. Yelowitz and his colleagues’ had reached the same conclusion a year earlier. A third study published in January 2021 had similar findings. There is a growing body of evidence supporting certain interventions more than others.
But slowing down transmission through (partial) lockdowns only goes so far — and the US didn’t take advantage of the time it had bought.
The US missed the chance to beef up its response during its partial lockdown
All of this comes with a catch: Certain restrictions may have been effective in limiting Covid-19’s spread initially, but there was likely no policy that would have eradicated the virus entirely. The virus spread too widely too quickly.
On top of that, America did not actually lock down very hard or for very long. The US version of “tight” Covid-19 restrictions was not the same as lockdowns in some European and Asian countries. Even progressive US states were relaxed by comparison.
People continued meeting in private settings, even if the gatherings were technically limited to 10 people or fewer. And all of these policies were more voluntary than mandatory; the US did not charge exorbitant fines for breaking lockdowns the way some other countries did.
“The pandemic would have sustained itself no matter what,” Adalja said. “This is a highly transmissible respiratory virus. It’s not something that could be eradicated or eliminated.”
There is an alternative timeline where the guidance focused on masks and other harm reduction techniques, he argued, instead of punitive and indiscriminate lockdowns that laid the seeds for their own backlash.
“We went into this abstinence-only mode, which we don’t do with any other infectious disease because it’s been discredited,” Adalja told me, “And I think that’s what caused this backlash and all of the acrimony over masks and whatever it might be.”
Making matters worse, the United States didn’t actually take advantage of the opportunity it had created for itself through the lockdowns.
Even if eradication was out of the question, America could have conceivably set up a system for testing, tracing, and isolating Covid-19 contacts that would have prevented small clusters of cases from spiraling out of control. That was the approach taken in global Covid-19 success stories like South Korea. A test-trace-isolate program is easier to set up when cases are still limited, as they were during the lockdown period of spring.
But instead, the US government failed to scale up its testing or contact tracing capabilities. As states relaxed their policies and the virus started spreading again, America was ill-suited to track it. The window that these early lockdowns had briefly opened was closed.
Author: Dylan Scott