Charity Navigator used to tell you if charities are well run. Now, it’ll also tell you if they’re getting things done.
The world’s largest independent charity evaluator is called Charity Navigator, and it lists hundreds of thousands of US-based charities with programs around the world, providing information to help donors decide whether they want to give money there.
Charity Navigator provides insights into a nonprofit’s financial stability and adherence to best practices for accountability and transparency.
But until recently, it’s been unable to answer one big question donors frequently want to know: How much good will my money actually do?
This week, Charity Navigator announced that will change. It’s acquired a smaller, impact-focused charity evaluator called ImpactMatters, which estimates how far your money goes when you donate it, and is rolling out a new Impact and Results rating that lets people choose charities by how much they get done.
When it comes to giving, what really matters is how much good your money will do. But until pretty recently, there often wasn’t any answer to that question that an average donor could find. The Charity Navigator acquisition demonstrates that impact is becoming more and more of a priority in the nonprofit sector — and that seems likely to be very good for the world.
Figuring out how much good charities do
Charity Navigator’s new rating system is the latest move in a growing trend: greater awareness that nonprofits vary immensely in how much good they accomplish. Even within a specific category, ImpactMatters cofounder Elijah Goldberg told me, some soup kitchens feed far more people with the same amount of money; some homeless shelters are able to offer quality shelter to far more people.
Donors care about that. In surveys, most of them list effectiveness as one of their top concerns when giving to charity, and many people don’t give to charity out of a vague sense that most charities don’t work very well. Stories about ineffective charities are remembered by donors, who often give less — and effective charities that do a lot of good tend to make for less memorable stories.
“Impact has always been seen as the holy grail,” Michael Thatcher, the CEO of Charity Navigator, told me. “At the end of the day, we want to know what our money has done, not necessarily where it got spent.”
But that’s easier said than done. Measuring a charity’s impact is time intensive and requires answering lots of difficult questions — which of these studies on the effects of education programs on test scores later in life do we believe? If the charity has limited data, how far can we generalize from the data it does have? If it collects no data on many of its programs, what should be our base assumption about their impact?
Historically, donors who care about effectiveness haven’t had many options. A decade ago, no charity evaluators that tried to arrive at systematic estimates of impact existed. Then GiveWell was founded, changing the landscape: It researched which charities get you the most impact for your money in the world, and eventually identified lots of global health and development programs. (Disclosure: I donate to GiveWell every year.) Research suggests that it’s extraordinarily cost effective to benefit children by treating them for malaria and intestinal parasites, and that many programs fail to do as much good as just giving people money.
That’s valuable work, Goldberg told me, but it leaves a gap: if a person is determined to donate to ending homelessness in their community, then knowing the world’s highest impact charities doesn’t help them. But GiveWell’s research-intensive project can’t easily be translated to ranking the thousands and thousands of US nonprofits that donors might be looking up.
“The best site out there [for evaluating impact] is GiveWell, but they’re only naming the top 10” best charities, Dean Karlan, ImpactMatters cofounder and a professor of economics at Northwestern University, told me. “So for someone who wants to support someone in their local community or education internationally, we saw this big gaping hole.”
ImpactMatters was an attempt to change that. Their impact evaluation process doesn’t involve as much heavy-lifting as GiveWell’s. They don’t attempt to compare across cause areas — soup kitchens are evaluated by how many meals they give out and homeless shelters by how many nights of shelter they provide — and they’ve tried to build a process that can be used to evaluate thousands of organizations, so that every charity can get a rating for impact and then, ideally, be motivated to improve it.
Because it evaluated so many charities, its process was a natural fit for Charity Navigator’s, which tries to offer a rating for every nonprofit there is available data for. Historically, it has based those ratings on financial indicators, like how much money the charity has in the bank, how responsibly it spends that money, and how much is spent on “overhead,” or administrative expenses.
The reason donors care about any of those things, of course, is that they care about how much good their money is doing. Whether a charity is well run and how it spends its money are easy-to-measure indications that ideally provide a window into the question that really matters: whether your donation makes a difference in the world. But they’re very imperfect indicators. A charity might have high administrative expenses because it works in an area where legal compliance is really important but still be high impact. A charity might increase its overhead expenses by hiring a secretary, but be significantly more effective as a result of having freed up program officers to spend their time on programs.
ImpactMatters uses those financial indicators but tries to go beyond them. “We’ve always been focused on understanding the nonprofit’s mission and then assessing mission success,” Goldberg told me.
Where impact is easiest — and hardest — to measure
Evidence Action is one charity highlighted on Charity Navigator’s new splash page explaining how its impact evaluations work. “$0.50 provides clean water to a person for a year,” the rating reads. That’s clean, straightforward, and easy to understand. This style of impact evaluation works really well for charities that provide clean water, or medication, or food and shelter.
“The approach of ImpactMatters — of cost per outcome — makes a ton more sense in certain areas of the sector,” Thatcher told me. “We’re starting with what they have, which works incredibly well in a services area.”
To be eligible for an impact rating under the current system, a nonprofit has to dedicate the bulk of its resources to programs “directly delivered to beneficiaries and reasonable to expect impact measurement for.” Furthermore, the nonprofit must mostly deliver results to beneficiaries who are different from its donors — membership clubs, religious organizations, and many performing arts groups do not meet this criterion.
Evaluating the impact of charities who do less direct work is important, too. But it’s much harder. Say an advocacy organization for affordable housing lobbies local politicians, contributes to getting bills introduced which will build more houses, publishes an election slate which hundreds of people refer to when voting, and hires a lobbyist to make the case for housing at the statehouse. What is their impact? ImpactMatter’s current approach can’t answer that question — though it’s working on it.
What we’ve learned about what makes charities work
What does a high-impact charity look like? Goldberg said that in ImpactMatter’s research, it’s found that it’s only partially how well run a nonprofit is — much of the difference in impact is a consequence of choices that the nonprofit made about who it will consider as beneficiaries and how it sets eligibility for its programs.
For example, “Scholarship programs that do need-based scholarships tend to be significantly more effective than those that do merit-based,” Goldberg told me. “The choice of geography and beneficiary population and program type ends up defining a lot of how effective you are.”
That means that as a donor, going in wanting to work in a specific program type or in a specific area will limit the impact of your donation. Being willing to consider charities in a different program or geographic area are major ways to have your donation go much further. But the ImpactMatters and Charity Navigator teams I talked to emphasized that they didn’t want to build a tool that just told people “Your money could go further if you picked a different program.” Instead, they argue, donors should research the programs that interest and inspire them — keeping an eye on impact while they do so.
The aim of an impact evaluator is two-fold: First, to help donors find charities where their money will do a lot of good, which encourages donors to give more and makes sure that more is accomplished with their money. Second, ideally an impact evaluator gives charities tools to learn more about how they operate and how to accomplish their mission.
“This is a journey, and one of the things we’re trying to do with the rating system is really work with nonprofits to adjust, adapt, and improve,” Thatcher told me. “There’s an interesting problem that we have in giving, which is that we expect everyone to get it right the first time. The ideal there is that we learn from our mistakes.”
“The goal is, by having more transparent data, you can at least start looking and seeing some other food bank is twice as effective and call them up and see how they do that,” Karlan told me. Once nonprofits have ways to discuss and compare impact, they can start making more of the choices that make them highly effective.
Millions turn to Vox each month to understand what’s happening in the news, from the coronavirus crisis to a racial reckoning to what is, quite possibly, the most consequential presidential election of our lifetimes. Our mission has never been more vital than it is in this moment: to empower you through understanding. But our distinctive brand of explanatory journalism takes resources. Even when the economy and the news advertising market recovers, your support will be a critical part of sustaining our resource-intensive work. If you have already contributed, thank you. If you haven’t, please consider helping everyone make sense of an increasingly chaotic world: Contribute today from as little as $3.
Author: Kelsey Piper
Read More
