Triff/shutterstock
Triff/shutterstock

The idea of publishing ratings of charitable organizations began to be realized in 2001, when Charity Navigator became the first organization to provide comprehensive ratings for a broad range of nonprofits. The organization has steadily increased its reach and has expanded from initially evaluating slightly more than 1,000 groups to currently covering over 160,000 nonprofits, with its recent introduction of a second rating system called Encompass that will ultimately evaluate nonprofit performance based on four key indicators.

Charity Navigator’s traditional star ratings—which it continues to publish—have largely been driven by financial, transparency, and governance and accountability metrics. What had largely been missing from those traditional evaluation metrics is how a charity translates resources to accomplishments, something that Charity Navigator seeks to address in its Encompass system, which was initially launched in July. The organization just announced another step toward refining its ability to address impact with its acquisition of peer charitable ratings organization ImpactMatters, made possible in part by the Bill and Melinda Gates Foundation.

Michael Thatcher, Charity Navigator’s CEO, explained why the organization undertook this acquisition. “Charity Navigator has been looking at expanding how we rate nonprofits. We want to tell users not only ‘where is my money going’ but ‘what has my money done?’” Thatcher said. Charity Navigator first started talking with ImpactMatters about the topic four years ago, and Dean Karlan, the economics professor who co-founded ImpactMatters in 2015, helped Charity Navigator run the analytics on its own experiments in rating charitable outputs.

Charity Navigator began to consider a closer connection with the outfit, which also made sense to ImpactMatters, its co-founder and Executive Director Elijah Goldberg said. “We were looking for another organization to join to expand our own reach and Charity Navigator was a natural candidate.”

The two started talking about joining forces toward the end of 2019, and those talks culminated in Charity Navigator’s just announced acquisition of the employees and assets of ImpactMatters, a transaction that the Gates Foundation supported financially. A grant of $150,000 covered the parties’ legal expenses and other costs of integration, which enabled the two organizations to move forward and complete the merger.

Victoria Vrana, who oversees the Data and Insights portfolio at the Gates Foundation, said that the funder has long supported increasing the supply of information about nonprofit organizations and improving data on philanthropic capital flows. Gates has supported both ImpactMatters and Charity Navigator in the past.  Goldberg pointed out that the foundation is one of the few funders of the sort of philanthropic infrastructure work illustrated by the two organizations.

As another example of its support of charitable ratings organizations, Vrana cited the Gates Foundation’s backing for the GuideStar and Foundation Center study that led to the 2019 merger of those organizations into a new nonprofit entity, Candid. In explaining Gates’s support for these sorts of combinations, Vrana explained that Gates isn’t promoting “top down consolidation” and doesn’t believe that approach works, but noted that “when nonprofits come to us with these proposals, it’s something Gates will consider, especially for our grantees.”

Charity Navigator was the first third-party nonprofit evaluator that was free to the public and all nonprofits. The group has steadily expanded its coverage, and currently rates approximately 9,000 groups using its star rating system, and over 150,000 other charities in its Encompass rating system. Charity ratings have proven to be very popular with the public. During Charity Navigator’s fiscal year ending in June, there were more than 8 million unique users of the website, a figure that rose 13% from the prior year.

As charity ratings have evolved, they have become important drivers of giving, especially among individual donors. In a 2015 study of the effect of third party charitable ratings, economics professor Barış K. Yörük found that Charity Navigator’s ratings had a positive and significant impact on charitable contributions received by relatively small and unknown charities, where a one-star increase in ratings was associated with a nearly 20% increase in the amount of contributions received.

At the same time, Charity Navigator has been aware that the criteria for obtaining an evaluation under its traditional star system presented a barrier to entry that effectively excluded many organizations, especially newer and smaller nonprofits, from the platform.

To address this, the new Encompass ratings platform allowed the group to add over 150,000 generally smaller charitable organizations to the site. These groups were initially evaluated on a “Finance & Accountability” metric, and with the acquisition of Impact Matters, more than 700 charities will now receive an “Impact & Results” rating based on an ImpactMatters evaluation.

Note that Charity Navigator considers these ratings tools to be in “beta mode” until it completes the Encompass system by incorporating “Leadership & Adaptability” and “Culture & Community” ratings. Once the two remaining elements of the Encompass Rating System are completed, Thatcher said that users of Charity Navigator’s ratings will have an ability to see a more holistic view of a nonprofit’s health that goes far beyond overhead ratios.

As charity ratings platforms have increased their profiles in the philanthropy world, observers have been critical of the systems’ inability to evaluate not just how a charity manages its finances and governance, but how well it accomplishes its mission—i.e., its impact. Critics say this has led to a fixation on concepts like “overhead” or “administrative expenses,” which don’t truly reflect an organization’s effectiveness. (For yet another take on this topic, check out a previous argument in IP that even impact is the wrong metric to focus on in charitable giving.)

In a 2014 interview with the Chronicle of Philanthropy, discussing a book that he co-authored with his wife, investment banker Sheryl WuDunn, New York Times columnist Nicholas Kristof criticized nonprofit ratings. “There is too much emphasis on inputs and not enough on impact.” Kristof said. “This has been worsened by an effort to create more accountability through sites like Charity Navigator. There is so much emphasis now on expense ratios that there is an underinvestment in administration and efficiency.”

This is a challenge that Thatcher acknowledges. He noted that Charity Navigator currently publishes impact information gathered by four groups: GuideStar, Classy, GlobalGiving, and ImpactMatters. With the acquisition of ImpactMatters, Charity Navigator hopes to improve its capability to portray a rated organization’s impact more fully. ImpactMatters itself currently rates a relatively small number of nonprofits (over 1,000), but Goldberg, who will assume a new role as Charity Navigator’s vice president for impact ratings, indicates that his number one priority is to expand that figure. One way in which he plans to do this is to build an app for the Charity Navigator website that enables highly structured data collection from nonprofits, taking advantage of Charity Navigator’s massive reach by giving organizations the ability to share their impact data directly with Charity Navigator.

There is no question that measuring impact is difficult. A group of experts recently met in New York to explore this issue, and issued a report suggesting ways to advance this effort. The report identified a need for more data infrastructure and improved technologies to advance impact management, such as standardized mechanisms for collecting impact data, assessing its quality, as well as standards for sharing, and reporting impact data.

Goldberg notes that ImpactMatters’s analysis currently only applies to direct service organizations with measurable outcomes, such as meals provided by a food bank. In terms of potential areas for expansion, Goldberg pointed out that health and education are tricky areas, but they represent enough of the sector that it’s important to take them on. For education, candidate program types include charter schools, youth development programs (at least related to education), and financial aid. For health, candidates could include free/subsidized clinics and community health programs.

Goldberg also noted that they don’t look at things like advocacy organizations. “It’s not that we don’t think it’s important, but we don’t think we can measure [advocacy] well. I think it’s a limitation that we can’t really measure advocacy—but we do think being able to do so could be a good investment.”

ImpactMatters co-founder Dean Karlan, who currently teaches at Northwestern University’s Kellogg School and who will remain on Charity Navigator’s expert committee for ratings, said it may be possible to expand into such challenging areas.

“There is a spectrum that goes from easy-to-measure to hard-to-measure. ImpactMatters initially avoided the harder to measure impacts. But in areas where the outcomes are difficult to measure, you could instead measure how often an advocacy group’s work gets cited or how often legislators use its work,” he said.

Share with cohorts