Recommendation Criteria for Animal Advocacy Charities
Before we select which organizations to recommend, we establish the criteria on which we will evaluate them as well as how we intend to assess those criteria. Below is a summary of the recommendation criteria we use to evaluate charities.
Criterion 1: Programs
A charity that performs well on this criterion is working in a domain with high potential for efficiency (for instance, farmed animal advocacy) and pursues interventions that are among those expected to be highly effective. This guiding information comes from reviews of relevant research and partly from our intervention evaluation process.
Criterion 2: Room for More Funding
A charity that performs well on this criterion has plans that cannot be fully accomplished with the expected funding from other sources, and the barriers to accomplishing those plans are monetary—not due to lack of time, lack of qualified personnel, or other non-monetary issues. If a charity has room for more funding, receiving money from donors influenced by ACE would be very likely to increase the charity’s total impact.
Criterion 3: Cost Effectiveness
A charity that performs well on this criterion has achieved relatively significant progress toward certain outcomes, with a budget that is smaller relative to other charities. If these cost-effectiveness calculations were produced with perfect information, they would be one of the most important factors in determining room for more funding. However, because our calculations are necessarily estimates, corroborative evidence is required.
Criterion 4: Track Record
A charity that performs well on this criterion has a record of achievement of incremental goals or demonstrated progress toward larger goals. Charities that have not existed long enough to fully assess their goal achievement may have strong potential, but donating to charities without track records is inherently risky.
Criterion 5: Leadership and Culture
A charity that performs well on this criterion has strong leadership and a healthy organizational culture. The charity acts responsibly to stakeholders, including staff, volunteers, donors, and others in the community. In particular, staff and volunteers are satisfied and engaged with their work, and the organization has a healthy attitude toward representation/diversity, equity, and inclusion (R/DEI).
Criterion 6: Strategy
A charity that performs well on this criterion has a clear and well-developed strategy. The charity’s overall mission puts a strong emphasis on effectively reducing animal suffering, and the charity responds to new evidence with that goal in mind, revisiting their strategic plan regularly to ensure they stay aligned with that mission.
Criterion 7: Adaptability
A charity that performs well on this criterion has the capacity to respond to areas of success and failure appropriately. Regular self-assessments should inform their decisions. If applicable, the organization has strengthened successful programs and discontinued less successful programs.
Criteria for Evaluating Potential Charities
Below we outline and elaborate upon the seven criteria we use to evaluate organizations as well as how we assess those criteria. This is the most recent version of these criteria. The previous version of this document is available in our Process Archive.
We have learned a great deal from conducting charity evaluations over the past several years. These updated criteria reflect new information that we think is relevant to our process, and we expect that they will continue to change over time as ACE gains more experience evaluating organizations.
Our current approach when weighing each criterion relative to the others is subjective. While we’ve worked hard to standardize our opinions, we think the best we can do is make calls on a qualitative, case-by-case basis rather than create explicit quantitative weights.
- Criterion 1: Programs
- Criterion 2: Room for More Funding
- Criterion 3: Cost Effectiveness
- Criterion 4: Track Record
- Criterion 5: Leadership and Culture
- Criterion 6: Strategy
- Criterion 7: Adaptability
Criteria Descriptions
Criterion 1: Programs
How Do We Look for Evidence of High-Impact Programs?
A charity that performs well on this criterion is working in a domain with high potential for effectiveness (for instance, farmed animal advocacy) and pursues interventions that are among those expected to be highly effective. To assess effectiveness, we consider the relevant empirical evidence, expert testimony, and theories as to what interventions have a high impact and check to make sure the charity is working on employing these interventions.
It’s hard to assess program impact. However, we think we still need to “keep an open mind” when it comes to interventions, as the evidence base for nonhuman animal interventions is still weak and some interventions have not yet received any study. Thus, this criterion can still be subject to substantial changes with new evidence arising.
What If a Large Charity Has Programs with Both High and Low Impact?
It’s important to note that, when donating to a charity, one is potentially donating to all of the charity’s projects and, therefore, is contributing to the marginal impact of the projects funded, not necessarily only the charity’s best. One might be tempted to think that this problem can be solved with restricted donations, however, this is generally not the case due to the problem of fungibility. Put simply, if a charity receives restricted funding to Effective Project #1, they can then use that donation to free up the unrestricted funding they would have put into Effective Project #1 and move that unrestricted funding to less effective Project #2.
Is Pursuing a Project with Low Impact Grounds for Disqualification?
One might consider a charity that is pursuing a project with low impact as unable to commit to prioritizing based on effectiveness. However, what does or does not have a high impact is not well-established or agreed upon across the movement, so perhaps such an instance is merely a difference of opinion.
Additionally, it is possible for these less effective projects to draw in large amounts of funding from donors who are unconcerned with effectiveness or generate positive press for the charity, and, therefore, these projects may have positive indirect impacts.
Criterion 2: Room for More Funding
The second criterion we examine is whether the charity has room for more funding and plans for growth. We look to recommend work that is not just high-impact but also scalable. When it comes to deciding where to allocate donations, it doesn’t matter how effective a charity is if they have no use for additional donations.
Why Might a Charity Not Have Room for More Funding?
For a charity, there can be other bottlenecks to expanding their operations than just money. A charity may not have or be able to hire the talent they need to manage that expansion. Plans for expansion could be, for instance, restricted by government legislation or a lack of necessary partnerships with other groups. Given the variety of potential problems, we need to verify that their effective interventions are bottlenecked specifically by money. Otherwise, donations might not be what the charity needs, and we would not recommend them to donors.
How Do We Evaluate the Charity’s Room for More Funding?
A first step in evaluating a charity’s room for more funding is to look at their financial data. In particular, we check how their finances developed in past years and whether they hold a sufficient amount of reserves. Based on the history of financial data, we also try to forecast the revenue that a charity may receive in the future, without additional funding directed to them.
In a second step, we try to assess the costs of a charity’s plans for expansion and which plans we think they can realize. We ask charities to list their plans for expansion. Charities may differ in how ambitious their reported plans are, independent of what they can realize. Such a difference in reporting could bias our estimates of the room for more funding. To counteract such a bias, we ask all charities not only for the expansions they already planned but also which expansions they would plan if their budget would increase substantially. Charities may also report a greater number of plans in order to inflate their room for more funding. To counteract this, we indicate our confidence in whether the charities’ expansion plans could actually be realized.
How Do We Monitor a Charity’s Room for More Funding?
Following a recommendation, a charity’s room for more funding may change unexpectedly. A charity could run out of room for funding more quickly than we expected if, for instance, a large donor fills the room for more funding before ACE-directed donors do. Or, a charity could come up with good ways to use funding beyond what we anticipated. At midyear, we check in with each recommended charity to review the funding they’ve received since the release of our recommendations, and we assess whether we still expect them to be able to effectively absorb additional funding at that point.
Criterion 3: Cost Effectiveness
What Goes into a Cost-Effectiveness Assessment?
A charity’s recent cost effectiveness provides insight into how well they have made use of their available resources and is a useful component in understanding how cost effective future donations to the charity might be.
In a perfect world with no uncertainty, a cost-effectiveness calculation would be all that is needed to evaluate a charity. One might also be tempted to think that if we have a sufficient guess as to how cost effective charities are, we have all we need to compare charities and can recommend whichever ones appear to be the most cost effective. However, for reasons discussed by GiveWell in “Why We Can’t Take Expected Value Estimates Literally” and by Peter Hurford in “Why I’m Skeptical About Unproven Causes,” this way of thinking is naive.
From 2014 to 2018, we had been creating quantitative cost-effectiveness models that compare a charity’s outcomes to their expenditures for each of their programs and attempt to estimate the number of animals spared per dollar spent. As these models have developed each year, we found that some repeated issues emerged:
- We were only able to model short-term, direct impact. Our attempts to model the medium/long-term or indirect effects of interventions were too speculative to be useful. This meant that we could not produce models at all for charities that were focused mostly on medium/long-term or indirect outcomes, and we often had to omit programs from the charities for which we did produce models.
- The estimates produced by the models were too broad to be useful for making recommendation decisions. Ultimately, we want each criterion to support our recommendation decisions, and we found that we often were not confident enough in the models to give them weight in those decisions.
- While we appreciate the value of using numbers to communicate our estimates and uncertainty, we found that by using numbers, our estimates were often misinterpreted as being more certain than we intended.
- The variation in cost effectiveness between charities was more dependent on which interventions the charity used rather than how they were implemented. This suggests that, rather than modeling the cost effectiveness of each charity, we would be better served to model the average cost effectiveness of each intervention and incorporate that into our discussion of effectiveness in Criterion 1: Programs.
How Did We Move Beyond Quantitative Cost-Effectiveness Calculation?
Given these recurring issues, since 2019 we have moved away from using a fully quantitative model and instead transitioned to a qualitative approach that, for each intervention type, compares the resources used and outcomes achieved across all the charities being reviewed. In the discussion, we have also included some aspects of each charity’s specific implementations of their interventions that seem likely to have influenced their cost effectiveness, either positively or negatively.
This approach does have limitations, as focusing on qualitative comparisons can lead us to be overly confident in our assessment. As such, we highlight where this approach may not work, and we have continued to put limited weight on this criterion as a whole when making decisions. That said, it has provided some insight into the cost effectiveness of all reviewed charities regardless of the timescale or directness of their work, allowing us to make comparisons that we were previously unable to make. We have focused on comparisons within interventions and not between interventions to avoid overlap with Criterion 1: Programs as well as to provide insight into how cost effective the charity might be in implementing new programs in the future.
Criterion 4: Track Record
What Is a “Strong Track Record of Success”?
A charity that performs well on this criterion has a record of achievement of incremental goals or demonstrated progress toward larger goals. Charities that have not existed long enough to fully assess their goal achievement may have strong potential, but donating to charities without track records is inherently risky.
What Would This Look Like? How Would We Find It in a Charity?
It’s easy to look at a charity’s website for this information, as charities usually like to share their claims of success in pages or newsletters. We also reach out to the charities directly and ask what outcomes they have achieved with their programs.
Often, charities tend to overstate their accomplishments. We need to make sure these accomplishments are backed up by evidence. To verify charities’ claims about the results of their programs, we use a methodology that relies on publicly available information as well as supporting evidence, including internal documents, media reports, and independent sources that we asked charities to provide. We also send a list of follow-up questions to charities to elaborate and provide additional supporting evidence for claims that we may not be able to initially verify.
What If We’re Considering a New and Unproven Charity?
Supporting a recently founded charity might be a powerful way for ACE to have an impact, as even small donations from ACE could make the difference between whether an eager start-up nonprofit succeeds or doesn’t succeed. However, doing so is taking a bit of a risk, since we won’t have this criterion to look at and this criterion is one of the most objective and reliable measures of management effectiveness. This presents a trade-off between risk and potentially higher rates of return that ACE would have to consider when conducting evaluations.
Criterion 5: Leadership and Culture
What Are Strong Leadership and Healthy Culture?
A charity with strong leadership and a healthy culture is one that is well-managed on a strategic and operational level. While this may look different across charities depending on the unique needs of each, there are some characteristics of a charity with strong leadership and a healthy culture that we can look for specifically.
First, a charity should have strong leadership that is attentive to the organization’s strategy and a governing board that is diverse in their occupational backgrounds and perspectives and oversees leadership independently. Second, a charity with a healthy culture acts responsibly with stakeholders, including staff, volunteers, donors, and others in the community. In particular, staff and volunteers should have a work environment where they feel engaged with their work and protected from harassment and discrimination. This includes a healthy attitude toward representation/diversity, equity, and inclusion. We also think that a charity should be transparent with donors and the general public.
Why Was This Included as a Metric?
An especially effective charity works well on an operational level. Healthy relationships among staff and well-developed methods and routines allow projects to be carried out efficiently. Strong leadership and healthy culture also enable the charity and its supporters to make long-term plans and build capacity in the animal advocacy movement as a whole through training and development.
Some aspects of strong leadership and healthy culture are specifically important in enabling ACE to work with and recommend a charity. For example, transparent charities benefit the movement as a whole by sharing information and enabling others to do stronger work. In addition, we would not be able to conduct a thorough review of a charity that is not willing to be highly transparent with us, and we would not be able to publish the review unless they are willing to be at least moderately transparent with the general public.
What Would This Look Like?
The charity should be able to demonstrate that their leadership is attentive to the organization’s strategy, implement policies that promote employee engagement, and create a work environment with a healthy attitude toward representation/diversity, equity, and inclusion. Leadership should bring on new staff and volunteers using an intentional training structure to build the capacity of individual staff and volunteers.
Criterion 6: Strategy
Why Was This Included as a Metric?
Given our commitment to finding the most effective ways to help nonhuman animals, we assess the extent to which the charity’s strategic vision is aligned with this commitment. Additionally, we assess the extent to which their strategic planning process incorporates the views of staff and board members and whether the frequency of this process is adequate, given the nature of their work. We think that charities that have a well-developed planning process are more likely to be successful at achieving their vision and mission.
What Would This Look Like? How Would We Find It in a Charity?
We identify charities with a strong organizational strategy by examining the charity’s overall vision, the way their activities fit into the overall animal advocacy movement, as well as their strategic plan, planning process, and goal setting and monitoring.
In most cases, the charity should have a well-developed strategic plan that is revisited regularly, and we believe that their strategic planning should clearly connect the charity’s overall vision to their more immediate goals. Ideally, the strategic plan should be developed through a formal planning process that includes input from all their staff and board members.
There are many different approaches to strategic planning, and often an approach that is well suited for one organization may not work well for others. Thus, in this section, we are not looking for a particular approach to strategy. Instead, we assess how well the organization’s approach to strategy works in their context.
Criterion 7: Adaptability
How Should Charities Respond to Success and Failure?
A charity with highly effective management will very likely have a robust and agile understanding of success and failure. This means the charity should gather and evaluate information about the performance of their programs as well as their operations and make adjustments accordingly. For example, we expect charities to strengthen their most successful programs, discontinue or modify their less successful programs, or address identified organizational issues to improve workplace culture and operational efficiency.
Why Was This Included as a Metric?
In the nonprofit world, feedback mechanisms are scarce, and declining bottom lines are not as apparent as in the for-profit world. A nonprofit may keep working even though they are not successful. Therefore, it is important that nonprofits include checks to ensure they keep advancing in their mission. Even if a nonprofit has evidence of their success, they should continue looking for ways to improve.
What Would This Look Like? How Would We Find It in a Charity?
We try to assess whether a charity is willing to change and make adjustments in light of metrics pointing one way or another. For instance, if a leafleting charity found that online video ads were more effective, or vice versa, would they make the switch? Has the charity made key changes in their methods before based on available evidence?
A charity that is responsive to success and failure learns from their current approach and then continuously updates their methods in response to new evidence.
