Updates to Our Charity Evaluation Criteria in 2024
Introduction
Animal Charity Evaluators is entering the time of year when we conduct in-depth reviews to update our list of Recommended Charities. To enhance transparency and provide insight into the evaluation process, we are sharing changes to our charity evaluation criteria that will apply to this year’s set of charities. These changes reflect our current thinking, and we may continue to refine and adapt them to ensure they are well-suited to the charities we evaluate.
Impact
We are introducing a new Theory of Change analysis to complement our Cost-Effectiveness analysis. Together, these two assessments will form the Impact criterion.
The Cost-Effectiveness analysis assesses the impact per dollar of charities’ most important programs. It explicitly models uncertainty by providing range estimates, allowing for quantitative comparison across charities. This year, we will create models tailored to each charity’s programs for greater accuracy, flexibility, and ease of interpreting of their impact.
The Theory of Change analysis is the qualitative counterpart to the Cost-Effectiveness analysis. It will be customized for each charity depending on the types of work they do, with the following objectives:
- Assessing the strength of evidence and reasoning underpinning the charity’s programs
- Identifying key risks and limitations of the charity’s work and how they plan to mitigate those risks
- Assessing the charity’s self-awareness and transparency about potential risks and limitations to their work
- Understanding how the charity’s different programs interact with each other to bring about change
- Understanding how the charity collaborates with and contributes to the broader animal advocacy movement
Note about the Impact Potential criterion
Last year, charities were assessed on Impact Potential, which indicated their potential to reduce animal suffering or improve animal wellbeing based on the groups of animals the charity’s programs target, the countries where they occur, and the intervention types they use. This year, we have improved and streamlined our Impact Potential scoring process by assessing only the most relevant factors and combining factors where possible. For example, we now consider a combined Animal Group/Country score rather than independently scoring animal groups and countries.
Impact Potential is no longer a standalone evaluation criterion. Instead, we consider Impact Potential scores when reviewing charities’ applications to help us identify their work’s potential strengths and limiting factors. We will likewise refer to these scores as needed to inform our Cost-Effectiveness and Theory of Change analyses. This ensures that we are making evidence-backed assessments while acknowledging that the reliability and relevance of impact potential scores will vary depending on each charity’s unique circumstances.
Room for More Funding
This criterion assesses how much money a charity can effectively use in the next two years (the length of our recommendation period) based on their historical financials and plans for growth. This includes assessing how their work in the next two years would differ under various funding scenarios and gauging whether there will be diminishing returns to their work over time. The results of the criterion ensure that ACE only recommends organizations that will put future funding to effective use and will guide our Recommended Charity Fund distributions to create the most impact for animals.
Organizational Health
We use this criterion to assess whether any aspects of an organization’s leadership or workplace culture pose a risk to its effectiveness or stability, reducing its potential to help animals. This year, we have shortened the staff engagement survey and reduced the list of questions we ask charity leaders. This change will help us gather the necessary information while reducing the time burden on charities. We have also collaborated with Scarlet Spark to update our questions and compile a database of resources to equip charities to improve their policies and processes.
The importance of this criterion is explained in our 2023 blog post Why We Assess Charities’ Organizational Health.
Acknowledgments
We extend our appreciation to the experts who offered feedback and guidance on our methods this year, including:
- Sjir Hoeijmakers (Giving What We Can)
- Martin Gould (Open Philanthropy)
- Vicky Cox (Ambitious Impact)
- Tania Luna and Alyssa Greene-Crow (Scarlet Spark)
- James Honda (Animal Defense Partnership)
- Adam Salisbury (GiveWell)
- Kim Huynh (Giving Green)
- Katrina Sill (Innovations for Poverty Action)
- Engin Arıkan (Animal Advocacy Careers)
