Recommendation Criteria for Animal Advocacy Charities
Before we select which organizations to recommend, we establish the criteria on which we will evaluate them as well as how we intend to assess those criteria. Below is a summary of the recommendation criteria we use to evaluate charities.
Criterion 1: Programs
A charity that performs well on this criterion has programs that we expect are highly effective in reducing the suffering of animals. Effective programs: i) work in high-priority cause areas or countries, ii) work toward high-priority outcomes, iii) target high-priority animal groups, and/or iv) pursue interventions that are expected to be highly effective.
Criterion 2: Room for More Funding
A charity that performs well on this criterion has plans that cannot be fully accomplished with the expected funding from other sources, and the barriers to accomplishing those plans are strictly monetary—not due to lack of time, lack of qualified personnel, or other non-monetary issues. If a charity has room for more funding, the funding that is expected to come from an ACE recommendation would be very likely to increase the charity’s total impact.
Criterion 3: Cost Effectiveness
A charity’s recent cost effectiveness provides an insight into how well it has made use of its available resources and is a useful component in understanding how cost effective future donations to the charity might be. Given the limitations of strictly quantitative approaches, we have moved away from using a fully quantitative model and have instead transitioned to a more qualitative approach that analyzes the resources used and the outputs achieved for each intervention type.
Criterion 4: Leadership and Culture
A charity that performs well on this criterion has strong leadership and a healthy organizational culture, including a healthy attitude toward representation/diversity, equity, and inclusion (R/DEI). The charity acts responsibly toward stakeholders, including staff, volunteers, donors, and others in the community. In particular, staff and volunteers feel engaged with their work and protected from harassment and discrimination.
Criteria for Evaluating Potential Charities
Below, we outline and elaborate upon each of the four criteria that we use to evaluate charities, as well as how we assess those criteria. This is the most recent version of our charity evaluation criteria. Previous versions are available in our Process Archive.
ACE has learned a great deal from conducting charity evaluations over the past several years. These updated criteria reflect new information that we think is relevant to our process, and we expect that they will continue to change as we gain more experience evaluating charities.
Our current approach is to subjectively weigh each criterion relative to the others. While we have worked hard to standardize our opinions, we think it is best to make calls on a qualitative, case-by-case basis rather than create an explicit, quantitative weighting system.
- Criterion 1: Programs
- Criterion 2: Room for More Funding
- Criterion 3: Cost Effectiveness
- Criterion 4: Leadership and Culture
Criteria Descriptions
Criterion 1: Programs
How do we look for evidence of high-impact programs?
A charity that performs well on this criterion has programs that we expect are highly effective in reducing the suffering of animals.
We use relevant empirical evidence, expert testimony, theories of change, and the Scale, Neglectedness, and Tractability framework to prioritize the key factors that enable us to identify the most effective programs to help animals. We consider the following factors and use information supplied by charities to analyze each one in more detail.
- Cause area. We prioritize programs working on cause areas that have a high potential for effectiveness. We currently prioritize farmed animal advocacy and wild animal advocacy.
- Outcomes. We currently find the arguments for an institution-focused approach1 more compelling than individual-focused approaches. Thus, we generally prioritize institution-focused outcomes such as improving welfare standards, increasing the availability of animal-free products, and strengthening the animal advocacy movement. We give lower priority to individual-focused outcomes such as decreasing the consumption of animal products, increasing the prevalence of anti-speciesist values, and providing direct help to animals.
- Countries. We prioritize countries with relatively large animal agricultural industries, few other charities engaged in similar work, and in which animal advocacy is likely to be feasible and have a lasting impact. In our charity selection process, we use Mercy For Animals’ Farmed Animal Opportunity Index (FAOI), which combines proxies for scale, tractability, and global influence to create country scores.2 We use our own data on the number of organizations working in each country to assess neglectedness.
- Animal groups. We prioritize programs targeting specific groups of animals that are affected in large numbers3 and receive relatively little attention in animal advocacy. We currently prioritize farmed fishes and farmed chickens, as well as programs targeting invertebrates. Given the large number of wild animals and the small number of organizations working on their welfare, we believe wild animal advocacy also has potential for high impact despite its lower tractability.
- Interventions. In line with our commitment to following empirical evidence and logical reasoning, we aim to use existing research to inform our assessments and explain our thinking about the effectiveness of different interventions. When available, we use relevant research and our research briefs to inform our assessment of interventions.
What if a large charity has programs with both high and low impact?
It’s important to note that charity donations may be used to fund all of a charity’s projects, not strictly its most effective ones. This problem can not always be solved by restricting donations to specific programs due to the problem of fungibility; i.e., if a charity receives restricted funding for effective Project #1, it can then use that donation to free up the unrestricted funding that would have gone toward Effective Project #1, and instead move that unrestricted funding to less effective Project #2.
Is pursuing a project with low impact grounds for disqualification?
One might consider charities that pursue low-impact projects as unable to commit to cause prioritization. However, what is considered high-impact is not well-established or agreed upon across the movement. In some cases, programs that seem to have a low impact—such as those that are not focused on farmed or wild animals—are effective in context of a charity’s larger goals; for example, an organization may run a companion animal program for the purpose of engaging the community and expanding its farmed animal programs. Additionally, projects that can be considered low-impact may have positive indirect impacts, such as generating significant funding from donors or garnering positive press.
Criterion 2: Room for More Funding
In this criterion, we examine whether a charity has room for more funding and plans for growth. We aim to recommend work that is both high-impact and scalable, and seek to ensure that a charity would be able to absorb and effectively utilize funding that an ACE recommendation may bring in. If a charity has a prior recommendation status, we examine whether it will continue to effectively absorb funding that comes from our recommendation.
Why might a charity not have room for more funding?
Charities may have non-monetary bottlenecks to expanding their operations, e.g., difficulty hiring the talent needed to expand, government restrictions, or a lack of necessary partnerships with other groups. Given the variety of potential problems, we need to verify that a charity’s effective interventions are bottlenecked specifically by money. Otherwise, donations might not be what a charity needs, so we would not want to recommend it to donors.
How do we evaluate a charity’s room for more funding?
Our first step in evaluating a charity’s room for more funding is to look at its financial data. In particular, we check how the charity’s finances have developed in previous years and assess whether it holds a sufficient amount of reserves. We ask the charity to provide estimates of its predicted revenue for the next three years, and we also ask whether it’s funded by any single major source of income as this can be associated with growth.
We then look at a charity’s plans for expansion and the associated costs. Charities may differ in how ambitious their reported plans are, and such a difference in reporting could bias our estimates of their room for more funding. To counteract such a bias, we ask all charities for the expansions they have already planned, as well as which expansions they would plan if they received an additional $100,000 or $1,000,000 per year—approximately the funding an ACE Standout or Top Charity recommendation would bring in, respectively. Charities may also report a greater number of plans in order to inflate their room for more funding. To counteract this, we provide an assessment of whether the charities’ expansion plans could actually be realized.
How do we monitor a charity’s room for more funding?
It is possible that a charity could run out of room for funding more quickly than we expect, or that the charity could come up with good ways to use funding beyond what we expect. If a charity receives a recommendation as a Top Charity, we check in mid-year about the funding it’s received since the release of our recommendations, and we use the estimates presented above to indicate whether we still expect the charity is able to effectively absorb additional funding at that time.
Criterion 3: Cost Effectiveness
What goes into a cost-effectiveness assessment?
A charity’s recent cost effectiveness provides insight into how well it has made use of its available resources and is a useful component in understanding how cost effective future donations to the charity might be.
In a perfect world with no uncertainty, a cost-effectiveness calculation would be all that is needed to evaluate a charity. However, for reasons discussed by GiveWell in “Why We Can’t Take Expected Value Estimates Literally” and by Peter Hurford in “Why I’m Skeptical About Unproven Causes,” it is necessary to take a more nuanced approach.
From 2014–2018, we created quantitative cost-effectiveness models that compared charities’ outcomes to their expenditures for each of their programs and attempted to estimate the number of animals spared per dollar spent. As these models were developed each year, some repeated issues emerged:
- We were only able to model short-term, direct impact. Our attempts to model the medium/long-term or indirect effects of interventions were too speculative to be useful. This meant that we could not produce models for charities that were focused mostly on medium/long-term or indirect outcomes, and we often had to omit programs from the charities for which we did produce models.
- The estimates produced by the models were too broad to be useful for making recommendation decisions. Ultimately, we want each criterion to support our recommendation decisions, and we found that we often were not confident enough in the models to give them weight in those decisions.
- While we appreciate the value of using numbers to communicate our estimates and uncertainty, we found that by using numbers, our estimates were often misinterpreted as being more certain than we intended.
- The variation in cost effectiveness between charities was more dependent on which interventions the charity used rather than how they were implemented. This suggests that, rather than modeling the cost effectiveness of each charity, it would be more useful to model the average cost effectiveness of each intervention and incorporate that into our discussion of effectiveness in our Programs criterion.
How did we move beyond a quantitative cost-effectiveness calculation?
Since 2019, we have moved away from using a fully quantitative model and instead transitioned to a qualitative approach that, for each intervention type, analyzes the resources used and outputs achieved.
We ask charities what outputs they have achieved with their programs using specific reporting guidelines. Often, charities tend to overstate their accomplishments. To verify charities’ claims about the results of their programs, we use a methodology that relies on publicly available information as well as supporting evidence, including internal documents, media reports, and independent sources. We also send a list of follow-up questions to charities to elaborate and request additional supporting evidence for claims that we may not be able to initially verify.
Criterion 4: Leadership and Culture
What is a strong leadership and a healthy culture?
A charity with a strong leadership and a healthy culture is one that is well-managed on a strategic and operational level. While this may look different across charities depending on their unique needs, there are some characteristics of charities with strong leadership and healthy cultures that we can look for specifically.
A strong leadership guides the organization competently and promotes a healthy culture. They are committed to transparency, although we do not expect all organizations to be transparent with the public about sensitive information. We recognize that organizations and individuals working in some regions or on some interventions could be harmed by making information about their work public. In these cases, we favor confidentiality over transparency. An organization with a strong leadership also develops and implements key human resources policies. As policies vary across countries and cultures, we do not evaluate charities based on their set of policies and do not expect effective charities to have all policies in place.
An organization with a healthy culture acts responsibly toward stakeholders, including staff, volunteers, donors, and others in the community. In particular, staff and volunteers have a work environment where they feel engaged with their work and protected from harassment4 and discrimination5. This includes a healthy attitude toward representation/diversity, equity, and inclusion (R/DEI). We use the terms “representation” and “diversity” broadly to refer to the diversity of certain social identity characteristics (called “protected classes” in some countries).6
Why was this included as a metric?
The way an organization is led affects its organizational culture, which in turn impacts the organization’s effectiveness and stability.7 Healthy relationships among staff and well-developed methods and routines allow projects to be carried out efficiently. Strong leadership and a healthy culture also enable charities and their supporters to make long-term plans and build capacity in the animal advocacy movement as a whole through training and development.
Some aspects of a strong leadership and a healthy culture are specifically important in enabling ACE to work with and recommend a charity. For example, transparent charities benefit the movement as a whole by sharing information and enabling others to do stronger work. In addition, we are not able to conduct thorough reviews of charities that are not willing to be highly transparent with us, and we are not able to publish reviews unless they are willing to be at least moderately transparent with the general public.
How do we look for evidence of a strong leadership and a healthy culture?
We review aspects of organizational leadership and culture by capturing staff and volunteer perspectives via our culture survey, in addition to information provided by top leadership staff (as defined by each charity).
We recognize that surveying staff and volunteers could lead to inaccuracies due to selection bias, and that surveys may not reflect employees’ true opinions as they are aware that their responses could influence ACE’s evaluation of their employer. In our experience, it is easier to assess issues with an organization’s culture than it is to assess the strength of an organization’s culture. Therefore, we focus on determining whether there are issues in the organization’s culture that have a negative impact on staff productivity and wellbeing.
We collect information about staff members’ perceptions of leadership and satisfaction with their compensation. To capture whether an organization also provides non-material incentives, e.g., goal-related intangible rewards, we elicit employee engagement using the Gallup Q12 survey. We consider an average engagement score below the median value of the scale a potential concern. We also ask staff about whether experiences of harassment or discrimination in the workplace have been addressed appropriately.
Additionally, we request charity leadership to provide information about human resources policies, including policies and activities regarding representation/diversity, equity, and inclusion (R/DEI).
The weightings used for calculating these country scores are scale (25%), tractability (55%), and regional influence (20%).
We don’t believe that the number of individuals is the only relevant characteristic for scale, and we don’t necessarily believe that groups of animals should be prioritized solely based on the scale of the problem. However, number of animals is one characteristic we use for prioritization.
Harassment can be non-sexual or sexual in nature: ACE defines non-sexual harassment as unwelcome conduct—including physical, verbal, and nonverbal behavior—that upsets, demeans, humiliates, intimidates, or threatens an individual or group. Harassment may occur in one incident or many. ACE defines sexual harassment as unwelcome sexual advances; requests for sexual favors; and other physical, verbal, and nonverbal behaviors of a sexual nature when (i) submission to such conduct is made explicitly or implicitly a term or condition of an individual’s employment; (ii) submission to or rejection of such conduct by an individual is used as the basis for employment decisions affecting the targeted individual; or (iii) such conduct has the purpose or effect of interfering with an individual’s work performance or creating an intimidating, hostile, or offensive working environment.
ACE defines discrimination as the unjust or prejudicial treatment of or hostility toward an individual on the basis of certain characteristics (called “protected classes” in some countries), such as race, color, ethnicity, religion, sex, gender or gender expression, sexual orientation, pregnancy or parental status, marital status, national origin, citizenship, amnesty, veteran status, political beliefs, age, ability, or genetic information.
Examples of such social identity characteristics are: race, color, ethnicity, religion, sex, gender or gender expression, sexual orientation, pregnancy or parental status, marital status, national origin, citizenship, amnesty, veteran status, political beliefs, age, ability, and genetic information.
