Recommendation Criteria for Animal Advocacy Charities
Before we select which organizations to recommend, we establish the criteria on which we will evaluate them—as well as how we intend to assess that criteria. Below is a summary of the recommendation criteria we use to evaluate charities.
1. Does the charity engage in programs that seem likely to be highly impactful? A charity that meets this criterion pursues interventions that are among those expected to be highly effective, without considering the specific implementation. Often this information will come partly from our intervention evaluation process. At the very least, the charity is working in a domain with high potential for efficiency (for instance, farmed animal advocacy) or is working in another domain with methods that appear to be exceptionally cost-effective.
2. Does the charity have room for more funding and concrete plans for growth? A charity that meets this criterion has plans that cannot be fully accomplished with the expected funding from other sources, and the barriers to accomplishing those plans are monetary—not due to lack of time, lack of qualified personnel, or other non-monetary issues. Receiving money from donors directed by ACE would be very likely to increase the charity’s total impact.
3. Does the charity operate cost-effectively, according to our best estimates? A charity that meets this criterion appears to be cost-effective in the goal of reducing animal suffering relative to other charities, when considering their budget and demonstrated successes. If these cost-effectiveness calculations were produced with perfect information, they would one of the most important factors necessary to consider beyond room for more funding. However, because our calculations are necessarily estimates, corroborative evidence is required. Read more about our cost-effectiveness estimates.
4. Does the charity possess a strong track record of success? A charity that meets this criterion has a record of successful achievement of incremental goals or of demonstrated progress towards larger goals. Note that this implies the charity has been in existence for some length of time. While very young charities may have strong potential to return large results for small initial amounts of funding, donating to charities without track records is inherently risky.
5. Does the charity identify areas of success and failure and respond appropriately? A charity that meets this criterion has both short- and long-term goals and can articulate signs that indicate whether they are moving towards or away from their goals. Regular self-assessments guide the charity’s program development. If applicable, the organization has responded appropriately in the past to signs that a program was not succeeding. Furthermore, they are able to innovate effectively: they regularly try small and large changes to programs, identify which changes improve outcomes, and act on that information.
6. Does the charity have strong leadership and a well-developed strategic vision? A charity that meets this criterion has leaders who seem competent and well-respected. The charity’s overall mission puts a strong emphasis on effectively reducing suffering, and the charity responds to new evidence with that goal in mind, revisiting their strategic plan regularly to ensure they stay aligned with that mission.
7. Does the charity have a healthy culture and a sustainable structure? A charity that meets this criterion is stable and sustainable under ordinary conditions, and seems likely to survive the transition should some of the current leadership move on to other projects. The charity acts responsibly to stakeholders including staff, volunteers, donors, and others in the community. In particular, staff and volunteers develop and grow as advocates due to their relationship with the charity.
Criteria for Evaluating Potential Charities
Below we outline and elaborate upon the seven different criteria we use to evaluate organizations, as well as how we assess those criteria. This is the most recent version of these criteria. The version of this document used in 2016 is available in our Process Archive.
Our current approach when weighing each criterion relative to the others is subjective. While we’ve worked hard to standardize our opinions, we feel the best we can do is make calls on a qualitative case-by-case basis when evaluating organizations, rather than creating explicit quantitative weights. We have learned a significant amount while conducting charity evaluations over the past several years. These updated criteria reflect new information that we feel is relevant to our process, and we expect that they will continue to change over time as ACE gains more experience evaluating organizations.
- Criterion #1: Does the charity engage in programs that seem likely to be highly impactful?
- Criterion #2: Does the charity have room for more funding and concrete plans for growth?
- Criterion #3: Does the charity operate cost-effectively, according to our best estimates?
- Criterion #4: Does the charity possess a strong track record of success?
- Criterion #5: Does the charity identify areas of success and failure and respond appropriately?
- Criterion #6: Does the charity have strong leadership and a well-developed strategic vision?
- Criterion #7: Does the charity have a healthy culture and a sustainable structure?
Criteria Descriptions
Criterion #1: Does the charity engage in programs that seem likely to be highly impactful?
How do we look for evidence of high impact programs?
It’s hard to assess program impact in a direct or easy way. What we intend to do is consider the relevant empirical evidence, expert testimony, and theories as to what interventions have high impact and check to make sure the charity is working on employing these interventions.
However, there still is a strong need at moment to “keep an open mind” when it comes to interventions, as the evidence base for nonhuman animal interventions is currently poor (for example, there is not yet any study with a control group, though several are now in development) and many potential interventions have not yet received any study. Thus, this criterion might be better used as a weak, heuristic approach to screen ineffective interventions out rather than screen effective interventions in.
What if a large charity has programs with both high and low impact?
It’s important to note that, when donating to a charity, one is potentially donating to all of their projects and therefore is meeting them at the marginal impact of the projects they choose to fund, not necessarily of their best. One might be tempted to think that this problem can be solved with restricted donations, however, this is generally not the case due to the problem of fungibility. Put simply, if a charity receives restricted funding to Effective Project #1, they can then use that donation to free up the unrestricted funding they would have put into Effective Project #1 and move that unrestricted funding to Ineffective Project #2. Thus, the donation did not accomplish any net increase to effective projects but rather resulted in a shift of funding, actually resulting instead in a net increase to ineffective projects.
Is pursuing a project with low impact grounds for disqualification?
This is a bit complicated. One might question a charity that is pursuing a project with low impact as not really understanding success and failure (see Criterion #5). However, what does or does not have high impact is not well-established and agreed upon, so perhaps the charity has a legitimate and justifiable difference of opinion.
Additionally, perhaps these ineffective projects are what draw in large amounts of donations from donors who are unconcerned with effectiveness, or perhaps these initiatives generate a lot of positive press for the charity, and therefore these projects have great indirect impact.
Therefore, it seems that pursuing a project with low impact should usually be a negative, but not grounds for disqualification since it may be the case that it has large, positive indirect effects.
Criterion #2: Does the charity have room for more funding and concrete plans for growth?
The first criterion we need to look at is whether the charity has room for more funding and plans for growth. When it comes to deciding where to allocate donations, it doesn’t matter how effective a charity is if they have no use for additional donations.
Why might a charity not have room for more funding?
For a charity, there can be more bottlenecks to expanding their operations than just money. Some charities may have a lot of money available to expand programs but not have the talent they need to logistically manage that expansion, or may be restricted by government legislation, or the inability to make partnerships with other groups, or any of a variety of different problems. For example, GiveWell once found that nonprofit Smile Train had a lot of money, but weren’t able to use it to hire needed developing-world doctors because the supply had been used up and there weren’t any more available to hire. More recently, GiveWell also found a similar problem with immunizations and micronutrient supplementations—there is substantial funding in this area, but there are logistical problems in creating the needed partnerships with the country governments, and these are problems that only time—not more money—can solve.
When evaluating the work of charities focused on nonhuman animals, we need to make sure that even if we find something effective, that the intervention in question is bottlenecked specifically by money and that more money is what will make the intervention happen. Otherwise, donations might not be what the charity needs and it would be counter-productive to recommend them to donors.
How can we evaluate the charity’s room for more funding?
The first way to evaluate room for more funding is to look at patterns in the financial data of the nonprofit. In particular, we could check to make sure the amount of assets they hold is not unusually large relative to the size of the charity and see if the charity has been expanding in past years. It would be good to check this on both the level of the charity as a whole and the level of individual programs/interventions, if applicable.
The second method is to ask the nonprofit directly. In GiveWell’s first year of research, they would ask as a part of their grant application process, “What would a significant increase in funding allow your charity to do that it couldn’t do otherwise?” Questions might get more specific about what a charity would do with certain levels of funding, for example, asking “what would your charity do with a one-time donation of $10K?” Though one must be careful as charities may have a tendency to tell the donor what they think the donor wants to hear.
How can we evaluate other aspects of room for more funding?
It’s not enough to evaluate room for more funding from the charity itself, however. Instead, one must also understand the risk that another donor would donate to the charity if ACE were not to move money to it. For example, before GiveWell could finish considering nonprofit GAVI for a recommendation, its $3.5 billion funding gap was completely filled by other donors.
Given that ACE is making donor recommendations this might not be an issue, but a large donor could potentially take out the room for more funding before ACE-directed donors could fill it. Thus, understanding the potential funding landscape for a charity by seeing what other funders in the space are doing is also important.
Criterion #3: Does the charity operate cost-effectively, according to our best estimates?
What goes into a cost-effectiveness calculation?
The goal of the cost-effectiveness calculation is to explicitly answer the question “How cost-effective is a donation to this charity/intervention, given what we know?” This is accomplished by creating an equation to get a figure of “years of nonhuman animal suffering averted per dollar,” which is a figure that appropriately takes into account the differences among nonhuman animals.
A lot of work would need to go into developing what this calculation would look like, how it would be adapted to different interventions, and how the information to complete the equation would be completed and updated. Because it’s so complex, we have published a comprehensive writeup on our use of cost-effectiveness estimates.
Why is a cost-effectiveness calculation not enough?
In a perfect world with no uncertainty, a cost-effectiveness calculation would be all that is needed to evaluate a charity. Indeed, one might be tempted to think that if we have a sufficient guess as to how many years of nonhuman animal suffering are averted, we have all we need to compare charities and can just go with whichever one appears the most cost-effective. However, for reasons discussed by GiveWell in “Why We Can’t Take Expected Value Estimates Literally” and by Peter Hurford in “Why I’m Skeptical About Unproven Causes,” this is naïve.
Put simply, there is a lot of opportunity for bias in cost-effectiveness measures, and measures generally need to be “regressed back to the mean” to adjust for bias and uncertainty. Therefore, we need to look at other criteria in order to calibrate our estimate.
Another possible way of thinking about this is that we are creating many weak arguments in favor of a particular nonprofit rather than one strong argument, which is the best we can do under the uncertainty that we will presumably be operating under.
How do we move beyond the cost-effectiveness calculation?
First, we define the following terms:
Effective management: a charity has effective management if it is well-run, well-managed, and accomplishes its mission as best as possible with its given resources.
High potential impact: a charity has high potential impact if it has picked a mission that is the most effective mission at reducing the suffering of nonhuman animals.
These concepts are distinct. We could have a charity that has high potential impact yet ineffective management—when it comes to public health, think an anti-malaria charity that wastes 90% of its money and staff time on fundraisers with incredibly poor returns and only uses 10% on net distribution.
We also could have a charity that has low potential impact yet highly effective management—when it comes to public health, think a charity that is focused on fighting breast cancer, yet is highly effective at getting National Science Foundation funding to the right Researchers, attracting top talent to the research field, establishing best practices at cancer early detection, etc.
Generally, the optimally cost-effective charity for nonhuman animals (i.e., the charity we want to move money to) is going to be the one with the most effective management (i.e., is most able to efficiently carry out its interventions) among the set or charities that have very high potential impact with respect to our larger cause of reducing nonhuman animal suffering (i.e., those charities that pick the optimal intervention or set of interventions for reducing nonhuman animal suffering).
The cost-effectiveness calculation gets us some of the way to assessing impact and effective management by taking these concepts into account, but looking at each of these two concepts more in depth from different angles can aid our assessment process.
Could it turn out that a charity has a potential impact so high that their management effectiveness doesn’t matter?
No. Even when potential impact is high, effective management will be needed to reach that potential; poor management could still lead to the charity having low—or even no—actual impact. Management effectiveness will always matter in complicated and more long-term ways as the charity seeks to scale up. Ultimately, it will always be better to donate to a charity that has high potential impact and effective management.
Additionally, we have fairly wide confidence bars over estimates of impact at the moment, so it might be too risky to go “all in” on measures of impact when issues of management effectiveness are raising red flags.
Criterion #4: Does the charity possess a strong track record of success?
What is a “strong track record of success”?
A charity that has highly effective management, by definition, should be able to accomplish its mission. Therefore, if the charity has been around long enough, it won’t be enough for them to merely understand what makes them successful, but they should generally be expected to have actually been successful by now and have accomplished some steps along the way to accomplishing their mission. Even if a charity has lofty goals that will take many years to accomplish (e.g., animal liberation), it still should have tractable subgoals on which it has made progress.
What would this look like? How would we find it in a charity?
It’s easy to look at a charity’s website for this information, as charities usually like to share their claims of success in pages or newsletters. If that doesn’t work, one can always reach out to the charities directly and ask what they’ve done so far with their donations. The real test will be carefully vetting the charity’s claims against common sense and evidence.
Generally, charities—even potentially highly effective ones like those that have been evaluated by Giving What We Can—tend to overstate their accomplishments. We need to make sure these accomplishments have evidence to back them up. An example of how to do this is to try to elaborate all the claims the charity makes in a particular document and then provide sources for as many of them as possible (for examples, see the Giving What We Can evaluations of climate change charities or the Copenhagen Consensus Report).
What if we’re considering a new and unproven charity?
Supporting a charity in its infancy might be a powerful way for ACE to have impact, as even small donations from ACE could make the difference between whether an eager start-up nonprofit (much like ACE once was) happens or doesn’t happen. However, doing so is taking a bit of a risk, since you won’t have this criterion to look at and this criterion is one of the most objective and reliable measures of management effectiveness. This is a trade-off between risk and potentially higher rates of return that ACE would have to consider when evaluating.
Criterion #5: Does the charity identify areas of success and failure and respond appropriately?
How should charities respond to success and failure?
Now that we’ve assessed impact, it’s time to turn our attention to effective management. If a charity has highly effective management, it will very likely have a robust and agile understanding of success and failure. This means the charity possesses detailed, thorough, and specific knowledge about what they want to achieve in the near and far term, and what feedback would tell them whether they are moving toward or away from that mission—as well as how they could move toward that mission better in a quick and iterative way. Put more simply, the charity can define “success,” knows what it looks like when they aren’t being successful, and acts in a way to gain quick feedback and update what they do accordingly. Furthermore, they are able to innovate effectively: they regularly try small and large changes to programs, identify which changes improve outcomes, and act on that information.
Why was this included as a metric?
Unfortunately, in the nonprofit world, feedback mechanisms are scarce. Unlike the for-profit world where a declining bottom line is apparent, nonprofits merely need to convince themselves and donors that they are successful, not actually be successful. It’s very easy for a nonprofit to keep spinning its wheels and think it is advancing on its mission, especially if that mission is vague and utopian. Therefore, it is important that savvy nonprofits include checks to ensure they don’t get caught up in these traps and have an understanding of where they are going. Even if a nonprofit has evidence of their success, they should continue looking for ways to improve, as it is highly unlikely that their programs are already perfect.
What would this look like? How would we find it in a charity?
To inquire into this metric, one should look at the website of the charity and the testimony of those working for the charity as to what mission they articulate and what plan they articulate for how to accomplish that mission.
Are their plans specific? Can they define outcomes that indicate success and failure as well as rationale for why those outcomes indicate success and failure? Do they care about self-assessment? Are they self-skeptical?
Is the charity actually measuring things? Are they measuring things with good methodologies? Are they measuring things that actually matter or are they just chasing certain metrics because they’re easy (i.e., measuring leaflets given instead of people converted)?
Moreover—and most importantly—does the charity show a willingness to change in light of metrics pointing one way or another? If a leafleting charity found that online video ads were more effective or vice versa, would they make the switch? Has the charity made key changes in its methods before based on evidence collected? Are they actively trying new things and working to collect new information that could help improve their programs?
A charity that is responsive to success and failure should be taking cues, as relevant, from agile program management or lean start-up management, whereby they seek to try out a particular approach or intervention, learn from that approach, and then update their methods before trying again—continuing to iterate their methods in response to testing and evidence. If a charity is hitting many of these points, then they are probably appropriately responsive to success and failure.
Additionally, such a charity will have goals that are SMART—specific, measurable, achievable, relevant, and time-bound. Good charities will have well-targeted goals that they can use to concretely assess where they are, what they want to achieve, and how they will get there. Having a time-bound is especially important as it keeps the charity accountable to their expectations of success.
Criterion #6: Does the charity have strong leadership and a well-developed strategic vision?
Why was this included as a metric?
An effective charity will very likely have strong leadership, as leaders’ values and skills determine its response to any changes in the environment or within the charity. Bad or dysfunctional leadership can spell disaster for the nonprofits with even the most impactful programs if not addressed.
A charity with a well-developed strategic vision will respond to new evidence and changing situations by continuing to pursue the programs that help achieve their goals. Given our own commitment to finding the most effective ways to help nonhuman animals, we additionally look for charities whose strategies are aligned with that goal. A charity that doesn’t share our own values of reducing suffering as effectively as possible may happen to have programs which we think are very good for animals, but could shift their strategy significantly in the future. For example, a group which promotes plant-based diets because of the health benefits to humans would be more likely to decide to allocate resources to programs that didn’t affect nonhuman animals (or perhaps even harmed them) than a group which promotes plant-based diets out of concern for animal welfare or rights.
What would this look like? How would we find it in a charity?
We identify charities with strong leadership and vision by looking to the leadership structure and organizational strategy, examining the composition and activities of the Board of Directors, the charity’s strategic plan, and the way their activities fit into the overall animal advocacy movement and our understanding of social change.
The charity’s goals should explicitly include helping animals and reducing suffering; ideally, these should be of primary concern in their own evaluations of their programs. The charity should also be working in an area which makes strategic sense when considered in the context of the field of animal advocacy as a whole. Charities working in particularly neglected areas, or performing especially well in areas vital to the animal advocacy community overall, do well in this consideration. It is also a good sign if the charity is involved in open dialogue with other advocacy groups and seeks out mutually beneficial collaborations to maximize their impact.
Members of the Board of Directors should have diverse occupational backgrounds and experiences, in order to provide the charity with a variety of perspectives on key strategic decisions. The Board of Directors should assist the charity’s staff by performing the duties typically assigned to a board in the context in which the charity operates, likely including strategic planning and financial oversight.
The charity should have a well-developed strategic plan which is revisited regularly, and which includes both shorter- and longer-term goals. Ideally, the strategic plan should be developed through a formal planning process led by the Board of Directors or senior staff and including input from other stakeholders (e.g., staff, donors, and beneficiaries).
Criterion #7: Does the charity have a healthy culture and a sustainable structure?
What is a healthy culture and a sustainable structure?
A charity with a healthy culture and a sustainable structure is one which is well-managed on an operational level. While this will look different depending on the unique needs of each charity, there are some characteristics of a healthy and sustainable charity which we can look for specifically.
First, a charity should be stable and sustainable under ordinary conditions, and should seem likely to survive the transition should some of the current leadership move on to other projects. The charity should not seem likely to split into factions, or to stop being able to raise the funds needed for its basic operations. Ideally, they should receive significant funding from multiple distinct sources, including both individual donations and other types of support.
Second, a charity with a healthy culture acts responsibly with stakeholders including staff, volunteers, donors, and others in the community. In particular, staff and volunteers develop and grow as advocates due to their relationship with the charity, and they have a work environment free of harassment and discrimination. The charity is transparent with donors and the general public and has a healthy attitude towards diversity—seeking to find and retain staff and volunteers from different backgrounds, since varied points of view improve a charity’s ability to respond to new situations.
Why was this included as a metric?
A charity that is especially effective will work well on an operational level. Healthy relationships among staff and well-developed methods and routines will allow projects to be carried out efficiently, without distractions caused by a lack of organization or recurrent small crises. A sustainable structure also enables the charity and its supporters to make long-term plans. Finally, a charity with a healthy culture builds capacity in the animal advocacy movement as a whole through staff and volunteer training and development.
Some aspects of a healthy culture and a sustainable structure are specifically important in enabling ACE to work with and recommend a charity. For example, transparent charities benefit the movement as a whole by sharing information and enabling others to do stronger work. But in addition, we would not be able to conduct a thorough review of a charity which was not willing to be highly transparent with us, and we would not be able to publish the review unless they were willing to be at least moderately transparent with the general public. We would also have trouble recommending a charity which was unstable enough that it might dissolve or substantially change its structure over the course of the year following our recommendation.
What would this look like? How would we find it in a charity?
The charity should be able to demonstrate sustained fundraising ability without too many drastic budget fluctuations. Ideally, the charity should be able to demonstrate this through multiple avenues—such as individual donations, grants, and revenue-generating programs consistent with their mission.
Leadership should bring on new staff and volunteers using an intentional training structure to build capacity of individual staff and volunteers. New staff and volunteers should efficiently learn how to operate key programs, and multiple staff and/or volunteers should have the training and knowledge necessary to operate key programs. Ideally, staff and volunteers will have histories of taking on additional responsibilities within the charity, and staff who leave the charity will often go on to work with other advocacy groups in positions they would not have been able to fill before their experience with the charity being evaluated. A combination of good documentation and staff training should help the charity remain strong during leadership transitions.
The charity should have staff from diverse backgrounds and with diverse characteristics. To fully benefit from staff diversity, the charity should see diversity in hiring not only as a restorative justice project benefiting employees, but as a resource for the charity. The charity should actively learn from internal diversity, by listening to diverse perspectives and using their insights to better approach challenges. The charity should have written policies outlining a course of action to be followed when addressing any incidents that may occur relating to harassment or discrimination.
The charity should be open with people who request information, ideally even if they’re not intending to donate or move large sums of money. Better yet, the charity may publish reports online for all to see, not just to those who request it. It is a particularly good sign if the charity is not afraid of publishing their mistakes and puts substantial documentation online in the style of AMF, the style of GiveWell, or the style of Charity Science, with full budgets, external reviews, audit reports, documentation of future plans, and other pertinent information. The more information a charity is able to share about both its successes and failures, the better it can help others to repeat its achievements but not its mistakes.
