Process for Evaluating Charities and Making Recommendations
ACE’s goal in evaluating animal charities is to find and promote the charities that work most effectively to help animals. We want the charities we recommend to be very strong, both in terms of the programs they choose to implement and in their characteristics—such as the way they make decisions and how well they find, develop, and retain skilled employees and volunteers. We promote the strongest charities we have evaluated as our Top Charities, and we recognize excellent charities about which we are slightly less confident, or which are slightly less effective, as Standout Charities. We do not rank or rate charities below these levels.
Our evaluation process consists of several rounds in which we consider progressively fewer charities in progressively greater detail. We give basic consideration to a very large number of organizations and then conduct comprehensive reviews of a small number of likely candidates for our recommendations.
After conducting these rounds of evaluation, we update our recommendations and the reviews published on our site. All our Top and Standout Charities have received comprehensive reviews, but not all charities for which we have written comprehensive reviews are in these categories.
This is the first round of our recommendation process. It is intended to (i) generate a list of animal charities and their focus areas and (ii) improve ACE’s overall understanding of the animal charity landscape. We need to have a sense of the types of animal charities in existence, and of how prevalent each is, in order to make strategic decisions about our research in general—as well as the evaluation process in particular.
Which organizations are considered?
Any organization can request to be considered in this round. We also maintain an internal list of charities to consider based on recommendations we have solicited from experts and the general public, other lists of animal charities maintained online, and charities we have encountered at conferences and through other means. We consider every organization that has requested consideration since the previous recommendation process, as well as most other charities on our list. We do not always consider organizations that provide direct care to animals—particularly companion animal shelters and rescues—because we find that these organizations are usually only able to help relatively few animals with a given amount of resources. We also do not consider charities if, from previous interactions with their leadership, we believe they would prefer not to be considered in our recommendation process.
What do we do to research each organization during this round?
We search for and visit each organization’s website to determine their general focus area and methods. We record basic information about the organization for future use. Our research in this round addresses the following questions:
- Is this a currently active animal advocacy charity with a presence on the internet? If a charity has no web presence, it will be difficult for us to learn enough about them to seriously consider recommending them, and our audience will have a hard time verifying our review and even donating to the charity.
- Is the organization within our scope of evaluation? ACE evaluates charities and nonprofits that work to benefit animals. We do not require charities to be registered in any particular country or under any particular framework; we would even consider evaluating a for-profit company if it had room for funding by individuals and strong evidence of cost-effectiveness in helping animals. We do not evaluate charities whose primary activity is making grants to other charities we could or do evaluate; this excludes some but not all foundations.
- What language or languages does the charity use on their website? ACE is based in the United States; we conduct the majority of our work in English. We look for charities to have a version of their website available in English; this signals that we will have an easier time learning about and communicating with the charity than if they operated only in another language. It also allows our audience to read about the charity in their own words. We also consider evaluating charities that appear very strong in other ways but do not have an English website, only if the charity and relevant ACE staff have a common language available. We do not currently have the resources to have entirely translated/interpreted interactions with any charity.
- What does the charity seek to accomplish, and what general tactics do they use to achieve this goal? These are big questions, and at this early stage of the process, we only try to understand them in broad terms. For example, we expect to be able to distinguish between a charity that provides sanctuary for farmed animals, one that provides sanctuary for big cats, and one that advocates for farmed animals by lobbying legislators. We do not necessarily expect to be able to distinguish more specific divisions of resources, such as between one charity that has a major program advocating for animals used in laboratories and a side program advocating for farmed animals, and another charity that has a major program advocating for farmed animals and a side program advocating for animals used in laboratories.
- Does the charity advocate or participate in violence or the destruction of property? ACE does not endorse violence or property destruction as a means to help animals. We make an exception for very small amounts of property damage in support of a larger project, such as when a lock is broken in order to take footage of a farming operation.
What do we publish as a result of this first round of our evaluation process?
We list charities that fulfill our criteria for basic consideration on our website with icons denoting their focus areas and links to their websites.
If we have considered a charity at only this level and it appears to be within our scope of evaluation, it is listed as “Considered” on our complete list of charities.
This is the second round of our evaluation process. It is intended to allow careful and detailed reflection on the most promising charities, bringing us a step closer to selecting our Top and Standout Charities.
Which charities are considered?
At the start of the recommendation process, we set a target number of charities to investigate in this round, based on our available resources. We always include the Top Charities from the previous review process. We evaluate Standout Charities every two years. We consider other charities we have previously included in comprehensive review rounds, based on how likely we think we would be to recommend them.
At this point, we are dealing with a relatively small number of charities. Some of the factors we consider are:
- Cause area. The area a charity focuses on is usually the main factor we use in deciding whether or not to include it in our round of comprehensive reviews. We prioritize evaluating charities working in areas where (i) large numbers of animals are involved and (ii) relatively little attention and funding is directed. In practice, this means that we evaluate many charities that focus on helping farmed animals, as there are so many farmed animals compared to other types of animals that animal charities typically focus on. We also evaluate charities that focus on increasing wild animal welfare (as opposed to charities focused on species or habitat preservation).
- Methods. The general methods a charity uses also have a significant impact on its cost-effectiveness. In particular, charities that rescue, rehabilitate, or provide sanctuary for individual animals are likely to spend much more per animal helped than charities that focus on creating widespread change through more leveraged means. Accordingly, we prioritize evaluating charities that focus on creating change through institutional outreach to governments and corporations, public outreach, and research and technological advancement.
- General likelihood of receiving a recommendation. We also consider other factors that affect whether a charity is likely to receive a recommendation from us as a result of our review, to the extent that we’re aware of these factors. If we have written a comprehensive review for a charity in recent years and not recommended them, we are not likely to write another review unless either some of our thinking on relevant issues has changed or we have reason to believe that the charity has changed in a relevant way.
- How much would we learn from conducting the review? We will generally learn more from evaluating a charity the first time than from evaluating a charity for the second time, unless the repeat charity has undergone serious changes recently or we have changed how we think about something relevant to their work. We will also learn more from evaluating a charity whose programs are very different from the programs of other charities we have included in comprehensive review rounds than from evaluating one whose programs are very similar to programs we have already investigated in detail.
Charities’ involvement in our comprehensive review process is voluntary and highly involved. As such, there are instances in which charities we intend to include in this round chose not to participate. With this in mind, we use the previous factors to select backup charities as well.
What do we do to research each charity during this second round of our evaluation process?
Once we have decided we would like to conduct a comprehensive review of a charity, we contact that charity, send them the updated Charity Evaluation Handbook to explain our review process in more detail, and eventually ask them a series of questions related to our charity evaluation criteria using written questionnaires. We read the materials submitted to us and other materials published by or about the charity we are evaluating, and we discuss each charity on several occasions. Sometimes we return to our contact at the charity with additional questions. We also send a culture survey to the charity’s staff in order to more fully understand the charity’s culture and work environment.
For charities we have previously conducted comprehensive reviews of, the process is very similar to the process for new charities. We ask the same questions to all charities, use the same evaluation criteria, and make an active effort to be as consistent as possible across all charities under review, while still considering the unique circumstances of each particular charity.
The research team solicits feedback from the board evaluations sub-committee and the Executive Director on each criterion in the comprehensive reviews.
What do we publish as a result of this round?
We write a detailed review for each new charity that participates in our comprehensive review round, and we update our previous review for charities we have written comprehensive reviews for already. The review includes a general summary section followed by sections explaining how well we think the charity performs in each of our evaluation criteria. Because we use information in our comprehensive review round that may not already be publicly available, we also publish charities’ responses to our questions as supplementary materials. We send our review and other supporting materials to the charity for approval prior to publishing.
Because these reviews rely on information that may be confidential, we sometimes make substantive changes to our reviews as a result of feedback from the charity concerned, in order to protect private information. We also correct factual errors or alter wording or emphasis—without affecting the substance of the review.
If the charity concerned agrees, we publish the comprehensive review and approved supporting materials on our website and list the charity as “Comprehensive.” If the charity does not agree to the review being published, we list the charity on our site as “Declined to be Reviewed/Published” and don’t publish the review or supporting materials (unless the supporting materials are approved separately).
ACE’s Evaluations Team and the Executive Director each individually decide whether each of the charities for which we’ve conducted a comprehensive review should be a Top Charity, a Standout Charity, or neither—or if we are not certain. We compare lists and discuss each charity. In some cases, we are very certain and agree with each other; in others, we disagree, or some of us remain uncertain about the optimal outcome.
After an extensive conversation on the subject, we reach a decision about each charity. We continue to talk over the next few days, however, to make sure that a large majority agrees with the decisions. We’re limited in how much detail we can provide about this decision process since most of it has to do with specific aspects of individual charities. We know how we plan to categorize each charity before we reach out to any of them about the publication of their review.
As of 2020, we have four Top Charities and nine Standout Charities. We don’t place a cap on the number of Standout Charities we can have in a given year. We want the number of Top Charities to remain small so that our recommendations provide a clear call to action. We don’t expect the number of Top Charities to grow substantially in the future.
We notify charities that they will be a Top Charity, Standout Charity, or that they did not receive a recommendation, and we ask them to approve the materials we want to publish as a result of their participation in our round of comprehensive reviews. We ask Top Charities to work with us to set up a system for tracking donations directed to them through ACE’s review so that we can understand the impact of our recommendations. This includes either including a check box on their donate page where donors can acknowledge ACE’s influence on their donation, or tracking unique URLs resulting from ACE links to the charity’s website. Additionally, we recommend that charities have an online donation platform with the capacity to accept donations in USD, as U.S. donors are more likely to complete a transaction that doesn’t involve a currency conversion. We also publish shortened versions of our reviews for Top Charities on our site, linking to the longer versions. This allows visitors to our site to receive a brief, clear summary of why we have recommended a charity, and to choose whether to proceed to reading our full evaluation so that the amount of information presented at one time is not overwhelming.