Process Leading to our 2021 Recommendations

Updates to Our Evaluation Process

We made several improvements to our charity evaluation process this year, including a more rigorous selection process, updated criteria for assessing charities, and an expanded and more systematic verification process.

We updated our 2021 charity selection process to be more comprehensive and rigorous by compiling a master list of about 2,800 charities from around the world and using a quantitative model to select charities based on important factors. We also updated our criteria for evaluating charities with the intention of better capturing and communicating the reasoning behind our recommendation decisions. To increase transparency in our assessments and improve readability, we decided to focus on the most relevant criteria for our recommendation decisions and reduce the number of criteria included in our reviews. In addition, we divided each criterion into the three main sections—Method, Information and Analysis, and Assessment—to clarify which areas of the review explain our method, our analysis, and our assessment per criterion.

ACE prioritizes using accurate and reliable evidence in our work. This year, we continued to intensify our verification process and devoted a slightly larger proportion of our time to verifying claims reported by charities. We relied on publicly available information, internal documents, media reports, and independent sources to verify claims, and we also followed up with charities for further information or details. For each charity, we verified at least one key result per program.

Our Selection Process

As outlined above, we greatly expanded our charity selection process this year. We began by compiling a master list of animal charities from several different sources, totaling approximately 2,800 charities. We then filtered down the list by removing charities with key words in their name that indicated they were unlikely to work in a high-priority cause area. We then collected additional data on the remaining charities, e.g., whether they work to reduce the suffering of farmed animals, whether they operate a sanctuary, whether they work to help wild animals, or whether they don’t do any of those activities. From this process, we identified around 700 farmed animal charities and around 40 wild animal charities.

In order to further refine the list of farmed animal charities, we used a quantitative model that prioritized charities based on the outcomes they work toward, the regions they work in, and the specific animal group(s) their programs target. Once we ran the model, we performed some rationality checks to make sure the model was working as expected. The evaluations committee then voted on which of the top 200 charities we should consider for selection, and through a process of iterative discussion and voting, selected charities to invite to review.

We aim to review around 15 charities each year. This year, we invited 25 charities to participate in the evaluation process, sending each of them a copy of our Charity Evaluation Handbook. Twelve charities declined to be reviewed, resulting in a total of 13 charities to evaluate: our four Top Charities from 2020, two Standout Charities that were last evaluated in 2019, and seven other charities that we had not evaluated in at least three years.

Evaluations

As part of the evaluation process, we asked each charity to provide information and documentation about their ongoing programs, accomplishments, finances, leadership, and culture. To assess workplace culture, we distributed a survey to each charity’s staff members and volunteers.

Once drafted, we sent the reviews to the charities for feedback and approval. Before granting approval, charities were given the opportunity to request edits, including removing confidential information or correcting factual errors. That said, all of our reviews represent our own understanding and opinions, which are not necessarily those of the charities reviewed. This year, all 13 charities for which we drafted reviews agreed to have their reviews published.

Recommendations

After drafting each comprehensive review (but before receiving approval from charities), the evaluations committee held several meetings to discuss the selection of Top and Standout Charities.¹ Initially, they held a meeting per charity to discuss each one in depth, identify areas of concern or disagreement, and hold a vote to gauge a tentative recommendation decision. Once the committee had held meetings for each charity, they held another meeting to look at all of the tentative recommendation decisions and ensure that each charity had been considered fairly. There was substantial initial agreement on the status of some charities but not others. The evaluations committee then presented their findings to the rest of staff and board members to gather feedback on the tentative decisions as well as some particular areas of uncertainty. Following this discussion, the evaluations committee held a couple of additional meetings to integrate the staff and board’s feedback and finalize the recommendation decisions, revisiting decisions for which the votes were split.

In the end, we selected three Top Charities. We think that, overall, each of our Top Charities perform well on our evaluation criteria. They each conduct effective programs, are able to make use of additional funding, and have a sustainable work culture. The evaluations committee had either consensus or a majority of supporting votes for each of the selected Top Charities.

We also selected seven Standout Charities. Our Standout Charities are those that we did not select for a top recommendation, but nonetheless chose to recommend to our supporters because we think they are promising. We think that donations to these charities seem likely to have a relatively high expected value. The evaluations committee had a majority of supporting votes for each of the selected Standout Charities.

Participation Grants

We offer our sincere gratitude to each of the charities that we evaluated this year. Participating in our process takes time and energy, and we are grateful to organizations’ willingness to be open with us about their work. To that end, we awarded participation grants of $2,000 to charities that we evaluated this year. For those charities affected by issues with the culture survey, we awarded an additional grant of $1,000–$2,000, depending on their size, to cover the additional time required for staff to retake the survey. These grants are not contingent on publication; we award grants to charities whose reviews we do not publish, assuming they made a good faith effort to engage with us during the review process.

1. The one exception is one member of the research team, who was not at all involved in the evaluation process for one charity due to a conflict of interest.