Highlights From Our 2022 Reddit AMA

Thanks for this question. We recognize this is an issue with the way we prioritized countries this year. We used a scoring system to prioritize countries based on the Scale, Neglectedness, and Tractability framework. We weighted tractability as the most important factor (36%), which might have resulted in us favoring the U.S. and Europe. We acknowledge this is a limitation of our method. We will work to improve our proxy for neglectedness and consider how we can give more weight to countries in the Global South.

Although all of our Top Charities are registered in the U.S. and primarily conduct their work there, the nature of their work benefits the movement at a more global level. For example, The Humane League (THL) has offices in Mexico, the U.K., and Japan and does movement-building work in neglected countries as part of the Open Wing Alliance (OWA). Faunalytics produces and summarizes research that benefits the movement more generally, and they have translated articles and completed partner projects with organizations in multiple regions. Wild Animal Initiative works to build an academic field that will lay the groundwork for developing interventions to help wild animals. This work is highly novel, and the interventions developed from it could be applied by organizations on a global scale. Finally, The Good Food Institute (GFI) conducts work in Brazil, India, Asia-Pacific, Europe, and Israel, as well as the U.S.

Of our 11 Standout Charities, six do all (or the vast majority) of their work in the Global South: Dharma Voices for Animals, Sinergia Animal, The Federation of Indian Animal Protection Organisations (FIAPO), and xiaobuVEGAN. Additionally, Mercy For Animals (MFA), Material Innovation Initiative (MII), and Fish Welfare Initiative do a significant amount of work in the Global South.

(Source)

Thanks for this question and your concern about how fair our evaluation process is. This is a topic that we are constantly discussing and always trying to improve.

Although we only evaluate a few animal advocacy organizations, these few organizations have been thoroughly selected from a comprehensive list of more than 600 organizations that we are aware of. We collect data on those organizations and use a quantitative model to narrow it down to a list of dozens of organizations that we carefully discuss and score before inviting charities to participate in our evaluation process. Our method still has some room for improvement, and we hope to refine it next year.

One of the limitations of our evaluation process is its focus on recommending organizations in the Global North. This issue is partially caused by the method we use to prioritize countries, the fact that the animal advocacy movement is more developed in the Global North, and the fact that we are more familiar with the movement in the region. We acknowledge this limitation, and we hope that our Movement Grants program partially addresses this concern by focusing on supporting more organizations in the Global South.

Our evaluation process is focused on recommending organizations that have a positive impact on animals. We think organizations that do not directly help animals (or do not describe themselves as such) can also have a great impact on animals, so we do not necessarily discard them from our process. We consider this a strength of our approach rather than a weakness, but we are interested in hearing what the main issues with this approach are. Please feel free to reach out if you have any particular ideas on how we can make our process more fair.

(Source)

Great question—you’re right that plant-based labels are often used for products that still harm animals, whereas vegan products are completely free of animal products. The difference can often be a strategic move that weighs up how effective a certain approach or intervention is likely to be in terms of the number of animals helped/saved. For example, it might currently be very difficult to get policymakers or other key stakeholders on board with making vegan commitments compared to making plant-based commitments. If charities are to help the most animals with their interventions, advocating for plant-based products may be more effective than advocating for vegan products in some cases.

When we evaluate charities like Dansk Vegetarisk Forening (DVF), we look at different interventions and outcomes of their work as part of our Programs criterion. One of the key outcomes we measure is decreased consumption of animal products, and for each charity we evaluate, we look at their key achievements and estimate how cost effectively they achieve their outcomes. This year, we estimated that DVF has highly effective programs and moderate to high cost effectiveness.

(Source)

Thankfully, the last time we heard from New Harvest, they noted that they have been able to extend their runway through 2023!

Since ACE last evaluated New Harvest in mid-late 2021, and we only re-evaluate charities every two years, we have not completed a comprehensive review for them since they encountered the financial difficulties you noted in your question. Therefore, their review on our website, including their Room for More Funding section, has not changed.

(Source)

One important reason why we recommend Faunalytics is because we see animal advocacy research as a highly promising but neglected cause area. As we set out in this 2021 research brief, advocacy research can impact the priorities set by animal advocates and inform the implementation of interventions, providing knowledge that can help advocates do their daily work. In the longer term, animal advocacy research could grow research fields and increase support from academics, researchers, other social sectors, and members of the public. As such, even small improvements made through research can substantially increase the number of animals helped if the research is applied by many animal advocates. We believe that Faunalytics’ research work is particularly high quality and has the potential to strengthen the animal advocacy movement in a way that will positively impact a huge number of animals.

Of course, the effects of Faunalytics’ programs on animals are indirect and difficult to measure. As with all animal advocacy interventions, there is a real need for a stronger base of evidence on the effectiveness and limitations of animal advocacy research. Fittingly, Faunalytics is playing an important role in filling such knowledge gaps.

Given the limited available evidence to support the effectiveness of any given intervention, we think that animal advocacy is more likely to be successful by continuing to fund a wide range of interventions. We believe that a broad, pluralistic animal advocacy movement is more likely to be resilient—and therefore more impactful—than a narrowly focused one. This belief is reflected in the wide range of work carried out by our Top and Standout Charities—all do amazing work to help animals in a variety of different ways.

(Source)

We agree that there are a lot of organizations doing great work for animals around the globe! During the process of selecting organizations to invite to our evaluation process, we consider a comprehensive list of hundreds of animal organizations that seem to be working in areas that we consider high priority. In part due to capacity constraints, we can only evaluate a few of those charities every year, and some of those charities may decline our invitation. Of the charities that accept our invitation, a few do not perform as well in our evaluation criteria as other organizations, and thus we decide not to give them a Top or Standout recommendation status. This is a difficult decision for us, but we expect that by thoroughly reviewing and comparing these organizations, we direct funds to the most effective ones. At the same time, we are raising the bar in the animal advocacy movement so that organizations strive to help animals as much as possible.

(Source)

It’s very considerate to wonder how you can do the most with your donation. When we evaluate charities, we specifically consider whether or not they could effectively absorb the funding that we expect they’d receive from our recommendation, and we consider that information when deciding their recommendation status. Even then, many of the organizations we recommend may still not have enough money to execute all of their plans. Therefore, all of our recommended charities have room for more funding and are good giving opportunities for donors of any level.

It’s also important for an organization’s financial stability and ability to plan ahead that they have diverse revenue streams, including a broad base of smaller donations, as well as larger gifts and grants from foundations.

If you give to our Recommended Charity Fund, your donation will go toward both our Top Charities and our Standout Charities. Donors who wish to support newer, less established animal advocacy organizations and approaches could consider supporting our Movement Grants program. This program funds groups working on various approaches to animal advocacy, especially those that are underfunded, target large numbers of animals, and are in regions with a relatively small animal advocacy movement.

(Source)

This year, we made changes to our overall decision-making process and evaluation criteria. You can read more about our charity evaluation criteria and evaluation process on our website. In addition to these pages, we will be posting even more details about our methodology for each of our evaluation criteria soon.

(Source)

This year, we provided much more insight into our evaluation process than we have in previous years. The specific numbers we used for effectiveness/impact are detailed in spreadsheets linked in our 2022 charity reviews. We also published a blog post that describes our 2022 evaluation process and how we reached our recommendation decisions. We hope these details help answer your questions about the effectiveness/impact of the charities we reviewed.

You raise an interesting point about “dollar per chicken-adjusted life year.” While we’d love to be able to use animal lives improved or saved as our outcome measure, there simply aren’t enough data available on effective animal advocacy to make confident assessments of cost effectiveness or impact based on quantified outcomes for animals. Therefore, ACE does not currently make these types of estimates for our recommended charities. For more context: This year, we aimed to find a compromise between a fully quantitative model and a more qualitative assessment, which we did by using quantitative scores as a proxy for cost effectiveness rather than estimating the number of animals spared per dollar invested. Based on past experience, fully quantitative models are more challenging to implement for comparing the cost effectiveness of different animal advocacy interventions because of the greater uncertainty that comes with measuring and influencing animal suffering/wellbeing (compared to fields like human health).

(Source)

All else being equal, we think the best (most morally good) action is the one that results in the highest net welfare, so reducing animal suffering is ultimately our priority. We don’t currently use moral weights to weigh the relative importance of animal species, type of suffering, or reducing suffering vs. saving lives, although we do believe that the lives of many animals (e.g., animals in factory farms) are so fraught with suffering that it would be better for these animals not to exist at all. We might consider using more explicit moral weights in the future.

However, individual researchers’ opinions of moral weights may have been subjectively factored into our assessment of animal groups. As outlined in our 2022 evaluation process page, ACE research team members scored the Scale, Tractability, and Neglectedness of each animal group, intervention, outcome, and country on a 1–5 scale, which we then used to calculate overall team averages. These averages then informed each charity’s Programs score, impacting our decision to recommend each of them or not.

We also don’t currently use quantitative estimates of the level of suffering averted in our cost-effectiveness assessment. This year, we used a qualitative scoring framework instead. We rated the interventions charities use to reduce suffering as well as charities’ recent achievements, taking into account the amount of funding spent on achievements and contextual information (e.g., animal group affected, scope of the achievement, etc.). This is where the level of suffering averted comes into play: We prioritized interventions and achievements affecting larger numbers of animals with higher levels of suffering. These ratings factored into the final numerical cost-effectiveness score (ranging from 1 = very low to 5 = very high).

(Source)

Yes, we aim to be as transparent as possible about our evaluation process, although sometimes we remove confidential information from our reviews and supplementary materials at a charity’s request. Evaluating animal charities can require a high level of confidentiality because organizations sometimes use institutional and pressure strategies that need to be kept confidential from the public or policymakers until the time is right.

However, for every charity we evaluated this year, we now publish spreadsheets with the details of our Programs, Cost Effectiveness, and Room for More Funding analyses. We also publish spreadsheets with the details of how we scored the priority levels of different groups of animals, countries, outcomes, and interventions. All spreadsheets are linked in our 2022 charity reviews. Because of the complexity of our cost-effectiveness analysis, we added notes to the spreadsheet explaining the input of each column. (See an example spreadsheet here.) Please do reach out if you have specific questions about what went into these analyses.

We are also constantly looking to improve our methods and have scheduled a meeting with GiveWell staff to learn more about their processes and assess which of their methods could apply to evaluating animal charities.

(Source)

Thanks for these questions! Wild animal suffering is one of our high priority areas, especially because of its great scale and high neglectedness. One of our Top Charities is Wild Animal Initiative, which is doing work to build an academic field that will lay the groundwork for developing interventions to help wild animals. This year, when scoring the priority of animal groups on a 1–5 scale, wild animals got a score of 4.1, which we categorized as very high priority. This year, we evaluated a couple of organizations that work on helping wild animals: Animal Advocacy Careers, which supports organizations working to help wild animals, and Aquatic Life Institute, which works to help wild-caught fishes. We did not evaluate more organizations focused on helping wild animals because we are aware of only a handful of these organizations. We would like to see and evaluate more organizations working on this cause area in the future.

We considered wild animal advocacy to an extent when designing our Menu of Outcomes and Menu of Interventions, but we recognize our method may be more biased toward farmed animal organizations. For example, our assessment of countries currently only applies to farmed animal organizations. We hope to incorporate wild animal advocacy into our charity evaluation and Movement Grants programs more in the future.

(Source)

Any of our current recommended charities, whether from 2021 or this year, are fantastic options for supporting effective animal charities and helping relieve the suffering of numerous animals. We stand by all our recommendations and don’t distinguish between charities that retained their recommendation status from last year and those that were recommended this year.

We appreciate your suggestion to provide a midterm update for recommended charities in their second year. We do provide all monthly donors to our Recommended Charity Fund with updates on charities’ achievements throughout the year. To avoid recency bias with annual donors and to ensure we remain aware of what’s happening at recommended charities, it does seem prudent to request a midterm update from them and share our findings with supporters. This is something we’ve considered doing, so we’re grateful to hear you give us that feedback.

(Source)

We are really grateful for the time and expertise that members of the effective altruism community have put into making suggestions for how we can improve our work, and once we have more capacity for reflection after the giving season, we will follow up to learn more and continue the constructive dialogue.

With regard to these questions, this year, we introduced a scoring framework to rate the relative priority of interventions charities use. We now make explicit which interventions we think are the most effective in reducing animal suffering. (See our Menu of Interventions page and prioritizing interventions spreadsheet.) Indeed, advocates and charities don’t always agree with our assessment, but as you rightly point out, we place a high value on being as impartial and evidence-driven as possible.

(Source)

This year, we categorized all interventions that organizations use into a number of types, and individual researchers then scored the priority levels of those types. We used a version of the Scale, Tractability, and Neglectedness framework and consulted our research briefs and other relevant research on intervention effectiveness to do so. A limitation of this approach was our lack of capacity to provide more details on each score. We hope to work on this and provide more details in the future. For now, we can share some reasons behind our capacity building scores: a wide range of intervention types within the capacity building category, lack of evidence of intervention effectiveness (especially in the shorter term), and low tractability due to the nature of capacity building interventions. However, we think capacity building interventions become increasingly important as we think longer term, and although we assessed capacity building for improving welfare standards and increasing the availability of animal-free products as moderate priority, we assessed capacity building interventions to strengthen the animal advocacy movement as high priority. See this spreadsheet for more details.

(Source)

We currently rate corporate outreach interventions and policy interventions that aim to improve welfare standards as very high priority. We believe corporate campaigns to improve welfare standards are likely to be highly cost effective overall. Simply counting the number of commitments would miss a lot of nuances, so as much as possible, we factor in contextual information (e.g., animal species affected, size of the group making the commitment, etc.) when rating the cost-effectiveness of welfare reform programs. We look forward to improving our methodology further in the coming years with the help of feedback from the community.

Note: We intend to publish an update on changes to our cost-effectiveness methodology in 2023, which will include any changes made as a result of the linked post and future correspondence with the author. For those interested in reading the detailed resolution, please check our EA Forum posts and/or our blog in the coming months.

(Source)

Our research team spends the majority of its time on our charity evaluation program, which aims to identify the best ways to help animals. Of our seven-person research team, only one person currently works on the Movement Grants program, and during the evaluation process, they also assist with evaluations. The research team is not involved in raising funds for our Recommended Charity Fund. This fund is distributed biannually to our Top and Standout Charities according to a formula and therefore requires minimal grant management.

ACE’s mission is to find and promote the most effective ways to help animals. Our research team focuses on the first part of our mission, and the communications and philanthropy teams are largely responsible for the second part: promoting the most effective ways to help animals. That is part of our mission because just knowing what is best and who is doing the best work is not enough. Without resources and support, organizations cannot do the necessary work.

Though there are multiple funding groups in the effective animal welfare space, this cause area is still largely underfunded, and most funding comes from a very select group of donors. ACE has a unique position in the space to encourage more people to give (including those from outside the EA community) and identify gaps in the movement to encourage plurality. That plurality seems necessary because, unlike human causes, there is much less data available and much greater uncertainty about the most effective programs across different times and regions. To be able to evaluate the most effective ways to help animals, we try to ensure that multiple interventions exist and are tested. In that way, our Movement Grants program supports our Charity Evaluations work and is informed by it.

(Source)

Sanctuaries do some great work and can be really valuable. We feel sanctuaries’ greatest value stems from their educational work and that they can increase their impact by focusing more on this area.

Given that direct care for animals can be very expensive, we currently believe that other interventions are likely to be significantly more cost effective. This is why we consider Direct Help to be the lowest priority of the six outcomes listed in our Menu of Outcomes, and why it does not tend to be a significant focus of our recommended charities. As with all of ACE’s work, we will continue to review the relevant literature. If research comes to light that indicates sanctuaries may be more effective than we had previously thought, we will update our stance accordingly.

(Source)

It’s true that even highly effective charities have more and less effective programs. However, for both our Cost Effectiveness and Programs criteria, we factor in how much funding a charity spends on their programs, including the percentage of funding spent on highly (cost) effective programs. Therefore, we are confident that the charities we think have highly impactful programs and evaluate as highly cost effective spend a large proportion of their funding on highly effective work.

We currently do not recommend or make restricted donations to charities to only be spent on a specific program for a number of reasons. First, while we spend considerable time reviewing existing empirical evidence for different interventions, we do not yet have strong enough evidence to be highly confident that one intervention is effective enough to warrant restricted donations. Second, there may be an issue with fungibility, meaning that restricted donations to one program can free up resources to spend on other programs. Third, programs can often not be viewed in isolation because they support and depend on each other (e.g., media work can be used to support policy campaigns).

All this being said, we do include an assessment of each of a charity’s programs in our reviews. We hope our reviews provide charities with valuable feedback about which programs seem highly impactful and cost effective and which may have room for improvement or could be scaled down.

(Source)

We don’t currently quantify our level of confidence in our estimates and recommendation decisions, but we might consider this for future evaluations as we improve our methodology.

(Source)

When we ask charities for their financial information, we ask how much of their expenditure is used for overhead as opposed to programs. We then divide these costs proportionally among their programs to assess their cost-effectiveness. This is because certain amounts of overhead is necessary for charities to do their work effectively and sustainably, and we don’t believe that the share of overhead costs negatively impacts overall effectiveness. For example, operations roles such as HR and fundraising are considered overhead, and it can be cost effective for organizations to invest into infrastructure and staff retention.

(Source)

Thanks for this good question. We agree this is a limitation of our current approach to measuring country neglectedness, and we plan to improve our approach in the future. As you suggest, a more robust approach would account for factors such as the size of organizations and number of individual animal advocates present in the country. We are also considering how we could incorporate other frameworks into our neglectedness scoring, such as World Animal Protection’s Animal Protection Index, which ranks 50 countries around the world according to their animal welfare policy and legislation.

We are also exploring making broader improvements to our country prioritization process: For example, when scoring countries based on scale, we are considering accounting for projected consumption of animal products in the future, as opposed to solely looking at current rates of animal product consumption. We are also considering how to best account for the interdependencies between different factors—recognizing, for example, that in any given country some interventions are going to be more neglected than others. We always welcome input, so please do feel free to reach out to us with any further thoughts and suggestions.

(Source)