Our recommended charity quiz has been updated to reflect the results of our most recent round of charity evaluations. In the process, we have made some minor adjustments to the way quiz results are obtained, discussed in more detail below. The charity quiz allows you to discover giving opportunities that match your interests and values by asking your opinions about some distinguishing features of our recommended charities.
Animal Charity Evaluators (ACE) is always looking for new ways to engage with our community and help donors find effective opportunities to protect animals. We believe that donors will have the greatest impact by supporting our recommended charities and/or our ACE Movement Grants (formerly “Effective Animal Advocacy Fund”). However, we do not rank organizations within the Top and Standout Charity categories. Our aim with the charity quiz is to allow donors to rank ACE’s recommended charities based on their personal interests.
While our in-depth reviews provide the greatest insight into our evaluation process and the strengths and weaknesses of our recommended charities, we recognize that the required time investment can be a barrier for some of our readers. The charity quiz will help you to quickly identify which of our recommended charities match your personal interests, giving you more time to explore those organizations in depth.
As in last year’s quiz, you are asked which of our evaluation criteria, outcomes, and types of intervention are most important to you. The outcomes that appear in the quiz now reflect the updated Menu of Outcomes for Animal Advocacy, which debuted last year. Since these three considerations are crucial to our charity evaluations, we hope that our quiz will also give you an insight into how we approach the evaluation process.
The following information is for those interested in more details about how the quiz works.
Questions
The quiz contains four questions.
1. Which of the following factors do you think are most important for comparing different charities?1
This question asks you to rank the importance of our evaluation criteria, which form the basis for all of our charity reviews. A charity’s total score for this question is affected both by our overall assessment of each charity’s performance on the seven evaluation criteria (as determined by ACE’s charity evaluation committee) and your ranking of which criteria are most important.
Your personal score for each criterion is calculated by six less the rank that you assign to that criterion (or zero if the criterion is not ranked). As a result, the criterion that you rank as most important is assigned the highest personal score.
The overall assessment score which the quiz calculates for a charity’s performance on a given criterion is determined by the following formula, using values from the overall assessments that appear in our charity reviews:
The formula shows how the overall assessment score is calculated as a sum of a performance score (ranging from 1 to 3) with a confidence score (ranging from -1 to 1). The sign of the confidence score is determined by whether the performance is strong, average, or weak.
The overall assessment score is then multiplied by your personal score to form a weighted score for a charity’s performance on a given criterion. A charity’s total score for this question is the sum of these weighted criterion scores.
2021 Updates: The confidence score for a criterion previously ranged from -2 to 2; it now ranges from -1 to 1.
2. Which outcomes do you think are the most important for helping animals?
This question asks you to rank the importance of the different outcomes for animal advocacy that a charity is trying to achieve. We distinguish between these outcomes in our theories of change for how a charity has an impact on animals, which we use to inform a charity’s performance on Criterion 1: Programs.
Your personal score for each outcome is calculated by six less the rank which you assign to that outcome (or zero if the outcome is not ranked). This score is only used to inform the next question (Which of the following interventions do you think do the best job of achieving these outcomes?).
This question has two purposes: (i) to identify your top two outcomes in order to determine which interventions to present to you in the following question; and (ii) to produce personal outcome scores, which will affect charities’ intervention scores.
2021 Updates: This question now reflects our updated Menu of Outcomes for Animal Advocacy.
3. Which interventions do you think most effectively achieve these outcomes?
This question asks you to rank the importance of different interventions. (An intervention is a specific type of program that a charity uses to help animals.) These interventions are part of our theories of change, which determine a charity’s performance on Criterion 1: Programs. ACE has published research on the effectiveness of different interventions.
If one of our charity reviews’ theory of change diagrams shows that a recommended charity influences one of your top two outcomes (as assessed in the previous question) by means of a particular intervention, that intervention will be asked about in this question. Otherwise, the intervention will not be asked about.
Some interventions which appear as distinct items in our theories of change are similar enough that we don’t ask about them separately in the quiz. For example, our theory of change for The Good Food Institute shows that they work toward increased availability of animal-free foods by legislative advocacy as well as by lobbying. In the charity quiz’s interventions question, users will sometimes be asked to rank the importance of influencing legislation, which is intended to encompass both legislative advocacy and lobbying.
Your personal score for each intervention is calculated by six less the rank which you assign to that intervention (or zero if the intervention is not ranked). When we ask about a broader type of intervention than those which appear in our theories of change, the personal score for the broader intervention determines the personal score for all narrower interventions associated with it. For example, a user’s personal score for legislative advocacy and their personal score for lobbying are both identical to their personal score for influencing legislation.
For every individual intervention that appears on a charity’s theory of change diagram, a charity’s review-based2 intervention score is determined for that intervention by the extent to which the intervention influences the outcomes you care about. This is how a review-based intervention score for a given charity is calculated:
- For each of your ranked outcomes the intervention directly influences (i.e., outcomes connected to the intervention by an uninterrupted arrow on the charity’s theory of change diagram), sum the following:
- The personal score for that direct outcome3
- If the direct outcome you ranked also has an indirect outcome you ranked:
- The fourth root of the personal score for that indirect outcome
- The square root of the product of the personal scores of the indirect outcome and the direct outcome which led to the indirect outcome
For each intervention, its review-based intervention score is then multiplied by your personal score for that intervention in order to form a weighted score. A charity’s total score for this question is the sum of all its weighted intervention scores.
2021 Updates: The quiz previously did not award points to an intervention for outcomes that were indirectly influenced by it. Working in neglected countries is no longer used to identify special interest Standout Charities, so it is no longer included in this question.
4. Are you interested in supporting ACE Movement Grants?
This question asks whether you are interested in supporting our Movement Grants program, which we think is a good option for many donors to consider. Donations to Movement Grants will be distributed to promising projects around the globe working to reduce animal suffering.
Donors who are interested in higher-risk/higher-reward investments may be more drawn to Movement Grants, while donors who are more risk-averse may find our recommended charities to be a better fit. Your answer to this question determines whether Movement Grants will be added to your list of personal recommendations at the end of the quiz.
2021 Updates: The Effective Animal Advocacy Fund was renamed to ACE Movement Grants.
Weighting
Outside of your personal scores and the scores based on ACE’s reviews, charities’ total scores are also weighted to ensure that each of our Top Charities gets recommended to a similar number of users and that each of our Standout Charities gets recommended to a similar number of users. Our thinking behind this weighting is the assumption that even though different recommended charities may appeal to different people, the effectiveness of donating to each of the Top Charities and each of the Standout Charities is roughly equal. We arrived at the current weighting by considering which charities did best with (i) uniform criteria/outcome/intervention weights, (ii) randomized answers, (iii) answers from ACE staff members, and (iv) answer sets corresponding to “test users” built to represent different perspectives in animal advocacy.
Results
2021 Updates: Your quiz results will now highlight how your top choice relates to your answers: Your top choice’s result will indicate if the charity performs strongly on your top-ranked criterion or if it implements a top-ranked intervention that influences your top-ranked outcome.
Collecting information
We record your answers in order to better understand the priorities of our readers and to improve the weighting of different charities in the future. ACE may later publish aggregated information about quiz responses alongside discussions of these findings.
Source code
The source code for the charity quiz can be found at this Github repository.
Respondents may select up to five of the following list items and rank them: estimated impact of programs, room for more funding, cost effectiveness, track record, leadership and culture, strategy, and adaptability.
Note that these scores are not based solely on our charity reviews, however, because they do take into account your personal scores for different outcomes.
In our Menu of Outcomes for Animal Advocacy, decreased consumption of animal products, direct help, and improvement of welfare standards are direct outcomes. Indirect outcomes are those which create change for animals by influencing direct outcomes: increased availability of animal-free products, increased prevalence of anti-speciesist values, and stronger animal advocacy movement.
Leave a Reply