We are excited to announce the launch of our updated recommended charity quiz. This quiz will allow you to discover giving opportunities that match your interests and values by asking your opinions about some distinguishing features of our recommended charities.
ACE is always looking for new ways to engage with our community and help donors connect with the most effective animal charities. We believe that donors will have the greatest impact by supporting our Recommended Charities and/or our Effective Animal Advocacy Fund. However, we do not rank organizations within the Top and Standout Charity categories. Our aim with the charity quiz is to allow donors to rank ACE’s recommended charities based on their personal interests.
While our in-depth reviews provide the greatest insight into our evaluation process and the strengths and weaknesses of our recommended charities, we recognize that the required time investment can be a barrier for some of our readers. The charity quiz will help you to quickly identify which of our recommended charities match your personal interests, giving you more time to explore those organizations in depth.
This year, in addition to updating our previous charity quiz to reflect our current giving opportunities, we also updated its methodology. The quiz now asks which of our evaluation criteria, outcomes, and types of intervention are most important to you. Since these three considerations are crucial to our evaluation process, we hope that our quiz will now also give you some insight into the thinking behind our evaluation process.
The following information is for those interested in more details about how the quiz works.
The quiz contains four questions.
1. Which of the following factors do you think are most important for comparing different charities?1
This question asks you to rank the importance of our evaluation criteria, which form the basis for all of our charity reviews. A charity’s total score for this question is affected by both your ranking as well as the overall assessment of each criterion, as determined by ACE’s charity evaluation committee.
Your personal score for the criterion is calculated by six less the rank that you assign to that criterion (or zero if the criterion is not ranked). As a result, the criterion that you rank as most important is assigned the highest personal score.
The overall assessment score which the quiz calculates for each criterion is given by the following formula, using values from the overall assessments in our charity reviews:
The performance score from one to three and the confidence score from zero to two are added to each other. The sign of the confidence score is determined by whether the performance is strong, average, or low.
The static overall assessment score is then multiplied by your personal score to form a weighted score for each criterion. A charity’s total score for this question is the sum of these weighted criterion scores.
2. Which outcomes do you think are the most important for helping animals?
This question asks you to rank the importance of the different outcomes for animal advocacy that a charity is trying to achieve. We distinguish between these outcomes in our theories of change for how a charity has an impact on animals, which we use to determine a charity’s performance on Criterion 1 (Does the charity engage in programs that seem likely to be highly impactful?).
Your personal score for each outcome is calculated by six less the rank which you assign to that outcome (or zero if the outcome is not ranked). This score is only used to inform the next question (Which of the following interventions do you think do the best job of achieving these outcomes?).
This question has two purposes: (i) to identify your top two outcomes in order to determine which interventions to present to you in the following question; and (ii) to produce personal outcomes scores, which will affect charities’ intervention scores.
3. Which interventions do you think most effectively achieve these outcomes?
This question asks you to rank the importance of different interventions. (An intervention is a specific type of program that a charity uses to help animals.) These interventions are part of our theories of change, which determine a charity’s performance on Criterion 1 (Does the charity engage in programs that seem likely to be highly impactful?). ACE sometimes publishes research on the effectiveness of different interventions.
The interventions that appear in this question are those that influence the top two outcomes ranked in the previous question.
Your personal score for each intervention is calculated by six less the rank which you assign to that intervention (or zero if the intervention is not ranked).
For every individual intervention, a charity’s review-based2 intervention score depends on how heavily it invests in the intervention (as given by the thickness of the arrows to the left and right of the intervention in our theory of change diagrams), and how much the intervention influences the outcomes you care about (as given by the ranks you assigned in the previous question to each influenced outcome). A thick arrow is worth two points, and a thin arrow is worth one point. The points for each arrow on the right side of the intervention are first weighted by your personal score for the outcome that the arrow points to. Then the points for each of these rightward arrows are added together and finally are multiplied by the point value of the arrow to the left of the intervention on our theory of change diagram.
The review-based intervention score is then multiplied by your personal score to form a weighted score for each intervention. A charity’s total score for this question is the sum of these weighted intervention scores.
We sometimes omit more peripheral interventions from charities’ theory of change diagrams; we gave charities partial points for some interventions like these (the equivalent of a thin arrow to the left of the intervention and half a thin arrow to the right for each influenced outcome). Charities that conduct work in neglected countries were also given points for the Working in Neglected Countries intervention, even if that was not explicitly represented in our theory of change diagram for that charity.
4. Are you interested in supporting ACE’s Effective Animal Advocacy Fund?
This question asks whether you are interested in supporting the Effective Animal Advocacy Fund (EAA Fund), which we think is a good option for many donors to consider. EAA Fund grant recipients include charities and programs that are doing promising work, but that do not necessarily meet the criteria to be evaluated by ACE. Donors who are interested in higher-risk/higher-reward investments may be more drawn to the Effective Animal Advocacy Fund, while donors who are more risk-averse may find our recommended charities to be a better fit. Your answer to this question determines whether the EAA Fund will be added to your list of personal recommendations at the end of the quiz.
Outside of your personal scores and the scores based on ACE’s reviews, charities’ total scores are also weighted to ensure that each of our Top Charities gets recommended to a similar number of users and that each of our Standout Charities gets recommended to a similar number of users. Our thinking behind this weighting is the assumption that even though different recommended charities may appeal to different people, the effectiveness of donating to each of the Top Charities and each of the Standout Charities is roughly equal. We arrived at the current weighting by considering which charities did best with (i) uniform criteria/outcome/intervention weights, (ii) randomized answers, (iii) answers from ACE staff members, and (iv) answer sets corresponding to “test users” built to represent different perspectives in animal advocacy.
We record your answers in order to better understand the priorities of our readers and to improve the weighting of different charities in the future. ACE may later publish aggregated information about quiz responses alongside discussions of these findings.
The source code for the charity quiz can be found at this Github repository.
Respondents may select up to five of the following list items and rank them: “Projected Impact of Programs; Cost of Expansion Plans; Track Record; Cost-Effectiveness; Response to Success and Failure; Leadership and Strategic Vision; Culture and Structure.”