This is an archived version of the Intervention Evaluation Process that was used prior to October 2017.

Evaluation Criteria for Animal Advocacy Interventions

There are many ways for an animal advocacy organization to pursue its goals, and choosing from these options requires understanding a number of quantitative and qualitative factors. Animal Charity Evaluators (ACE) finds it highly useful to standardize the methods by which proposed interventions are evaluated.

This evaluation form tries to draw out as much information as possible, including qualitative evaluations and rough quantitative estimates of the interventions’ impacts. However, because the field of animal advocacy research is still very young, we expect that our understanding will remain incomplete in many cases.

The evaluation is organized into five areas of focus. (I) type of intervention, (II) certainty of success, (III) barriers to entry, (IV) expected indirect effects, and (V) expected direct effects and overall efficiency analysis. Areas II through III might be of particular interest to organizations deciding what sort of intervention to pursue. Areas IV and V estimate the expected results from staging the intervention, both direct and indirect, and conclude with a calculation sheet that attempts a cost-benefit analysis.

Because we expect the cost-benefit calculations to include significant amounts of error and uncertainty, it is important to supplement these with additional reasoning regarding the likely effectiveness of the interventions. Such additions are especially valuable where they use different evidence than that which is primarily responsible for our cost-effectiveness estimate, since when only weak evidence is available, many independent lines of evidence can be analyzed, and if all agree, a strong conclusion can still be made. As an illustration of the importance of including these auxiliary measures, an intervention with high barriers to entry (e.g. very large startup costs) might not be possible for some organizations, while an intervention with high uncertainties of success (i.e. higher risks) is likely to also require more estimations in our evaluations, and thus be prone to more error in our calculations.

Here, we discuss each area briefly. A more detailed version of this form including questions to help guide the analysis, the full calculation sheet, and discussion of possible sources of error is also available.

Table of Contents

• Area I: Type
• Area II: Certainty of Success
• Area III: Barriers to Entry
• Area IV: Expected Indirect Effects
• Area V: Expected Direct Effects and Overall Efficiency Analysis

Area I: Type

There are a number of ways to classify a proposed intervention. This section briefly addresses several of these variables as a way to identify distinctions between intervention types that may be relevant. Variables included here are:

• Activism philosophy
• Activism approach
• Target demographic
• Target animal(s)
• Size of campaign
• Time delay between intervention and results
• Increase/decrease in human suffering
• Characteristics of organization conducting intervention
• Appeal to other organizations

Area II: Certainty of Success

This section focuses on the riskiness of an intervention. If efficiency is held constant, riskier campaigns offer a relatively low chance of achieving a relatively high level result. If our estimates of expected value were perfect, we might not care about certainty of success except as it affects organizations’ willingness to attempt a campaign. However, we will not be able to provide perfect estimates of expected value, because in the real world we can never have perfect information. Our estimates of expected value are particularly likely to deviate from reality in the case of risky campaigns, since they succeed only some of the time. Additionally, based on a finite and often small number of attempts, we cannot know the actual success rate with great precision. We consider several sources of information about how risky an intervention is and how good our efficiency estimate is likely to be.

II.1 Precedents

The success of future campaigns can be estimated in part by the success of past similar campaigns.

II.2 Flexibility of Goals

Campaigns that have lofty, all-or-nothing goals are inherently riskier than campaigns with continuous spectrums of potential positive outcomes.

II.3 Scientific Certainty

There are a variety of scientific disciplines that study issues relevant to the success of animal activist interventions. The extent to which scientists have succeeded at understanding factors relevant to a particular intervention affects our certainty that the intervention will have a positive effect.

II.4 Other Uncertainties

These include all uncertainties that have not yet been discussed. Some may be highly specific to the particular intervention. We also include consideration of the number of steps between successful accomplishment of the intervention’s immediate goal and material benefits for animals, as long causal chains can easily be disrupted by unforeseen factors.

Area III: Barriers to Entry

There are a myriad of factors to consider when evaluating how difficult an intervention will be to stage. This area of the evaluation examines the complexity of human skill required, the degree of work intensity needed, and the difficulty involved in obtaining the required material resources. This is not meant to duplicate the valuation mechanism laid out in Area V (Efficiency), but rather to provide a separate metric to measure the challenges inherent in putting on a campaign, with no reference to a conversion to fiscal units.

III.1 Skill Required

It is important to know what sort of expertise is required from a campaign’s staff. The more specific knowledge and/or special skills required, the more difficult a campaign will be. For example, a campaign to challenge an anti-whistleblower law will need significant legal expertise and political skill.

III.2 Work Required

The calculation of work required can be defined as a calculation of the intensity of effort required from an intervention’s staff members.

III.3 Resources Required

It is further useful to examine the complexities involved in obtaining the necessary resources for an intervention. The procurement of materials for an intervention may require difficult-to-obtain permissions, such as copyright waivers, or an extensive search for uncommon objects.

Area IV: Expected Indirect Effects

Oftentimes, a campaign will have effects that reach beyond its intended goals. These may be positive (i.e. in-sync with the ethics driving the campaign) or negative (i.e. working against the change effected by the campaign). There are multiple factors to consider when evaluating external influence. These effects are especially difficult to estimate precisely because they are often diffuse and because even when campaigns have carefully tracked their progress towards their explicit goals, they may not have attempted to track other effects. However, diffuse and indirect effects of some campaigns may be significant parts of the campaign outcome, so we consider any evidence about these effects as a factor in evaluating interventions.

IV.1 Positive influence

Good externalities may come from a campaign, including by the message of the campaign being spread through a chain effect, the campaign inspiring other activist work, or the public becoming generally more educated about issues related to the campaign.

IV.2 Negative influence

Bad externalities may also arise from an intervention, including through a campaign alienating potential allies or through the outcome of the campaign actually producing more harm than good for animals due to factors unforeseen by the organizers.

Area V: Expected Direct Effects and Overall Efficiency Analysis

This area of evaluation is designed to produce a single ratio of dollars spent per unit of suffering reduced or per number of animal deaths averted. We believe this is an enormously important factor to consider when planning an intervention, as different types of campaigns can have vastly different cost/benefit ratios. Given the limited amount of funds available to an organization of any size, this section explores ways of making sure that those resources are used optimally. We break down this calculation into component expenditures and results. If we could rely entirely on the results of this section, for many purposes we could omit the other sections of the evaluation entirely. However, in reality our estimates of both costs and expected results will be fallible, so reasoning about an intervention’s likely effectiveness from other perspectives (as above) allows us to be more confident of our conclusions than we could be about a direct efficiency analysis alone.

This is an archived version of the Detailed Criteria for Evaluating Interventions used prior to October 2017.

Intervention Evaluation Form

The following document contains a template for evaluating virtually any intervention designed to help animals. We use this template to guide our intervention research and make it available for others to use in comparing interventions on their own. Observations about the intervention being evaluated can be inserted in the Evaluation and Error-tracking subsections for each Area. 

There are many ways for an animal activism organization to pursue its goals, and choosing from these options requires understanding a number of quantitative and qualitative factors. Animal Charity Evaluators (ACE) finds it highly useful to standardize the methods by which proposed interventions are evaluated. This is why we use the following evaluation criteria. The criteria are divided into five areas of focus. For each criterion, we propose a method for summarizing everything that can be known about a given intervention. We hope that this will allow us to compare various campaigns across multiple dimensions and thereby maximize the reliability of our recommendations.

At the moment, there are many unknowns regarding the science, economics, and sociology surrounding animal activism interventions. Much of what is now known about how campaigns translate into reductions in animal suffering comes from small samples, anecdotes, or estimation. For this reason, this evaluation tries to provide as much information as possible, while still recognizing the limits of this young field of research. We do this by providing room for significant qualitative evaluation, while also attempting to give rough quantitative summaries of our work.

The evaluation is organized into five areas of focus. (I) type of intervention, (II) certainty of success, (III) barriers to entry, (IV) expected indirect effects, and (V) expected direct effects and overall efficiency analysis. Areas II through III might be of particular interest to organizations deciding what sort of intervention to pursue. Areas IV and V estimate the expected results from staging the intervention, both direct and indirect, and conclude with a calculation sheet that attempts a cost-benefit analysis.

Theoretically, this final cost-benefit consideration, which estimates the expected return on an organization’s investment, is the only important piece of this evaluation template. However, it is important to supplement this with additional reasoning regarding the likely effectiveness of interventions because we expect these calculations to include significant amounts of error and uncertainty. Such additions are especially valuable where they use different evidence than that which is primarily responsible for our cost-effectiveness estimate, since when only weak evidence is available, many independent lines of evidence can be analyzed, and if all agree, a strong conclusion can still be made. Additionally, given the diversity of organizations, with different resources, goals, and tolerance for risk, we thought it would be useful to give a more holistic picture of what each of our examined interventions entails, while still providing our best guess as to how efficient each one is at reducing animal suffering. As an illustration of the importance of including these auxiliary measures, an intervention with high barriers to entry (e.g. very large startup costs) might not be possible for some organizations, while an intervention with high uncertainties of success (i.e. higher risks) is likely to also require more estimations in our evaluations, and thus be prone to more error in our calculations.

Each area of interest includes an evaluation section with specific questions as well as areas for free-response. Additional forms allowing for summary of the evaluation overall are provided at the end of the document. We suggest using the provided forms during the evaluation process to ensure that all areas have been addressed.

This evaluation process is not without subjectivity. It also requires imperfect estimates of quantitative values. We hope that by publishing the materials used in conducting evaluations, these subjective judgements and estimates will be transparent to our audience. In the case of specific interventions, difficulties in evaluation may arise from:

• limited data regarding certain aspects of past campaigns,
• limited information about the connection between the immediate goals of the campaign and help for animals (e.g. neuroscientific or economic questions that have not been answered),
• biases in the information sources we find,
• limited staff of evaluators (limited ability to cross-check judgements),
• limited time in which to conduct the evaluation,
• other areas we have not accounted for.

Even when it is imperfect, we believe that formal analysis is often a useful supplement to decision-making. We strive to make clear where there are gaps in our knowledge so that our conclusions can be integrated appropriately with other sources of information.

Table of Contents

• Areas of evaluation:
  • Area I: Type
    • I.1–I.9 Classifications of Campaign Type
    • I.10 Evaluation
  • Area II: Certainty of Success
    • II.1 Precedents
    • II.2 Flexibility of Goals
    • II.3 Scientific Certainty
    • II.4 Other Uncertainties
    • II.5 Evaluation and Error Tracking
  • Area III: Barriers to Entry
    • III.1 Skill Required
    • III.2 Work Required
    • III.3 Resources Required
    • III.4 Evaluation and Error Tracking
  • Area IV: Expected Indirect Effects
    • IV.1 Positive Influence
    • IV.2 Negative Influence
    • IV.3 Evaluation and Error Tracking
  • Area V: Expected Direct Effects and Overall Efficiency Analysis
    • V.1 Expenditures
    • V.2 Results (“Profits”)
    • V.3 Evaluation
      • V.3.1 Calculation Sheet
    • V.4 Calculation Sheet Discussion
    • V.5 Error Tracking
• Suggested method for the summary of all evaluative information

Area I: Type

There are a number of ways to classify a proposed intervention. These variables will not be expounded upon in detail, but will be included so as to more accurately describe nuanced differences between various campaign types. If multiple campaigns are being analyzed to contribute to an understanding of the general efficacy of an intervention, some classifications may need to be applied to each campaign separately (see for example I.5 Size of Campaign.)

I.1 Activism Philosophy

A campaign might take an “incremental steps” approach to change, or follow a hardline all-or-nothing mentality. It might work to reform existing institutions or to oppose them completely. The following approaches are helpful to consider:

• Animal welfare approach / animal protectionism: attempting to improve the living conditions of animals without necessarily opposing a priori their use by humans for food/labor/etc.
• Animal rights approach: attempting to enforce a set of rights guaranteeing animals certain inalienable protections from harm. This could include such things as a right to be free from involuntary confinement regardless of quality of life.
  • Abolitionist approach: a sub-category of the animal rights approach. It seeks to achieve its goal of ending all uses of animals as human property by an all-or-nothing approach. It does not accept incremental change if such change allows for the continued use of animals as human property.
• Antispeciesism approach: treating certain abuses of animals as discrimination on the basis of species membership. This is not necessarily mutually exclusive with the previous two approaches.

I.2 Activism Approach

An intervention might work through direct action, legislative or political lobbying, research and development, or other means.

I.3 Target Demographic

The target demographic is the intended audience for a campaign. A campaign may target multiple demographics, and for some interventions, different campaigns may target different demographics.

I.4 Target Animal(s)

Target animals are the intended beneficiaries of a campaign.

I.5 Size of Campaign

The size of a campaign can be measured in terms of its duration, the amount of money available to sustain it, the number of people involved, and the size of the sponsoring group.

I.6 Time Delay between Intervention and Results

Time delay is a measure of how long will it take for a campaign to effect meaningful change.

I.7 Increase/Decrease in Human Suffering

Humans are animals too, and the effects of a campaign upon human welfare should not be ignored.

I.8 Characteristics of Organization Conducting Intervention

The success of a campaign may depend on aspects of the organization conducting the campaign including its age and size, track record with other interventions, quality of staff, and degree of commitment to the intervention. This section is particularly important for determining the applicability of results to campaigns conducted by organizations other than those whose work is considered in this evaluation.

I.9 Appeal to Other Organizations

A campaign may appeal to other organizations, funders, or individuals, bringing in support that could not be planned for from its launch. For past campaigns, to what extent has this occurred, and with what organizations? For possible campaigns, what other organizations may lend support, and what circumstances would make this most likely?

I.10 Evaluation

Activism Philosophy	Activism Approach

Target Demographics	Target Animal(s)	Size of Campaign	Time Delay between Intervention and Results

General Summary of Area I (Type). Please also address, if applicable, relevant details about how the intervention is likely to affect human interests; the capability of the organization planning the intervention; and the intervention’s appeal to other organizations.

Area II: Certainty of Success

Campaigns with identical efficiency ratings can still be differentiated. One way to do this is to focus on the riskiness of an intervention. If efficiency is held constant, riskier campaigns offer a relatively low chance of achieving a relatively high level result. If our estimates of expected value were perfect, we might not care about certainty of success except as it affects organizations’ willingness to attempt a campaign. However, we will not be able to provide perfect estimates of expected value, because in the real world we can never have perfect information. Our estimates of expected value are particularly likely to deviate from reality in the case of risky campaigns, since they succeed only some of the time, and, based on a finite and often small number of attempts, we cannot know the actual success rate with great precision.

II.1 Precedents

The success of future campaigns can be estimated in part by the success of past similar campaigns.

II.1.a Existence of previous attempts

To base predictions on past performances, it must first be established whether or not the proposed intervention has ever been attempted before. If so, it will be useful to know how many times it (or something similar) has been attempted, and whether those attempt(s) were staged in the same setting and context as the intervention currently being considered. The setting and context include such considerations as target demographic, geographical location of campaign, etc. It is generally assumed that unprecedented campaigns are, all things being equal, riskier than campaigns with solid precedents.

II.1.b Variation in efficiency of previous attempts

If a similar intervention has previously been staged, it is important to understand the variation in both costs and results. Campaign types with a history of unpredictable costs and outcomes are riskier than those with a less chaotic performance record. Wild fluctuations in run costs or in successful outcomes achieved will translate directly to wildly fluctuating measures of efficiency.

II.2 Flexibility of Goals

Campaigns that have lofty, all-or-nothing goals are inherently riskier than campaigns with continuous spectrums of potential positive outcomes. Campaigns that do not have fallback victories will be riskier than campaigns that do. For example, a low-visibility campaign to convince a corporation to stop supporting a certain type of animal testing may fail to produce any positive result if it is not successful. It is thus important to find out if there is only one goal set for the campaign, or if there are a variety of possible outcomes that will satisfy the purpose of the campaign. Campaigns that cast wider nets are less risky than ones that focus on narrow areas of improvement. For example, a gestation crate awareness campaign may have multiple potential goals, including getting its audience to adopt vegan diets, getting its audience to purchase pork from more humane sources, etc.

II.3 Scientific Certainty

There are a variety of scientific disciplines that study issues relevant to the success of animal activist interventions. The extent to which scientists have succeeded at understanding factors relevant to a particular intervention affects our certainty that the intervention will have a positive effect.

II.3.a Nonhuman neuroscience

There is not always clear scientific consensus regarding the neural complexity of nonhuman animals. A greater understanding of an animal’s neural networks allows us to be more certain that it is affected by allegedly harmful practices. For example, a campaign focusing on fishing may have to temper its scientific certainty more than a campaign focusing on pigs (given what is known about the latter’s neuroanatomy, and what is not known about the former’s). In making this determination, it may be necessary to read into the neuroscientific and/or psychological literature on the target animal population in question. We expect this to be a particularly controversial area of research, which will likely evolve along with scientific understanding of nonhuman animal consciousness.

II.3.b Sociology, psychology, and economics

Many campaigns rely upon assumptions about how humans will respond to an intervention, how societies will adapt to proposed change, and/or how economic forces will react to a campaign. It is thus necessary to uncover any ambiguities in the relevant literatures supporting the social science assumptions of an intervention. For example, the reliability of data on recidivism rates amongst new vegans will be important in estimating the certainty of success of a diet-focused campaign.

II.3.c Ecology

The expected effects of an intervention may rest on ecologically-based predictions. These interventions also must be rated based on reliable ecological data. For instance, it may be important to understand how factory farms impact their surrounding environment before being able to reliably quantify the effects of an anti-factory farming campaign.

II.4 Other Uncertainties

There may be other miscellaneous risk factors not yet addressed.

II.4.a Known unknowns regarding an intervention

Some risks are unpredictable, but others may be anticipated with proper planning. Before beginning an intervention, it is advisable to go through all components of a plan of action in order to determine what might go wrong, however unlikely. This is purposefully vague, as this method can be thought of as a “catch-all” for types of uncertainties thus far not addressed. For example, a certain legislative campaign may have to take into consideration the possibility of a cooperating political partner losing a reelection bid. Another campaign may have to take into consideration unfamiliarities in the culture of their target demographic, or a lack of certainty that all members of the campaign team can stay on for the duration of the campaign.

II.4.b Distance between intervention and improvement in animal welfare

The distance between a campaign and the animals it intends to benefit can be defined as the degree of separation or number of steps between an intervention’s activities and the end result of actually helping animals. For example, a campaign to care for rescue animals has a direct impact on animal welfare, while a campaign to train new animal activists would expect to achieve far-removed, indirect results (training → new activists → new campaigns → improvement in animal welfare). Similarly, a campaign to pass a new animal welfare law would also be further removed than caring for animals directly. A greater distance between intervention and result leads to a greater uncertainty of success.

II.5 Evaluation and Error Tracking for “Certainty of Success”

Evaluation for II.1.a

How many times has a similar intervention been attempted before?

1	2	3	4	5	6	7
1= No past attempts 7=Multiple organizations have started this type of campaign each year for at least the past decade.
Notes:

Has a similar intervention ever been attempted in the same context as the one being evaluated?

1	2	3	4	5	6	7
1= Few similarities, and no exact parallels, exist between past campaigns and one being evaluated, in terms of target demographic, geographic location, organizational similarity, etc. 7=Multiple campaigns have been previously attempted that have identical or near identical contexts as the one being evaluated, in terms of the factors listed above.
Notes:

Evaluation for II.1.b

How widely have previous campaigns varied in total results?

1	2	3	4	5	6	7
1= Very wide variation; some campaigns have achieved remarkable results, while others have been completely unsuccessful or counterproductive 7=Very little variation; it is difficult to detect any meaningful difference in results
Notes:

How widely have previous campaigns varied in efficiency?

1	2	3	4	5	6	7
1= Very wide variation; some campaigns have had large results with few resources, while others have been completely unsuccessful or counterproductive 7=Very little variation; all variation in results can be explained by variation in resources put in
Notes:

Comment on what is known about the reason(s) for the above mentioned variations between campaigns. Specifically, what variables were external and which were specific to a particular campaign/organization.

How well have previous predictions of results matched actual results?

1	2	3	4	5	6	7
1= Very poorly; advance predictions do not appear to correlate with campaigns’ eventual success 7=Very well; successful campaigns were in general widely expected to succeed and unsuccessful ones were clearly longshots
Notes:

How well have previous predictions of costs matched actual costs?

1	2	3	4	5	6	7
1= Very poorly; most campaigns have either raised funds they were unable to use or had unexpected expenses leading them to need more funding than projected 7=Very well; campaign costs have generally matched the planned budget closely
Notes:

Evaluation for II.2

How many different avenues of success does the campaign have available?

1	2	3	4	5	6	7
1= The campaign has only one major goal or path towards success 7=The campaign has several distinct goals that could each be reached independently
Notes:

How many intermediate positive outcomes does the campaign have?

1	2	3	4	5	6	7
1= Partial success on any of the campaign’s goals has no clear benefits 7=Any step towards any of the campaign’s goals has proportional benefits
Notes:

Evaluation for II.3.a

How well understood are the neurological abilities of the primary group of animals targeted by the campaign?

1	2	3	4	5	6	7
1= Understood mostly on an intuitive level 7=Well understood behaviorally and on a neurological level
Notes:

How well understood are the neurological abilities of any other groups of animals targeted by the campaign?

1	2	3	4	5	6	7
1= Understood mostly on an intuitive level 7=Well understood behaviorally and on a neurological level
Notes:

Comment on the choice of target group(s) of animals by the campaign. Address how well the goals of the campaign align with the ability of the recipient animals to meaningfully benefit from those goals, based on what is known about their neurobiology.

Evaluation for II.3.b

How well understood are individual and social group responses to this type of campaign?

1	2	3	4	5	6	7
1= Understood mostly on an intuitive or theoretical level 7=Many rigorous studies show very similar responses
Notes:

How well understood are economic factors affecting the outcome of this campaign?

1	2	3	4	5	6	7
1= Understood mostly on an intuitive or theoretical level 7=Many rigorous studies show very similar responses
Notes:

Evaluation for II.3.c

How well understood is the primary ecological impact of this intervention?

1	2	3	4	5	6	7
1= Understood mostly on an intuitive or theoretical level 7=Many rigorous studies show very similar responses
Notes:

How well understood are any other ecological impacts?

1	2	3	4	5	6	7
1= Understood mostly on an intuitive or theoretical level 7=Many rigorous studies show very similar responses
Notes:

Comment on any perceived oversights or uncertain assumptions in the scientific, sociological, economical, and/or ecological rationale behind this campaign.

Evaluation for II.4.a

How many known uncertainties affect the outcome of the campaign?

1	2	3	4	5	6	7
1= None 7=Several different sources of uncertainty not addressed above
Notes:

How strongly do known uncertainties affect the outcome of the campaign?

1	2	3	4	5	6	7
1= No known uncertainties affect this campaign 7=The success of the campaign hinges upon one or more uncertain factors not explicitly addressed by the campaign itself
Notes:

What outside actors and external events are expected to be able to affect this campaign’s success?

What unanticipated problems have befallen past campaigns? How were they handled?

Evaluation for II.4.b

List the chain of events connecting this intervention to a change in animal welfare, in as much detail as possible.

How many steps separate the actions of the campaign from a change in animal welfare?

1	2	3	4	5	6	7
1= None; the campaign helps animals directly 7=Very many; the chain is so long that it is difficult to think about in detail
Notes:

How many different actors are involved in the chain of events between the success of the campaign and the change in welfare?

1	2	3	4	5	6	7
1= None; the campaign helps animals directly. 7=Very many; the chain is so long or involves such complicated events that it is difficult to identify all actors or groups involved
Notes:

Summary

General Summary of Area II (Certainty of Success). Summary may include relevant comments about the following topics: what has been learned from previous campaigns; how narrow or flexible are the goals of the campaign; how clearly is the necessary scientific, sociological, economic, and ecological science upon which the campaign’s success relies understood; what is the (metaphorical) distance between the campaign and the animals it is attempting to help; and what other uncertainties should be noted.

Error Tracking

• How rigorously documented were the reports from past similar campaigns? How reliable is the information that defines the precedents for the campaign?
• What ambiguity exists in the scientific consensus regarding the mental abilities of relevant animals, the sociological and economic effects necessary for the campaign to make a difference, and the campaign’s ecological effects?
• What assumptions were made regarding “other considerations?” What factors may not have been considered?

Area III: Barriers to Entry

There are a myriad of factors to consider when evaluating how difficult an intervention will be to stage. This area of the evaluation examines the complexity of human skill required, the degree of work intensity needed, and the difficulty involved in obtaining the required material resources. This is not meant to duplicate the valuation mechanism laid out in Area V (Efficiency), but rather to provide a separate metric to measure the challenges inherent in putting on a campaign, with no reference to a conversion to fiscal units. This provides part of a cross-check for Area V, since the higher the barriers to entry for a campaign, the better its probable results must be to maintain a constant level of efficiency. In addition to providing a check on the efficiency as calculated below, knowing the barriers to entry for an intervention can help identify which types of group can likely conduct it successfully.

III.1 Skill Required

It is important to know what sort of expertise is required from a campaign’s staff. The more specific knowledge and/or special skills required, the more difficult a campaign will be. For example, a campaign to challenge an anti-whistleblower law will need significant legal expertise and political skill.

III.2 Work Required

The calculation of work required can be defined as a calculation of the intensity of effort required from an intervention’s staff members.

III.2.a Intensity of the work

Intensity is an attempt to quantify how exhausting, physically taxing, and mentally draining campaign-related work is. A more demanding workload indicates a more difficult campaign plan. For example, a campaign to engage strangers on a city street and talk to them about veganism may require exhausting days with lots of mentally taxing social interaction.

III.2.b Hours per week

The number of hours of work per week demanded from staff members compounds the effect of the intensity of their work per unit time. It will be useful to know how many hours per week are required from each type of staff member both at a minimum and at a maximum. A campaign may require a greater commitment around major events.

III.2.c Number of people needed

The number of staff members required for a campaign can be a limiting factor. Like the number of work hours required per week, this value may fluctuate over time.

III.3 Resources Required

It is further useful to examine the complexities involved in obtaining the necessary resources for an intervention. The procurement of materials for an intervention may require difficult-to-obtain permissions, such as copyright waivers, or an extensive search for uncommon objects. For instance, a campaign to develop a new meat substitute that requires a rare chemical has inherent difficulties associated with resource obtainment.

III.4 Evaluation and Error Tracking for “Barriers to Entry”

Evaluation for III.1

How many specialized skill sets are required?

1	2	3	4	5	6	7
1= None 7=Many unrelated and specialized skill sets
Notes:

What special skills are required in conducting this campaign?

How much expertise is required in the most demanding area?

1	2	3	4	5	6	7
1=Very little experience or skill 7=Many years’ experience or extensive professional training and advanced degrees
Notes:

Comment on the expected ease with which qualified staff members will likely be recruited. This may involve such factors as the job market for a certain profession, the average interest in animal activism from people with certain qualifications, etc.

Evaluation for III.2.a

How demanding is the typical workload associated with the campaign?

1	2	3	4	5	6	7
1= Light; working on the campaign involves little stress and is likely pleasant 7=Heavy; working on the campaign involves significant physical or mental labor at most times
Notes:

What challenges are involved with the daily work of the campaign?

Evaluation for III.2.b

How many hours per week does the typical staff member work on the campaign when it is making its minimum demands?

1	2	3	4	5	6	7
1= 1 or fewer 7=80 or more
Notes:

How many hours per week does the typical staff member work on the campaign when it is making its maximum demands?

1	2	3	4	5	6	7
1= 1 or fewer 7=80 or more
Notes:

Note that if the campaign involves several different types of staff roles with different time demands, it may be helpful to answer the above questions for each role rather than for the typical staff member.

Evaluation for III.2.c

How many staff members are involved with the campaign when it is ongoing but at the time when fewest people are working on it?

1	2	3	4	5	6	7
1=It runs automatically with at most one person supervising it 7=Over 50
Notes:

How many staff members are involved with the campaign at the time when the most people are working on it?

1	2	3	4	5	6	7
1=It runs automatically with at most one person supervising it 7=Over 50
Notes:

Evaluation for III.3

How many rare or difficult-to-obtain materials does the campaign require?

1	2	3	4	5	6	7
1= None 7=Several which are crucial to the success of the campaign
Notes:

How severe is the difficulty of obtaining the hardest-to-obtain material needed for the campaign?

1	2	3	4	5	6	7
1= There are no materials which will be difficult to obtain 7=There may be an insuperable obstacle to obtaining this material
Notes:

What resources does the campaign require that may be difficult to obtain?

Summary

General Summary of Area III (Barriers to Entry). Summary may include relevant comments about the following topics: the level of complexity of the skill sets needed for the campaign; the intensity of the work; and the feasibility of acquiring needed material resources

Error Tracking

• What differentiates the campaign from past similar campaigns? Could any of these factors affect the difficulty of achieving the campaign’s goals? Keep in mind that it may be difficult to ascertain which factors may have contributed to a past campaign’s success/failure.
• How rigorously documented were previous campaigns? Could estimates of staff time spent etc. be in error?
• Have environmental factors changed which would affect either the skills required to conduct the campaign or the difficulty of obtaining necessary materials?

Area IV: Expected Indirect Effects

Oftentimes, a campaign will have effects that reach beyond its intended goals. These may be positive (i.e. in-sync with the ethics driving the campaign) or negative (i.e. working against the change effected by the campaign). There are multiple factors to consider when evaluating external influence. These effects are especially difficult to estimate precisely because they are often diffuse and because even when campaigns have carefully tracked their progress towards their explicit goals, they may not have attempted to track other effects. However, diffuse and indirect effects of some campaigns may be significant parts of the campaign outcome, so we consider any evidence about these effects as a factor in evaluating interventions.

IV.1 Positive Influence

Good externalities may come from a campaign.

IV.1.a Chain effect

A chain effect is when those affected by a campaign spread the message of the campaign and thus extend the influence of the campaign’s message. For example, an online advertising campaign might have a particularly high positive chain effect, due to the ease with which a viewer can share a website or video link with others.

IV.1.b Benevolent slippery slope

It is possible that the success of the campaign will indirectly inspire other positive victories due to social pressure, or desire to imitate. For example, if multiple states enacted bans on gestation crates, some pork producers in other states might bow to public pressure and voluntarily phase them out themselves.

IV.1.c General education

General education is when a campaign instills a certain mindset in some subsection of the public that will go on to produce beneficial outcomes. For example, a campaign to oppose “ag-gag” laws laws might cause some people to develop a negative attitude towards animal agriculture in general, regardless of whether or not the campaign itself is successful.

IV.2 Negative Influence

Bad externalities may also arise from an intervention

IV.2.a Alienation

An imperfect campaign may run the risk of turning some potential supporters away from the cause and/or inciting a backlash against the goals of the campaign. For example, a public awareness campaign that uses offensive imagery to make its point may risk alienating potential supporters and/or fueling the influence and credibility of its detractors.

IV.2.b More harm than good

There is also be a chance that the intended results of a campaign might actually be more harmful than beneficial to animals. This is a difficult-to-spot type of negative externality because it requires Researchers to recognize when something is being overlooked in how a certain abusive system would respond to a successful attack against it. For example, a campaign against the consumption of beef might merely push people to consume more chicken (which would be particularly bad as it takes the meat of roughly 200 chickens to equal the same amount of meat from a single cow).

IV.3 Evaluation and Error Tracking for “Expected Indirect Effects”

Evaluation for IV.1.a

How large a chain effect does the intervention likely have?

1	2	3	4	5	6	7
1= No reason to expect a chain effect 7=Campaigns often go viral, with a high percentage of people who see hear the message sharing it in close to the original form
Notes:

Evaluation for IV.1.b

How likely is the campaign to inspire other victories?

1	2	3	4	5	6	7
1= No reason to expect the campaign to inspire indirect victories 7= Most similar campaigns have inspired successful imitations or victories beyond their expected scope
Notes:

Comment on any evidence (especially taken from observations of past campaigns) suggesting ways in which this intervention may directly or indirectly inspire other activism movements.

Evaluation for IV.1.c

How significant a change in public mindset is this campaign likely to cause?

1	2	3	4	5	6	7
1= No change; the public will be entirely unaware of or uninterested in the campaign 7=A significant change; the campaign will reach most of the population with a message that is highly contagious and represents a significant departure from current norms
Notes:

Evaluation for IV.2.a

How much alienation is this campaign likely to cause?

1	2	3	4	5	6	7
1= None; this campaign has no apparent controversial features 7=Significant alienation; a large contingent of potential allies will likely take offense at some aspect of the campaign, and the campaign may be perceived as representing the overall position of a large number of animal advocates
Notes:

Comment on the ways in which this campaign might alienate certain members of the public. Specifically, who might be alienated, and how damaging might the alienation be, in terms of negative impressions carrying over to other similar organizations?

Evaluation for IV.2.b

How high is the risk that the campaign may be more harmful than beneficial?

1	2	3	4	5	6	7
1= Extremely low risk; the campaign will have few, highly controlled effects 7=High risk; the campaign ignores significant parts of the system it is working in that pose a threat to animals
Notes:

What specific practices does the intervention seek to change? Would the replacement practices necessarily be an improvement for the animals involved?

Summary

General Summary of Area IV (Expected Indirect Effects). Summary may include relevant comments about the following topics: indirect reactions to the campaign (positive and negative); and long-term externalities of the campaign (positive and negative)

Error Tracking

• What assumptions have been made?
• How reliable is the data regarding past campaigns?
• How reliable is data predicting expected responses to various campaign tactics?

Area V: Expected Direct Effects and Overall Efficiency Analysis

This area of evaluation is designed to produce a single ratio of dollars spent per unit of suffering reduced or per number of animal deaths averted. We believe this is a useful factor to consider when planning an intervention, as different types of campaigns can have vastly different cost/benefit ratios. Given the limited amount of funds available to an organization of any size, this section explores ways of making sure that those resources are used optimally. We break down this calculation into component expenditures and results. If we could rely entirely on the results of this section, for many purposes we could omit the other sections of the evaluation entirely. However, in reality our estimates of both costs and expected results will be fallible, so reasoning about an intervention’s likely effectiveness from other perspectives (as above) allows us to be more confident of our conclusions than we could be about a direct efficiency analysis alone.

V.1 Expenditures

A successful intervention requires the input of resources, both financial and otherwise. It is important to understand and quantify the full extent of the costs to run a campaign.

V.1.a Monetary costs

Monetary costs include both upfront costs to begin an intervention and maintenance costs to sustain an intervention over time. Upfront costs are expenditures that go towards things such as the recruitment of volunteers, the purchase of office supplies, consulting fees, etc. Such expenses can be a limiting factor for an intervention if insufficient liquid capital is available. Maintenance costs are expenditures that are expressed in the form of dollars per unit time or per event. They include things such as material expenses (e.g. pamphlets, gas for vehicles, etc.) and money allocated to train and compensate workers (e.g. salaries for staff, costs involved in educating staff, costs involved in training volunteers for an event, etc.). Maintenance costs also involve non-specific overhead, such as office leases, travel expenses, and salaries/training costs that are not specific to any one event within a campaign. The amount of monetary resources needed for a campaign directly relates to how long a campaign can be sustained for and whether it can even be undertaken in the first place.

V.1.b Personnel

Personnel includes the number of people needed and the level of expertise needed to run an intervention. The number of people needed includes staff, long-term volunteers, single-event volunteers, outside help, consultants, and media coverage by animal activist journalists. While the cost of salaries is covered under “monetary” expenses, there is a clear opportunity cost of using up the time of people who are committed to working on animal activism projects. A poorly planned campaign could pull activists away from other, more effective interventions. The level of expertise required from the personnel includes both pre-existing expertise and the amount of training it is necessary to provide to new recruits. This could range from little to no expertise (e.g. one-off volunteers recruited for discrete events), to high expertise in animal law, high expertise in policy, high expertise in campaign management, high expertise in data analysis, etc. Recruiting the services of a person with a high level of expertise relevant to animal activism carries a higher opportunity cost, given that person’s theoretical ability to do more for a different activism campaign than the average activist.

V.1.c Time

Time refers to the time expenditure by an intervention’s staff, including both time committed during the intervention and time committed to pre-campaign training and planning. The former category involves such things as hourly shifts at events, weekly staff time, one-off consulting appointments, etc. This again is proportional to the magnitude of opportunity cost. TIme committed before an intervention begins includes time for event preparation, market research, experiment conducting, data collection, etc. This is time invested in laying the groundwork for a campaign. A campaign with a very short startup time may in some cases be preferable to a campaign that has slightly more potential effectiveness per unit time, but which also has a long startup time.

V.1.d Unknowns

Unknowns describes the category of unplanned expenses. While unknown expenses are by definition unexpected, it is possible to attempt to consider a range of possible emergency expenses and multiply the cost of each by the likelihood of the unplanned event occurring. This category includes such expenses as legal payments from being sued by a group or person, replacement costs for event materials lost on-site, etc.

V.2 Results (“Profits”)

After estimating expenditure of resources, we estimate the expected “return on investment,” measured by the degree of success a campaign achieves in reducing animal suffering or saving animal lives.

V.2.a Breadth

The breadth of an intervention’s results is defined as the number of people reached by the campaign. It can be expressed as the product of the number of people contacted per event or per time, and the number of events or units of time within the intervention. It is important to understand what is meant by the phrase “reached by the campaign”. A person does not have to be directly contacted by a staff member in order to be “reached”. A legal campaign to change the way a certain farmed animal is raised would (if successful) “reach” every person who consumes that animal product.

V.2.b Depth

Depth measures the extent of impact upon animal welfare the average person reached by an intervention will have. This is expressed as the product of multiple terms: the percent of people contacted that end up reducing animal product use (this is to be broadly construed as any lifestyle change that reduces a person’s negative impact on animal welfare, either by completely abstaining from the use of certain animal products or by switching to more humane animal use infrastructures or taking other actions that reduce animal suffering such as volunteering or donating to reduce animal suffering); the amount of time a person who reduces animal product use continues that reduction; the number of animals affected by the animal product use reduction; and the improvement in welfare for each of those animals affected. Each of these factors will here be defined.

The percent of people reducing animal product use is the fraction of people reached by an intervention that make certain measurable changes to their lifestyles (whether voluntarily, as in a dietary change, or involuntarily, as in consuming animals that are raised more humanely under new laws). There may be multiple categories of animal use reduction, with different accompanying conversion rates. For example, from the group of all people reached by an intervention, it might be found that 1% converted to veganism, 6% lowered their overall chicken consumption by a fifth, and 10% will only purchase cage-free eggs.

The amount of time a given lifestyle is adopted is defined as the length of time before a person adopting a new lifestyle is expected to experience recidivism back to their old lifestyle or back to a less humane lifestyle. This time may be different for each type of lifestyle change. The net good done by any given convert is proportional to the amount of time they stick with their newly adopted habits.

The number of animals affected by animal product use reduction measures the number of animals that are expected to have their welfare improved by a lifestyle change made by a person reached by an intervention. Again, this number might be different for each type of lifestyle change. This compensates for the fact that reducing, for example, an average omnivore’s beef intake has a much weaker effect than reducing the same person’s chicken intake, given the relative values of pounds of food produced per animal for cows versus chickens.

Finally, improvement in welfare per animal affected quantifies the decrease in suffering per animal affected by the campaign. Not all changes to an animal’s life are equal. For example, there needs to be a way to compare and rank the gain associated with a chicken being moved from a battery cage to an open pasture with the gain associated with a less painful slaughter method being used on a pig. This is a complex determination. It takes into consideration the improvement in living conditions for each animal, the mental complexity of each animal (i.e. their ability to experience those living changes as ethically meaningful increases in mental well-being), and the number of animals saved from slaughter.

V.2.b.2 Notes about improvements in welfare

The determination of the variable “improvement in welfare” also requires a decision regarding how to weigh the unit of “lives saved” against “lives improved”. Some animal activists take “reduction of suffering” as their bottom line, and value saving a life only so far as this prevents the suffering associated with animal death. Other activists also include the increase in pleasure as a factor in calculating animal welfare; they would regard lives saved as valuable both for the avoidance of the pain of death and for the gain of life’s pleasures available to an animal spared from death. FInally, some animal activists find intrinsic value in preventing animal deaths, regardless of suffering averted or pleasure increased.

The determination of suffering reduced requires some sort of an understanding of animal neurology, as discussed above in the form of “mental complexity.” This is a difficult variable to quantify, but remains incredibly important to the overall assessment, since less sensitive (with respect to the ability to experience pain) animals should be given less ethical weight compared to more sensitive animals. It is our hope that our ability to rank the welfare of various animals will improve as research into nonhuman animal consciousness progresses. For now, this crucial variable remains a grey area within the analysis.

V.3 Evaluation

Please comment on what has been empirically observed regarding the effect of relevant human lifestyle changes on animal welfare and on what has yet to be empirically determined.

Evaluation for V.1.a

What upfront and maintenance costs does the intervention have, including costs of supplies, salaries, and office space? How consistent are these costs between previous campaigns?

Evaluation for V.1.b

How many staff and volunteers are needed for the intervention? Do staff or volunteers require special skills and expertise? If the skill and expertise of staff and volunteers is not fully reflected by the cost of paying and training them, consider including the gap (the opportunity cost of shifting them to this project from any other) in the costs of the campaign accounted for in the calculation sheet.

Evaluation for V.1.c

How long does the pre-campaign phase of the intervention last? How much time is committed during the intervention? How consistent are these times between different past campaigns? (Be sure to include all the time the campaign takes, including pre-campaign activities, when calculating the costs of the intervention.)

Evaluation for V.1.d

What expenses have occurred for some past campaigns but not others? What other emergency or unplanned expenses might a campaign implementing this intervention incur? How likely is each one? Which of these expenses would be one-time expenses, and which might happen repeatedly over the course of the campaign (or become more likely the longer the campaign continues)?

Evaluation for V.2.a

How many people are reached directly by a typical campaign? How many people change their behavior due to the campaign without being aware of this? How much does the breadth of campaigns of this type vary?

Evaluation for V.2.b

What changes do people reached by the campaign make that will affect animals? What percentage of people reached by the campaign make each of these types of change? How much does this vary between different campaigns implementing this intervention? How well-documented and understood are the changes that people make in response to the campaign? For each change, how long do people typically retain the change? How deep of an impact does this make upon animal welfare? How many animals are affected, and how do their conditions change? Again, how well understood are these effects?

Evaluation for V.2.b.2

Does this intervention result in animal lives spared or in animal lives improved, or both? Is there exceptionally strong or weak evidence that the animal experiences the results of the intervention as a reduction of suffering or increase of pleasure?

V.3.1 Calculating efficiency

We offer a way to calculate as objectively as possible the efficiency for a given campaign measured in the units of animal suffering or animal deaths avoided per US dollar equivalent expended.

View the Blank Calculation Sheet in PDF, or the Example Calculation Sheet in PDF.

General Information
	Unit of intervention length	________			a1	How the length of the intervention is measured. E.g. “days”, “weeks”, “number of events”
	Estimated length of intervention, in intervention units (a1)	________			a2	Use unit defined in a1

Expenditures (unit=US$)		Pessimistic (Highest) Estimate	Realistic Estimate	Optimistic (Lowest) Estimate
One-time expenses						Expenditures that go towards things such as the recruitment of volunteers, the purchase of office supplies, consulting fees, etc.
	Cost of material resources	_____	_____	_____	b1	Supplies, leases, consulting fees, etc.
	Cost of recruitment and training	_____	_____	_____	b2	Advertising, value of time spent interviewing, etc.
	Personnel costs	_____	_____	_____	b3	Salaries, etc. paid out during startup period
	Upfront costs	_____	_____	_____	b4	=b1+b2+b3
Recurring expenses						Expenditures that are expressed in the form of dollars per unit time or per event
	Material expenses per intervention unit	_____	_____	_____	c1	Supplies, travel, etc.
	Personnel cost per intervention unit	_____	_____	_____	c2	Salaries, training, etc.
	Maintenance costs per intervention unit	_____	_____	_____	c3	=c1+c2
Expenditures total		_____	_____	_____	d1	=b4+c3*a2
Results		Pessimistic (Lowest) Estimate	Realistic Estimate	Optimistic (Highest) Estimate
Unit of suffering					e1	The unit by which the results of an intervention are measured. This may be “animal lives saved”, “years or year equivalents of a factory farmed hen’s life averted” (see V.2 and V.4 in evaluation guidelines), “years of farmed captivity averted”, or something different.
Direct suffering avoided per intervention unit		_____	_____	_____	f1	Measured in terms of unit of e1, for all direct results of an intervention (e.g. directly negotiating for the release of an animal from a factory farm)
Indirect suffering avoided
	Number of people reached by campaign per intervention unit	_____	_____	_____	g1	A person does not have to be directly contacted by a staff member in order to be “reached”. They must merely encounter the campaign in some capacity, including living under a legal jurisdiction being targeted by a legislative campaign.
	Proportion of people contacted likely to adopt lifestyle change 1	_____	_____	_____	g2.1	The expected percentage of people reached by the campaign that adopt a specific change (coded as change “1”) to their lifestyle which is anticipated to benefit animals (any lifestyle change that reduces a person’s negative impact on animal welfare, either by completely abstaining from the use of certain animal products or by switching to more humane animal use infrastructures).
	Proportion of people contacted likely to adopt lifestyle change 2	_____	_____	_____	g2.2	Same as for g2.1, but for lifestyle changed coded “2”
[Insert the appropriate number of rows as necessary, for lifestyle changes 3, 4, etc.]
	Proportion of people contacted likely to adopt lifestyle change n	_____	_____	_____	g2.n	Same as for g2.1, but for lifestyle changed coded “n”
	Indirect suffering avoided per person contacted per event	_____	_____	_____	g3	Multiply g2.1, g2.2,…, g2.n each by their respective “lifestyle multipliers” (see chart) and then sum the resulting products.
Results Total		_____	_____	_____	h1	=f1+g1*g3*a2

Final Total: the proposed intervention has a calculated efficiency of h1/d1, for a campaign of length a2, with results being measured in the unit e1.
Pessimistic:	Realistic:	Optimistic:

Lifestyle multiplier chart: This chart is only valid for results measured in units of “years of factory farm suffering or its equivalent averted”, with no distinction being made between different animals, and with wild-caught fish not being accounted for.
Lifestyle	Mean years of retention (i.e. years before a person is expected to no longer follow an adopted lifestyle; based on ACE research)	Years suffering avoided per year (based on ACE evaluations)	LIFESTYLE MULTIPLIER: Total years of suffering avoided
Veganism	6.2	9.6	59.52
Vegetarianism	6.2	8.5	52.7
Meatless 1x/week	6.2	1.2	7.44
Conscientious carnivore (avoids most or all factory farmed meat)	6.2	4 (Assumption: non-factory farming techniques cause less than half the amount of suffering as factory farming techniques, then adjusted downward to account for confusing labeling)	24.8

For units of suffering (g1) Other than the one shown in the chart above (e.g. to account for wild-caught fish, or to only count lives saved, a new lifestyle chart will be required. It should use the following format (add as many rows as necessary):
Lifestyle	Mean years of retention (i.e. years before a person is expected to no longer follow an adopted lifestyle)	Units of suffering avoided per year	LIFESTYLE MULTIPLIER: Total units of suffering avoided
_____	_____	_____	_____

Optional “multiplier” section, which may be used to adjust expenditure / results analysis for unknowns and externalities. This is offered as an optional section due to the particularly speculative nature of these variables:

Cost Multipliers		Pessimistic (Highest) Estimate	Realistic Estimate	Optimistic (Lowest) Estimate
Unanticipated costs		_____	_____	_____	i1	A number between 1.00 and 2.00, according to the following guidelines: A 1.00 would correspond to a campaign that does not expect any unanticipated costs, and a 2.00 to an exceptionally unorganized intervention with virtually nothing planned.
Unanticipated revenue		_____	_____	_____	i2	A number between 0.00 and 1.00, where 0.50 represents an incredibly visible, popular intervention that is likely to attract large donations, and where 1.00 represents a poorly visible and/or unpopular intervention that is not likely to attract any donors at all. As a rough guideline, a 1.00 would correspond to a campaign not expecting any donations at all, and a 0.50 is a campaign that expects to attract $1 in donations for every $2 spent (thus effectively halving expenditures).
Results Multipliers		Pessimistic (Lowest) Estimate	Realistic Estimate	Optimistic (Highest) Estimate
Negative backlash		_____	_____	_____	j1	A number between 0.00 and 1.00, according to the following guidelines: 0.00 for an intervention that is expected to alienate more people from the cause of animal welfare than it expects to positively effect, and 1.00 for an intervention that is expected to have no negative backlash.
Social momentum		_____	_____	_____	j2	A number between 1.00 and 1.50, where 1.00 represents campaigns where the target audience is very unlikely to spread ideas relating to animal welfare, and 1.50 for a campaign with a target audience that is very likely to spread those ideas.

Adjusted Final Total: the proposed intervention has a calculated efficiency of (h1*j1*j2)/(d1*i1*i2), for a campaign of length a2, with results being measured in the unit of e1.
Pessimistic:	Realistic:	Optimistic:

V.4 Calculation Sheet Discussion

A few points on the sheet:

• To express the magnitude of animal suffering avoided within a formula, it is necessary to convert to a common unit. It is difficult to express this as an objectively defined value, so the suggested way to measure the results of a campaign is to list the types and number of animals affected, and the specific ways in which their suffering is expected to be reduced. Ideally, the various forms of suffering prevented by a campaign could all be converted to a common unit. The unit for “results” (i.e. measurement of campaign success) has been defined here as years of factory-farm level suffering (i.e. the amount of suffering experienced across one year on a factory farm) averted. This is a crude unit, as it does not allow for distinctions to be made between different types of animals (with different types of consciousness), and makes it hard to deal with suffering that happens outside of the factory farm industry. It should also be noted that the calculations in the “lifestyle multiplier” chart within the calculation sheet only takes into consideration farmed fish, not wild-caught fish. If an alternative unit is desired, one “YBHS” (“years of battery hen suffering”) could be defined as the negative utility produced by keeping an average hen in a battery cage for one year. A year of non-battery conditions for hens could be estimated as a factor of the YBHS (e.g. it has been suggested in the Wageningen Journal of Life Sciences (Del Mol, et al., 2006) that cage-free facilities reduce the suffering of an egg-laying hen by half; thus, a year of cage-free hen suffering prevented would be equivalent to only 0.5 YBHS avoided). Other non-hen animals could have their yearly suffering also calculated as a factor of the YBHS, after taking into consideration both the nature of their environment and the capacity for the animals to feel pain relative to that of a hen.This is perhaps a more controversial method of calculation, as it requires making estimations about the relative abilities of different animals to experience suffering. While it is clear that the expected suffering of, say, a gestation crate-confined pig is many orders of magnitude stronger than that of a fly caged in a jar, it is not as clear that the same pig should be given greater weight than a battery-caged hen. However, estimations must be made for the purposes of this calculation. To count all animals as equal is itself an estimation, so the decision cannot be easily avoided.
• The expenditures/results multiplier sections include somewhat arbitrary (though intuitively reasonable) coefficient values. This is an attempt to quantify non-directly-calculable variables, in order to adjust the final values accordingly. As more research is done and more information is known about specific campaigns, these coefficients could change dramatically.
• The “total expenditures” and “total results” boxes (f1 and m1) give values specific to a campaign of the length described in a2. It is theoretically scalable to any campaign length, though unless upfront costs scale with campaign length, accuracy will be improved by revising the calculation to address the new length directly.
• The final output is given in terms of unit of animal welfare increase per cost in US dollars.
• The “lifestyle multiplier” key refers to the number of years of factory-farm level animal suffering are expected to be averted due to a person beginning to adopt a certain lifestyle (See I.2.b for more details). This is calculated by: [mean years person is expected to maintain lifestyle] * [number of years of animal suffering avoided per year of lifestyle].

V.5 Error tracking for “Efficiency”

• How reliable is the scientific data on the causal relationship between campaigns and behavioral changes?
• How reliable is the scientific data on the causal relationship between behavioral changes and improvements in animal welfare?
• How much direct knowledge exists about the costs and outcomes from past campaigns?
• How reliable are budget reports from past, similar campaigns? Could budgets mistakenly fail to list some costs and/or record unrelated expenses as campaign costs? Could some budgets fail to temporally synchronize expenditures with results?
• Has there been significant economic or social change since the time of past, similar campaigns that might invalidate their quantification of costs/outcomes? What other changes have happened (such as with supply chains for campaign resources) since past, similar campaigns?
• How reliable are estimates about things like the number of people expected to attend a campaign event, the number of campaign events to be held within a given time span, etc.?

Summary of the Evaluation

Activism Philosophy	Activism Approach

Target Demographics	Target Animal(s)	Size of Campaign	Time Delay between Intervention and Results

General Summary of Area I (Type).

Code	Question an (R) in the Score column indicates that a score of 1 would be generally considered preferable to a score of 7	Score (1-7)
II.1.a	How many times has a similar intervention been attempted before?
	Has a similar intervention ever been attempted in the same context as the one being evaluated?
II.1.b	How widely have previous campaigns varied in total results?
	How widely have previous campaigns varied in efficiency?
	How well have previous predictions of results matched actual results?
	How well have previous predictions of costs matched actual costs?
II.2	How many different avenues of success does the campaign have available?
	How many intermediate positive outcomes does the campaign have?
II.3.a	How well understood are the neurological abilities of the primary animal group targeted by the campaign?
	How well understood are the neurological abilities of any other animal groups targeted by the campaign?
II.3.b	How well understood are individual and social group responses to this type of campaign?
	How well understood are economic factors affecting the outcome of this campaign?
II.3.c	How well understood is the primary ecological impact of this intervention?
	How well understood are any other ecological impacts?
II.4.a	How many known uncertainties affect the outcome of the campaign?	(R)
	How strongly do known uncertainties affect the outcome of the campaign?	(R)
II.4.b	How many steps separate the actions of the campaign from a change in animal welfare?	(R)
	How many different actors are involved in the chain of events between the success of the campaign and the change in welfare?	(R)

General Summary of Area II (Certainty of Success)

Code	Question an (R) in the Score column indicates that a score of 1 would be generally considered preferable to a score of 7	Score (1-7)
III.1	How many specialized skill sets are required?	(R)
	How much expertise is required in the most demanding area?	(R)
III.2.a	How demanding is the typical workload associated with the campaign?	(R)
III.2.b	How many hours per week does the typical staff member work on the campaign when it is making its minimum demands?	(R)
	How many hours per week does the typical staff member work on the campaign when it is making its maximum demands?	(R)
III.2.c	How many staff members are involved with the campaign when it is ongoing but at the time when fewest people are working on it?	(R)
	How many staff members are involved with the campaign at the time when the most people are working on it?	(R)
III.3	How many rare or difficult-to-obtain materials does the campaign require?	(R)
	How severe is the difficulty of obtaining the hardest-to-obtain material needed for the campaign?	(R)

General Summary of Area III (Barriers to Entry)

Code	Question an (R) in the Score column indicates that a score of 1 would be generally considered preferable to a score of 7	Score (1-7)
IV.1.a	How large a chain effect does the intervention likely have?
IV.1.b	How likely is the campaign to inspire other victories?
IV.1.c	How significant a change in public mindset is this campaign likely to cause?
IV.2.a	How much alienation is this campaign likely to cause?	(R)
IV.2.b	How high is the risk that the campaign may be more harmful than beneficial?	(R)

General Summary of Area IV (Expected Indirect Effects)

Final Total: the proposed intervention has a calculated efficiency of (_______), for a campaign of____, with results being measured in____.

Pessimistic	Realistic	Optimistic

Were the optional modifiers used in calculating this total?

Yes/No

Final Determination (Overall Summary and Recommendations)