Vegan Outreach (December 2014)

How Vegan Outreach Performs on Our Criteria

Criterion #1: The Organization Has Concrete Room for More Funding and Plans for Growth

VO has limited room for additional funding for its most central program, distributing leaflets on large, public college campuses in the United States. Present funding levels allow coordinators and volunteers to leaflet once or twice per semester on most of the campuses they cover, and VO has found that leafleting more frequently in the same locations has diminishing returns, based on leafleter experience. However, this program is not in danger of ceasing to be effective once all target locations have been covered, as new students enter college each year. VO has not found diminishing returns when they return to a campus where they leafleted during the previous semester or year.

VO is currently experimenting with a “Pay-Per-Read” program in which people receive rewards for reading one of their booklets and answering a survey a month later. This program is estimated to cost approximately $0.44 per person, and if it is deemed successful, could likely use considerable amounts of funding. They also recently began a “Vegan Mentor” program to provide support to aspiring and new vegetarians and vegans.

VO has plans for growth, and therefore room for increased funding, by expanding their operations in Canada, Mexico, and Australia. In the past, VO employees based in the US have occasionally taken leafleting tours in Canada or Mexico. These operations have been more expensive than leafleting at similar schools in the US because of the need to ship leaflets internationally, and sometimes there have been other problems related to the fact that coordinators needed to travel to and work in another country. However, if they expand these operations, they will be paying leafleters who are based in the country where they are working and may be able to reduce printing and shipping costs by printing locally or shipping in larger batches. VO’s operations in Australia are very recent; this year they started printing some leaflets in Australia, because shipping rates to Australia are too high. They are paying a local activist to distribute them. So far she reports success similar to leafleters in the US. By expanding their operations in Australia, they could print leaflets there at a lower cost per leaflet, possibly making overall printing and distribution cost similar to the cost in the US.

We think that VO can continue to use funding at the current yearly level about as efficiently as it has in the past, and that they may be able to use additional marginal funding if their new programs operate successfully. Right now, all their international programs are more expensive to run than their US programs, per leaflet distributed. We think their programs in Australia and Canada will probably remain slightly more expensive than the US program, because of being at smaller scales in countries where the population is more spread out and the cost of living is similar or higher. The Mexico program may in time be cheaper to run because of lower labor costs. However, right now we aren’t sure whether each leaflet will have a similar impact to a leaflet distributed in the US, or whether the impact will be greater or smaller. We also have similar questions regarding Canada and Australia, but cultural and economic differences between these countries and the US are not as great, so we believe that information gathered in the US is more likely to be applicable to these programs. As their new “Pay-Per-Read” and “Vegan Mentor” programs are new and currently being tested, we don’t know how effective they will be or how much additional money they could use to operate those programs.

Criterion #2: A Back-of-the-Envelope Calculation Finds the Organization is Cost-Effective

VO runs only one major program, printing and distributing leaflets. Although leaflets may have varying effects by the type of distribution (including whether at a college or at another venue), the area in which they were distributed, or the concentration in which they were distributed through a population, we don’t know enough about these differences to work them into our cost-effectiveness estimate with any reliability. Therefore, in order to estimate VO’s cost-effectiveness as an organization, we consider their total expenses compared to the number of leaflets they produce and distribute. We then estimate how many animals are affected using information about leafleting generally.

From July 2012 to June 2013, VO distributed a total of 2,540,617 leaflets with a budget of $800,398. This works out to a total cost of around 32 cents per leaflet. The marginal cost of producing and distributing additional leaflets is lower, since some costs, including the design and development of new literature, do not scale directly with the number of leaflets printed. VO ships copies of its most popular leaflets to interested individuals for a suggested donation of 11 cents per leaflet at the time of this review. We believe that this is a good representation of the marginal cost of printing a leaflet in a large batch (as is typical for VO) and shipping it to a final destination as part of a relatively small batch (but not individually). This is probably lower than the marginal costs of printing and fully distributing a leaflet, since many VO leaflet distributions are organized or entirely conducted by paid staff of VO or other organizations.

To consider the full costs of distributing the leaflets, including additional distribution costs for leaflets shipped to and distributed by paid staff of other organizations, we used the statistics on Adopt A College to estimate the number of leaflets distributed by organizations other than VO during the 2012-2013 fiscal year. We reviewed the top 100 leafleters for each of the time periods during this year,¹ and marked those that gave an organization name other than “Vegan Outreach” or “VO” as either their name or the organization they were affiliated with. In total, these people and groups distributed 809,894 leaflets during 2012-2013. We take this as the estimated number of leaflets that VO printed and shipped, but whose final distribution costs were partially covered by other groups.² For these leaflets, we estimate that VO covered about ⅔ of the total printing and distribution costs, based on a conversation with David Coman-Hidy of The Humane League, which distributed 768,665 VO leaflets in 2013. Using these estimates, the total cost of printing and distributing an average VO leaflet is about 35 cents. Again, the marginal cost would be lower than this average cost, due to economies of scale.³

We can use our estimates about the effectiveness of leafleting in general to convert these costs to a number of animals affected per dollar. This eases comparison between groups working with very different methods. However, doing so introduces many additional sources of error, and such estimates should be used with caution. Our Leafleting Impact Calculator is based on the rates at which subjects in a study conducted by Farm Sanctuary and The Humane League reported discontinuing consumption of certain animal products after receiving a leaflet similar to those distributed by VO. It also takes into account elasticities of supply and demand estimated by economists. Using a cost of 35 cents per leaflet together with that calculator, we estimate that VO spares around 1.87 animals from life on a factory farm per dollar in its budget. However, due to uncertainty around both the effects of leaflets on diet choices and the market elasticities for animal products, the bounds around our estimate are wide, ranging from .1 animals/dollar to 20.05 animals/dollar. From this we conclude that VO is working within a good range of cost-effectiveness, comparable to other charities we have reviewed at this depth. Due to the wide error bounds and numerous unconsidered factors on this and other cost-effectiveness estimates,⁴ we must also consider other criteria seriously when determining which group to support.

Criterion #3: The Organization is Working on Things That Seem to Have High Mission Effectiveness

VO works primarily on leafleting college campuses in the US to promote awareness of farmed animal suffering and encourage young people to act on this awareness by changing their diets. Focusing on farmed animals, which are the easiest group of animals to help cheaply at a large scale, is likely to be very effective. We think individual outreach, of which leafleting is one variety, is a necessary precursor to many possible systematic changes. Targeting young people also has high expected effectiveness, since they are more receptive to new ideas and, if they adopt new ideas, will have a long time to put them into practice and attempt to spread them to others. In colleges specifically, young people also have many opportunities for leadership and for influencing relatively large social groups.

VO leaflets do not emphasize additional approaches to helping farmed animals beyond changing one’s diet individually or encouraging friends and family to do the same. As such, the effectiveness of leafleting may be hampered because even people who are strongly affected by the leaflets influence animals primarily through changes in their diet and potentially in the diets of a few other people. Other activities which specifically attempt to leverage the connections and abilities of people they reach, or to target particularly influential individuals, may have higher effectiveness for this reason.

Criterion #4: The Organization Possesses A Robust and Agile Understanding of Success and Failure

VO has historically measured the success or failure of their leafleting efforts largely through the spontaneous reports of individual leafleters and recipients of leaflets. This method does provide continuous updates on the state of operations, since leafleters gather impressions of their own success at every leaflet distribution. VO is also beginning to supplement this source of evidence with other streams, such as the results of the Farm Sanctuary/The Humane League study on leafleting and Faunalytics’s work regarding the reading level of animal advocacy materials. These sources allow some access to aspects of success or failure that may not occur within the direct perception of leafleters.

Currently, VO is experimenting with several new programs (“Pay-Per-Read” and “Vegan Mentor”) which are attempting to track the effect of their literature distributed both in print and online. They don’t have much data yet, but we are encouraged that they are trying variations in their approach and seeking to measure impact.

We have substantial concerns about measuring the impact of a program informally through the feedback of the people conducting the program. Because leafleting is a physically and emotionally demanding task, volunteers and staff who engage in a large amount of leafleting are likely to be heavily self-selected for qualities that would cause their perception of leafleting’s effects to be positive. For instance, they may be optimistic people who focus heavily on positive responses and do not notice negative ones. They may also have high expectations about how likely recipients of leaflets are to change because of the leaflets. The act of leafleting can reinforce these high expectations, as recipients of leaflets who did change their diets after receiving them are more likely to approach leafleters and tell them than are recipients of leaflets who did not make any changes.

Without a systematic attempt to track the experiences of all (or a representative sample of) leafleters and leaflet recipients, the most positive reports will therefore receive the most weight. Leafleters with very much experience naturally have more perspective on any given event, and therefore their opinions will be taken especially seriously. However, leafleters who have doubts about the effectiveness of their activities are unlikely to continue those activities or increase their involvement in them, so the opinions of experienced leafleters will almost always be unusually positive. Similarly, recipients of leaflets who did not appreciate receiving a leaflet or experience a change because of it are unlikely to seek out the organization that gave it to them to say so.

An additional concern with applying this method to leafleting in particular is that, although VO tracks the number of people who approach leafleters to say they have changed their diet because of receiving a leaflet, most of the feedback they get is from leafleters, not from leaflet recipients. Leafleters, in turn, mainly notice how willingly people take leaflets and whether they are interested in reading and discussing them immediately after receiving them. They have little access to information about the ultimate intended effects of the leaflets.

We are encouraged that VO is moving towards an understanding of the success or failure of their efforts that takes into account more types of information. However, they have yet to establish a track record of using these types of information, particularly where they conflict with the prevailing reports of leafleters. For instance, in explaining the development of the new Your Choice booklet, VO cites the opinions of leafleters that existing materials were beginning to look dated for a young audience, the findings of the Farm Sanctuary/The Humane League study, and the findings of Faunalytics’s work on readability as all being in support of the changes between the new booklet and other VO materials. In their efforts to examine the effectiveness of their new programs, they show that they are concerned with effectiveness, both in their willingness to experiment with new ideas based on findings from other organizations and their interest in measuring their impact through follow-up surveys.

Criterion #5: The Organization Possesses a Strong Track Record of Success

VO has a long and solid track record of producing and distributing leaflets that advocate for farmed animals, against industrial agriculture, and for a vegan diet. Statistics on the Adopt a College site show a pattern of generally increasing distribution from 2003 through the present, which is one of the longest track records we’ve seen, particularly when it comes to a track record of engaging in a specific program.

As is the case for any group attempting to influence individuals’ behavior, it is extremely difficult to establish how strong VO’s track record is with regard to its ultimate impact upon animals. Individual behavioral changes are very difficult to measure and track, particularly when contact is limited, as it is between VO and the recipients of its leaflets. The wealth of the available anecdotal evidence is enough to be suggestive that VO has had a positive effect, but it is not enough to determine the magnitude of that effect. Recent studies have also found positive effects of leafleting and have attempted to determine the approximate magnitude of these effects, but there is still significant uncertainty that may not be feasible to resolve.

We consider VO’s track record of accomplishments strong overall. While groups influencing government and corporate policies are able to more concretely demonstrate their short term effects on animals, ultimately we believe that spreading changes in the way individuals think about and behave towards animals is extremely important. While we do our best to give all available evidence appropriate weight, we do not want to penalize groups engaging in this work because of the inherent difficulties of measuring its effects.

Criterion #6: The Organization Has Strong Organizational Leadership and Structure

VO has recently undergone a significant leadership transition with the departure of its Executive Director and Director of Development, each after several years in their position. The current Executive Director, Jack Norris, was one of the co-founders of the organization and has been involved in leadership roles since 1993. However, it is difficult to assess the strength of a leadership team that has recently changed composition significantly.

VO has a highly developed method for dealing with turnover and training new employees and volunteers at lower levels. Volunteers are encouraged to leaflet on their own using instructions provided on the website, but are also added to a list and contacted by outreach coordinators when they plan to visit a college or even in the appropriate area, so that they can leaflet with an experienced leader if they want. Outreach coordinators are usually hired after acquiring significant experience with leafleting, either as a volunteer for VO or in some other way. They also receive a handbook detailing how to approach their responsibilities.

Criterion #7: The Organization is Transparent

VO maintains a roughly average level of transparency among the organizations we have reviewed at the same level of depth. They produce an annual report containing financial and basic organizational information, and will distribute it to anyone who asks. They also work cooperatively with other animal advocacy organizations and share some information about what they find successful and unsuccessful on their website. They were cooperative and prompt in responding to our requests for information, which required more detailed and more sensitive information than is available online.

1. Fall 2012, Spring 2013, and Venues 2012 

2. There are several possible sources of error in this estimate. First, as with the total number of leaflets distributed, we rely on self-reports by leafleters here; some distributions may not have been recorded on the Adopt a College site and some may have been recorded with approximate or erroneous numbers of leaflets distributed. Second, we did not use all available data, as there were 328 leafleters in each school semester and 288 leafleting other venues in the period we considered. The data we considered accounted for 2437824 leaflets in total. Third, leafleting totals at non-school venues are aggregated by calendar and not by fiscal year; we used the 2012 calendar year because a substantial amount of such leafleting occurs over the summer, but this is not perfectly synchronized with the other time periods we are considering. Finally, some leafleters we marked may have been participating in distributions organized by Vegan Outreach or for which there were no costs to entities besides Vegan Outreach (no paid staff or travel expenses), and some leafleters we did not mark may have been participating in distributions organized by paid staff of other organizations. 

3. “Not including the costs of printing, it costs [THL] 2¢ to distribute a Veg Starter Kit and 4-6¢ to distribute a leaflet.” and “Factoring in both groups’ costs for THL distributing Vegan Outreach leaflets, the cost per leaflet rises to about 12¢.” 

4. For instance, here we have taken our estimate of leafleting’s effectiveness in producing diet change from a single small study using self-reported data. This estimate is corroborated by our own, even smaller, study. However, even in combination this is not enough information to conclusively determine leafleting’s effectiveness. We also apply those studies, both done on and around college campuses, to all Vegan Outreach’s leafleting efforts, although many leaflets are distributed at concerts and other venues. Both studies surveyed participants only once and asked about only a few months of their experience, but we apply data from other sources to estimate the length of time they will maintain any diet changes; these findings may not be valid for this situation. Both studies also found high rates of change relative to the changes expected from similar interventions designed to elicit other behaviors; it is not yet clear whether this reflects an unusually good combination of target audience and message or problems in the study designs. We did not consider possible long-term effects of leafleting, such as a general spreading of concern for farmed animals not reflected immediately in eating habits. Leafleting is one of the better-studied interventions used by the animal advocacy movement to change individuals’ behavior, and this is only a partial list of factors we did not account for in this estimate, so all our estimates must be taken as general guides only. 

Research Documents

• May 2014 Budgetary Distribution
• April 2014 Conversation with Jack Norris
• September 2014 Conversation with Jack Norris
• May 2014 Cost Effectiveness Estimate Spreadsheet