Conversion Rate of Leafleting | Animal Charity Evaluators

This article describes part of the reasoning used in our 2014 analysis of leafleting, which does not reflect our current views. We would now consider a much broader field of evidence when evaluating the effectiveness of leafleting.

Our data about the effectiveness of leafleting comes from a study conducted in fall 2012 by The Humane League and Farm Sanctuary. Leaflets were distributed to college students, and Researchers returned 2-3 months later to conduct a survey about dietary change. Only students who reported receiving a leaflet were allowed to complete the survey.

The code that follows is the R code used in our analysis. If you would like a copy of the data used to replicate or extend our analysis, please contact us.

Of 489 respondents:

16 stopped eating red meat
8 stopped eating chicken
5 stopped eating fish
2 stopped eating eggs
3 stopped eating dairy

Other respondents reported reducing their consumption of these foods, and some reported that others had also been influenced by their dietary change. But we don’t know how to quantify the impact of those actions, and we think they are more subject to mis-reporting than the figures cited above, which are already subject to response and social desirability biases.

We calculate the proportion of the sample that stopped eating each type of animal product.

x

x/489

## [1] 0.032720 0.016360 0.010225 0.004090 0.006135

We’ll consider the minimum of the three numbers for stopping meat consumption (as the number of vegetarians that can be “patched together” from these results), the number of respondents who stopped egg consumption, and the number of respondents who stopped dairy consumption.

We’ll model each behavior change according to a binomial distribution, assuming each respondent in a given group had the same probability of giving up meat, eggs, or dairy as each other respondent. For each group and type of product, we’ll construct a Jeffreys 95% credible interval for the proportion of people who stop eating the product. This Bayesian interval, based on the non-informative Jeffreys prior, also has the good frequentist property of being equally likely to lie above the true value of the parameter as below it.

qbeta(c(0.025, 0.975), 5.5, 484.5)

## [1] 0.003912 0.022265

This suggests that between 0.4 and 2.2% of the population receiving leaflets went vegetarian.

qbeta(c(0.025, 0.975), 2.5, 487.5)

## [1] 0.0008509 0.0130554

Between 0.1 and 1.3% of them cut out eggs.

qbeta(c(0.025, 0.975), 3.5, 486.5)

## [1] 0.001731 0.016281

Between 0.2 and 1.6% of them cut out dairy.

These credible intervals account for random variation between the population who received the leaflets and the actual sample obtained. They do not account for systematic bias in the data, including the possibilities that people who changed their diets were more likely to recall receiving a leaflet and that people deliberately responded in ways they thought would please the Researchers. These sources of error are most effectively controlled through study design, and corrections for them do not appear in our current analysis.

Resources

Humane League Labs. (July 19, 2013). Report: Which leaflet is more effective?