Conversation with Eliza Scheffler, Research Analyst at GiveWell

Phone call between Jon Bockman, Allison Smith (ACE) and Eliza Scheffler (GiveWell) on April 10th, 2014

Jon Bockman: Executive Director of Animal Charity Evaluators (ACE)
Allison Smith: Director of Research at ACE
Eliza Scheffler: Research Analyst at GiveWell

Summary by ACE intern Amanda Lueke; approved by Jon, Allison, and Eliza.

E: What tactics do you use at ACE?

A: We use a lot of different tactics at ACE. We do independent research on leafleting and humane education, and synthesize that information with what happens when people go vegetarian. We’re also coming up with how to determine how effective interventions are so we can know which to recommend. We’re currently evaluating organizations from a previous ACE list, and updating that is a priority. We have a list of 30 organizations working on agriculture or working on a topic with some overlap, and did research based on public information to find out which are the most promising. We’re talking to people to figure out which ones to recommend.

J: Because we don’t know which interventions are the most effective due to lack of studies, we’re also trying to look at how open the organization is to updating their tactics and evaluating new evidence, and how badly funding is needed. Despite the need for more evidence, we’ve come up with recommendations by using seven criteria, which are modeled after GiveWell’s system.

E: That makes sense. How are conversations going?

A: We have a list of questions we ask. We ask for the organization’s goals, recent funding experience, what the funding would go towards, how they evaluate their programs, and which programs are the most successful and why. We also ask about how they would change their priorities if needed, the nature of how they are organized, and their transparency policy.

J: We also ask for a detailed budget and breakdown of time allotted to each project.

E: That’s great. Do you expect to make recommendations as you go or are you planning a launch date?

A: We’re relaunching in Mid-May. After the relaunch we’ll be updating in December.

E: I’m happy to give you a description of GiveWell’s process. In some ways we’re structured differently. GiveWell is investigating factory farming as a potential cause for further investment. We’re speaking with Researchers, funders, and organizations working in the area. The questions we’re asking are similar, especially those related to success stories, the organization’s track record, opportunities for change, and how much funding is needed. For factory farming, we’re not looking at direct outcomes per dollar input because the most useful data we’ve seen is much more qualitative. Quantitative data works well in health, but we intentionally put factory farming in GiveWell Labs because we’re not expecting that kind of evidence. Factory farming involves so many different factors like changing the political landscape or changing consumer behavior, and those things don’t lend themselves as well to rigorous or randomized study. Changing behavior depends so much on the population, there isn’t and probably can’t be that kind of rigorous evidence for things like what it took to pass Proposition 2 in California. At this point we’re looking for qualitative data and to get a landscape of the field, and create a map of organizations and funders, and compile common themes and answers from our interviews. We don’t expect to choose an organization to fund with our current research, just to get a view of the field and an idea of how to make recommendations after we have more time and research.

J: How many organizations have you spoken to?

E: Ten to fifteen organizations.

J: How are you deciding who to talk to?

E: The field of factory farming is small, so we started with a list of some groups that focus specifically on farmed animals, looked at their websites, and asked if they knew of any other organizations we should talk to. We started thinking narrowly about factory farming organizations as a group, and as we had more conversations we broadened the list to include organizations that looked at factory farming from a less traditional angle. An example is Food and Water Watch. They’re a less obvious organization that focuses on human health and the environment, but animal welfare is a secondary benefit of their work on animal agriculture.

A: We’ve looked at a few health-focused organizations for similar reasons.

J: When you’re looking for an overall feeling of the field, when will you know you’ve found it? What’s the end point for this shallow investigation, and how will you decide if factory farming is worth pursuing?

E: We’re moving towards writing up our findings and all our conversations. We’ll publish as much as possible and have written answers to our questions. This will help us understand where it stands in comparison to other topics and if the field is really devoid of funding. We have twenty shallow investigations right now to help us select causes to promote, so we have to look at factory farming in comparison to those other topics. We’ve already talked to major groups in the field, but we want to have more conversations. For now, we are not planning to talk to groups that work on the state level, because we’re not quite ready to look at the dynamics on the state level.

A: What do you see as the major opportunities to create change?

E: A few broad strategies come to mind as potentially especially promising. Corporate outreach and engagement seems promising, for example, gestation crates are being phased out because major food retailers are signing on to stop using them. Prop 2 was a major success, and other state measures may be promising, but federal legislation does not seem as promising because the meat industry has such a strong influence in Congress and ballot measures are not an option. Investigative reporting is promising, as it’s embedded at the heart of animal welfare education and lawsuits against the meat industry. Ag-gag laws could become a major problem because they threaten investigative reporting. When I write something up for GiveWell it will include what the experts with whom we’ve spoken have pointed to as promising.

J: Seems like most of what you recommend are large-scale actions. Do you have any feelings about grassroots outreach like leafleting?

E: I really don’t have a good sense of impact of those. I’m concerned that there’s only a certain percentage of the population that could be converted to vegetarianism, many of whom would probably end up changing their diets whether or not they receive leaflets, and there are some people in the population who will never be convinced. Reduced consumption of factory-farmed meat is one goal, but I’m not focused on vegetarianism as the singular desired outcome. A side benefit of grassroots outreach is building email lists for organizations that send out “action alerts.” For example, an action alert could be sent to prompt people to email a large food retailer such as McDonald’s to tell them to improve the welfare of the animals from their suppliers. Building the movement on the ground for corporate outreach is a desirable outcome of grassroots outreach.
It is my personal impression that there is greater awareness about the problems of factory farming among younger generations. However, factory farming at the scale we see today is itself a fairly recent innovation, so I don’t feel confident in attributing any increased awareness to an increase in certain education or outreach strategies.

A: I’m wondering how grassroots strategies overlap with the generational change we discussed. It seems impossible to disentangle the increasing awareness of vegetarianism from grassroots outreach.

J: Absolutely. You need grassroots action to work in tandem with legal and corporate reform. There needs to be public outcry. These different areas need to work together.

E: Yes, you certainly need public support for legal or corporate reform, but one of the questions that is difficult and important to answer is what is needed at this point to bring about those reforms – where are the gaps? Maybe the public support for reform is ahead of corporate policy, and that support should be leveraged to get corporations to implement reforms. Or maybe it’s the opposite, that corporations aren’t open to changing because there’s no demand from the public. Where do you feel there are gaps?

A: I don’t know where the gaps are, categorically.

J: I do agree with Eliza, awareness has grown from ten to twenty years ago. There’s a much higher awareness that factory farming exists and that it’s bad, but maybe people don’t know the specifics of the problems associated with factory farms. Maybe demand is there, but people need to get on board with specific campaigns.

A: There’s a gap between awareness of the problem and being willing to do or pay anything to change it. Consumers have to be willing to pay more for corporations to change their practices in a way that could raise prices.

J: That’s a great point and having participated in dozens of outreach campaigns, people are still shocked to find out the extent of the abuse in factory farms. Just hearing about an investigation in a factory farm or a food recall doesn’t make them want to take action.

E: There’s also confusion in different labeling programs. People who would be willing to pay for more humanely raised meat don’t know what to buy because the standards for certain labels aren’t clear. One way to reduce farm animal suffering could be to shift current meat consumption to higher-welfare, small-scale meat producers. Factory farming is supported by federal government subsidies; making small-scale producers more competitive could help.

J: When I think about that there’s no way, even with subsidies, that there’s enough land to go back to small scale production.

E: Oh yeah, it would still require meat reduction. So if meat consumption comes from small farms, it would require a large reduction in how much is consumed.

J: There still needs to be a demand for reduction. We have a definite need for reduction campaigns, and I’m not as sure about others. I’m going to go with what I’m sure about. It’s good to talk about.

E: Yeah.

J: Now that we’ve told you about our process, do you have any advice on what we’re doing? We’d love to hear your thoughts.

E: I poked around your website, the questions you list seem really valuable. There’s very little information out there. I clicked on interventions and it only listed three you’re studying. I think that even if you disagree with other strategies it’s still useful to lay out all the different strategies. You’re probably collecting a lot of information right now, but you should also lay out charities and interventions you’re not looking into. It’s helpful for donors. Try to be really transparent and include open questions and concerns for each intervention, even with the ones you do end up promoting. You should publicly recognize gaps in your research. It’s beneficial to be able to provide a lot more information for people who could have good feedback for you. I’m interested to see what your studies find, but I think it’s really hard to get any reliable data there and understand causality. Randomization can help mitigate these concerns though. You couldn’t collect the same sort of data for corporate interventions, but I wouldn’t rule them out for that reason; progress has to be compared in different ways. One thing I noticed was that you are looking at the history of social movements. I think it could be really valuable to identify trends and general strategies that could inform your thinking about this. We have a history of philanthropy project and we hope to learn what has worked or not worked in the history of philanthropy; I’m interested to see what comes of it.

J: Those are really great suggestions. I agree that it makes sense to look at other interventions and give a summary of them for people’s information and that’s something we should look into.

A: Right now we have a list of interventions somewhere but no summaries of any but the three.

J: We have some information on a volunteer and advice page where we make recommendations on interventions for people interested in volunteer work. It makes sense for us to cover things like corporate reform and legislative change like you mentioned. I think the thought behind us looking at interventions is not to provide the end all, but more so because it’s something we can add to the mix of what’s effective in interventions and charities. Because there is so little research done in these areas, we could help be the leader a little bit and help put some examples out there. We’ll put out the best studies we can. One thing I’m proud of that we put out there is a survey guidelines project. We’ve put a list of questions in hopes that we have a question bank that groups can use for their own research and we can be an example to other organizations on what can be done so hopefully they’ll use good practices and we can get some valuable data. We don’t necessarily think it should end there. There’s such a shortage of good research that it’s important to get that ball rolling. Would you agree Allison?

A: I think it’s also about comparing interventions to each other which is something I really want us to do. With something like legislative action you have really clear successes or failures whereas with something grassroots, if you don’t make any attempt at conducting some sort of random trial, you’re working on the success of yes, we distributed a million leaflets. You have no idea what that means in terms of actual consequences. No comparisons are possible.

E: Even within the same intervention maybe.

A: Even within very similar interventions maybe. Let’s compare a million leaflets to showing 40000 people this video. I don’t know where to start with this if we don’t even ask people later what they’re eating or what they intend to be eating.

E: Definitely.

J: I also like what you said about pulling in people from the effective altruism community online to become more involved and I think that makes a lot of sense as well.

E: We see our donors as a resource in our research process. Some of them want to do a lot of thinking about this on their own and some want to read about us on our website and decide they trust us. The main reason we value transparency so much is we provide all those people the ability to consider how we inform our arguments and disagree with us where they do, and they can comment and email us if they want more information. It helps us be open to their suggestions and criticisms and people pointing out what we might have missed. It’s been a valuable feedback loop to have in our system and part of the reason why we have this credibility. We don’t want to take advantage of perceived credibility and authority and keep people from considering the drawbacks of our charities or potential concern about interventions. We think it’s responsible to put that out there so that people can understand our holistic assessments.

J: I definitely wholeheartedly agree with that approach and we’ll definitely make an effort when we’re updating our website and recommendations we will publish as much as we can and try to list what we’re most confident in and what we see as uncertain and where we might be concerned.

E: I know we haven’t published much yet from our investigation of factory farming, but if you had any feedback for me or GiveWell I’d be happy to hear it.

J: It sounds like you’re in a more exploratory process than we are in now. It’s a very different goal between the two organizations but I think it’s great that you’re taking these initiatives. Talking to the authorities is a great start and I’m looking forward to seeing what you publish. Once we go through this process ourselves, I imagine we’ll learn some things. I’d love to keep an open dialogue so we can learn as much as possible.

A: It seems like we’re doing very different projects right now and from very different philosophical standpoints by not being involved in strategic cause selection the way GiveWell Labs is. We’re more engaged with an audience that might not be okay with us promoting something like small-scale animal farmers even if that’s an important piece of a systemic transition away from factory farming.

E: That makes sense.

J: Great, I’ve really enjoyed the call. Anything else?

E: No, thanks for all the questions.