October 2014 Conversation with The Humane League

A conversation with The Humane League’s Executive Director David Coman-Hidy and volunteers Sabine Doebel and Zach Groff that took place on October 14th, 2014. They spoke with Allison Smith, ACE’s Director of Research. View the full review.

Who is typically involved in setting up/running/analyzing your studies? Who does what?

The basic questions addressed in a study come from a slate of studies that Nick Cooney and David set up when they started Humane League Labs.

They try to get a lot of feedback from different people when they are setting up a study, and they have increased the number of people they get feedback from over time. Usually Nick and Harish Sethu play a role in setting up survey questions and they also get outside feedback and, if possible, involve the team that does the analysis. Sabine and Susan Gabriel were involved in both setting up and analyzing the most recent batch of five studies.

THL staff and interns conduct the actual study, and if there is an in-person survey involved it can require the efforts of a lot of people. The Boston office also plays a big role. For studies conducted online, responses are generated via Mechanical Turk.

They have had Statistics Without Borders analyze some of their data. Susan worked on some studies recently. Sabine has been analyzing the current batch and Zach may help with the analysis in the future. Sabine recently completed a Psychology PhD and she knows about experimental design and is eager to apply inferential statistics where appropriate. Thus far, she’s been able to do this for some of the questions from the current batch of studies; the data in other studies is not clean for applying statistical tests to, she thinks partially because of the limitations of the software that was used to administer the survey. (Sabine let us know later by email that she had developed a plan for applying appropriate statistical tests to that data.) Sabine became involved with The Humane League after the survey used in these studies had already been conducted.

What changes have been made to the study process over time?

Many of the studies have very different designs so there aren’t clear changes they have made according to David. They are now engaging in more consultation during the design process and incorporating more feedback from the community, and as a result the recent surveys have been more professionally put together than older ones. They’re planning to go further in this direction with their planned study of the effectiveness of Facebook ads; they’ve been following the design .impact is developing and considering points from it and expect to partner with MFA and a donor to get their version conducted by a professional research group.

Have you implemented changes to programs based on what you learned from the studies?

Thus far, they’ve primarily talked to groups that print literature (like MFA and Vegan Outreach) because they do not really produce leaflets, which a lot of the studies have been on. Each new study is sent to representatives from MFA, VO, HSUS, ASPCA, CIWF, PETA, FARM, etc. as soon as it is published.

THL has also used some of the findings to inform their own messaging. From David’s follow-up email:

The largest change has been in the leaflets that we use. We used to ask for Compassionate Choices but give out a mix of those, EIYLMs, etc., but now we began giving out far more Something Better booklets in the wake of our study, and then we transitioned to the Your Choice, which are now more widely available and based on the SB booklets.

The next biggest impact would be our video choice: we use What Came Before, which outperformed the competing videos in obtaining veg starter guide orders/downloads. Soon, our video component video will result in editing the video in our ads based on the results.

Here are some of the smaller changes that I can think of:

• change in Humane Ed presentations to speak more about low cost/availability of veg food (we’ve changed the talk in Boston, Baltimore, and Charlotte so far, and will train the whole staff in this soon)
• change in Humane Ed presentations to directly address how family dinners etc. can be dealt with (we’ve changed the talk in Boston and Baltimore so far, and will train the whole staff in this soon)
• rhetoric of campaign videos/emails/petition text is about bad conditions (for the animal cruelty sections), rather than killing or animal intelligence (for example)
• our literature and fundraising materials (while very limited) continue to use a mix of animals, rather than just the chicken and fish who make up the majority of farmed animals