Our Approach to Assessing Leadership and Culture

Introduction

The way an organization is led can affect its organizational culture, which in turn can impact the organization’s effectiveness and stability.¹ We believe that a healthy workplace culture is an essential component of any effective organization—that’s why Leadership and Culture is one of our four charity evaluation criteria. When evaluating charities on this criterion, our main goal is to assess whether organizations appear to have leadership and culture issues that are substantial enough to affect our confidence in their effectiveness and stability.

Our 2022 Leadership and Culture Assessment

Gathering Information

During our 2022 charity evaluations, we reviewed aspects of organizational leadership and culture by examining information provided by top leadership staff (as defined by each charity) and by capturing staff perspectives via our staff culture survey.² Administering our culture survey to all staff members is an eligibility requirement to be recommended by ACE as a Top or Standout Charity. However, ACE does not require that all individual staff members at participating charities complete the survey. At the request of charities, we also distribute a version of our culture survey to volunteers working at least five hours per week. From time to time, we receive unsolicited testimonials from current or former employees or volunteers of the charities we evaluate.³ We use these findings, plus information that’s available online, to inform our leadership and culture assessments.

Assessing leadership

There appears to be no consensus in the literature on the specifics of the relationship between board composition and organizational performance.⁴ Therefore, we refrained from making judgments on board composition, and instead we took note of the following key aspects of leadership: (i) whether the Executive Director is a board member; (ii) whether there has been a recent leadership transition; (iii) leadership staff’s transparency with ACE; and (iv) the organization’s transparency with the public. However, these factors did not play a big role in our recommendation decision. Additionally, in the culture survey, we asked staff to identify the extent to which they feel that leadership competently guides the organization.

Assessing organizational policies and workplace culture

We asked organizations undergoing evaluation to provide a list of their human resources policies, and we solicited the views of staff (and volunteers, if applicable) through our culture survey. We recognize that surveying staff and/or volunteers could (i) lead to inaccuracies due to selection bias and (ii) may not reflect employees’ true opinions, as they are aware that their responses could influence ACE’s evaluation of their employer. In our experience, it’s easier to assess issues with an organization’s culture than it is to assess its strength. Therefore, we focused on determining whether there are issues in the organization’s culture that have a negative impact on staff productivity and wellbeing.

We assume that staff in the nonprofit sector often have multiple motives or incentives: They receive monetary compensation, experience social benefits from being part of a team, and take pride in their organization’s achievements for a cause.⁵ Because nonprofit wages are typically lower than for-profit wages, our survey asks all staff about wage satisfaction. In cases where volunteers responded to our culture survey, we typically asked organizations to provide their volunteer hours, because due to the absence of a contract and pay, volunteering may indicate special cases of uncertain work conditions. Additionally, we requested the organization’s benefit policies regarding time off, health care, training, and professional development. We also considered how many of our listed policies (13 general policies and eight representation, equity and inclusion and harassment/discrimination policies) charities have in place.⁶ While certain policies might be deemed priorities,⁷ we did not assess which specific policies from our list are most important for charities to have. Additionally, we made exceptions for charities working in regions where these policies are not common practice.

To capture whether the organization also provides nonmaterial incentives, e.g., goal-related intangible rewards, our culture survey included the 12 questions from the Gallup Q¹² employee engagement survey. We considered an average engagement score below the median value of the scale a potential concern.

Information we used to internally score staff satisfaction and policies:

• Staff satisfaction with wages: 1–5 scale
• Staff satisfaction with benefits: 1–5 scale
• Staff engagement: 1–5 scale
• General HR policies: 1–5 scale
• Harassment and discrimination policies: 1–5 scale

ACE believes that the most effective animal advocacy movement is one that is safe and inclusive for everyone. Therefore, we collected information about policies and activities regarding representation, equity, and inclusion (REI). We use the term “representation” broadly in this section to refer to the diversity of certain social identity characteristics (called “protected classes” in some countries).⁸ Additionally, we believe that in most countries, effective charities must have human resources policies against harassment⁹ and discrimination¹⁰ and that cases of harassment and discrimination in the workplace¹¹ should be addressed appropriately.¹² When cases of harassment or discrimination from the last 12 months were reported to ACE by current or former staff members or volunteers, we assessed whether the charity appropriately addressed those cases. We did this by considering staff perceptions of whether the reported cases were handled appropriately. If confidentiality permitted, we also asked leadership how they addressed the situation.

Information we used to assess harassment and discrimination:

• Staff reports of harassment/discrimination experienced: yes/no
• Staff reports of harassment/discrimination witnessed: yes/no
• Staff perceptions on leadership response to harassment/discrimination reports

Arriving at an Overall Assessment of Leadership and Culture

Rather than trying to develop an overall score for leadership and culture, we considered the scores for the different types of information we gathered to discuss and vote on whether we had any major concerns about a charity’s leadership and culture.¹³ To this end, our research team identified areas of potential concern, and we asked follow-up questions to the charity’s leadership to clarify details. The research team then held a series of meetings to discuss our findings and cast votes on whether our concerns were substantial enough to affect our confidence in a charity’s effectiveness and stability.

Changes from Previous Years

Until 2020, we attempted to evaluate the strength of charities’ leadership and culture without being in a position to verify many of the claims we received regarding harassment/discrimination. We recognize that this was a mistake. ACE is not a watchdog organization, nor do we have the expertise or capacity to investigate claims related to leadership and culture. We have since changed our approach to assessing this criterion. We now focus on determining whether we have any concerns that are substantial enough to affect our confidence in the effectiveness and stability of the charity being evaluated.

Additionally, we did not have an internal scoring framework in previous years. By shifting to a scoring framework, we aim to be in a better position to compare charities with one another, be more objective in our assessments of organizational culture, and give insights into how we arrive at our conclusions.

Limitations and Planned Improvements

One limitation of our approach to assessing organizational culture is the tension between being transparent (both with the charities we review and the public) about the issues reported to us and protecting the confidentiality of the people who come forward to express concerns. ACE prioritizes protecting the confidentiality of respondents to our culture surveys and those who reach out to us with unsolicited testimonials. This means that we sometimes cannot share specific information with charities or the public. For example, if a culture survey respondent highlights a specific concern, we might not be able to share that concern with the charity (even if they ask us with the good intention of addressing it) because doing so might risk identifying the survey respondent. This can lead to understandable frustration for charities that want to make things right. There is no clear solution to this tension between transparency and confidentiality. ACE will continue to prioritize protecting the confidentiality of those who share sensitive information with us.

A second limitation is our inability to verify the claims made by respondents to our culture surveys or those who report unsolicited testimonials to us. As previously noted, ACE is not in a position to take on an investigative role. However, it should be considered that false accusations are also insightful with regard to organizational culture. In organizations with a healthy workplace culture, we would not expect false claims to be made.

An additional criticism ACE has faced is the need for more clarity over how much weight we put onto our Leadership and Culture criterion when making our recommendation decisions. This is a fair point to make, as our lack of clarity can lead to confusion among donors and other ACE supporters about how we decide which charities to recommend. Moving forward, we plan to better clarify how we factor Leadership and Culture into our recommendation decisions. As a step toward that goal, we included an Overall Recommendation section at the end of this year’s charity reviews. In this concluding section, we noted (when relevant) where some charities performed better on Leadership and Culture than others.

1. Rousseau (1990), cited in Kartolo et al. (2022)  

2. At the request of charities, we translate the culture survey into their preferred language. In 2022, we hired a professional translation service to translate our culture survey into French, Slovak, Ukrainian, Russian, Telugu, Portuguese, and Polish. ACE staff also translated the survey into Spanish. We used Google Translate to translate the responses we received into English. 

3. ACE is not a watchdog organization, nor do we have the expertise or capacity to conduct deep investigations of these reports. 

4. Du Bois et al. (2007) 

5. Clark and Wilson (1961), as cited in Rollag (n.d.) 

6. Our evaluation process for human resources policies uses an assessment system that we have adapted from Charity Navigator. 

7. Nonprofit Leadership Center (2021) 

8. Examples of such social identity characteristics include: race, color, ethnicity, religion, sex, gender or gender expression, sexual orientation, pregnancy or parental status, marital status, national origin, citizenship, amnesty, veteran status, political beliefs, age, ability, and genetic information. 

9. Harassment may occur in one incident or many, and incidents can be nonsexual or sexual in nature. ACE defines nonsexual harassment as unwelcome conduct—including physical, verbal, and nonverbal behavior—that upsets, demeans, humiliates, intimidates, or threatens an individual or group. ACE defines sexual harassment as unwelcome sexual advances; requests for sexual favors; and other physical, verbal, and nonverbal behaviors of a sexual nature when: (i) submission to such conduct is made explicitly or implicitly a term or condition of an individual’s employment; (ii) submission to or rejection of such conduct by an individual is used as the basis for employment decisions affecting the targeted individual; or (iii) such conduct has the purpose or effect of interfering with an individual’s work performance or creating an intimidating, hostile, or offensive working environment. 

10. ACE defines discrimination as the unjust or prejudicial treatment of or hostility toward an individual on the basis of certain social identity characteristics. 

11. ACE defines the workplace as any place where work-related activities occur, including physical premises, meetings, conferences, training sessions, transit, social functions, and electronic communication (such as email, chat, text, phone calls, and virtual meetings).  

12. ACE recognizes that a lack of reporting does not necessarily mean that there are no issues at an organization, and it may indicate that staff don’t feel comfortable reporting issues.  

13. We keep the details of our scoring framework internal to protect the confidentiality of the charities we review.