How Are We Doing? A Multiple Constituency Approach to Civic Association Effectiveness

Abstract

Increased citizen participation in policy processes through voluntary civic associations warrants an analysis of their effectiveness, which this article undertakes using a multiple constituency framework. We find a gap in the literature on nonprofit effectiveness where theoretical and empirical studies have mainly focused on organizations that directly provide tangible goods and services. We propose a multiple constituency approach to evaluate and understand the implications for assessing the organizational effectiveness of community-based advisory civic associations. We empirically analyze the evaluation of Los Angeles neighborhood councils by three different constituency groups—citizen participants, street-level bureaucrats, and city council staffs. We find that the effectiveness ratings of the constituency groups are dissimilar on different dimensions of effectiveness. These findings suggest that the multiple constituency framework holds theoretical and practical value for understanding the organizational effectiveness of voluntary associations, where the different goals of various stakeholders lead to different views on effectiveness.

Keywords

organizational effectiveness multiple constituency approach voluntary civic associations advisory community organizations

The purpose of this article is to analyze the organizational effectiveness of voluntary civic associations using a multiple constituency framework. In an era of devolution, evaluating the effectiveness of the nonprofit and voluntary sector has become increasingly important particularly because of issues of accountability (Alexander, Brudney, & Yang, 2010; Murray, 2010). Despite the elusiveness of organizational effectiveness as a researchable concept (Bluedorn, 1980; Goodman, Atkin, & Schoorman, 1983), continuous efforts have been made to understand the effectiveness of nonprofit and voluntary organizations (Andrews, Ganz, Baggetta, Han, & Lim, 2010; Dart, 2010; Herman & Renz, 1997, 2004; Sowa, Selden, & Sandfort, 2004). Evaluation research, however, has largely focused on nonprofit organizations that directly deliver tangible goods or services (Forbes, 1998; Papadimitriou & Taylor, 2000; Shilbury & Moore, 2006). In this article, our focus is on the effectiveness of voluntary associations with indirect and intangible outputs and outcomes. In particular, this research concentrates on a set of civic associations that are active on broad community issues and play a semiformal advisory role to local governments (Berry, Portney, & Thomson, 1993). Despite charges of decreasing social capital (Putnam, 2000), the role of voluntary civic associations continues to grow with their increasing involvement in policy processes, where they have exerted greater influence in local governance (Musso, Weare, Bryer, & Cooper, 2011).

Voluntary civic associations differ from firms, public agencies, and bureaucratic nonprofits that deliver goods and services as they “depend on the voluntary efforts of their members, … govern themselves through elected volunteer leaders, and enable their members’ collective voices to be heard” (Andrews et al., 2010, p. 1192). Our study focuses on advisory civic associations that primarily provide an advocacy function in the community rather than tangible goods or services. We adopt an expanded definition of advocacy, which “focuses on attempts to change policies or influence the decisions of any institutional elite, government, and state institutions through enhancement of civic participation to promote a collective goal or interest” (Schmid, Bar, & Nirel, 2008, p. 581). This broad definition includes not only traditional advocacy groups but also community-based advisory organizations that aim to influence decision-making processes, which are the focus of this study. Although nonprofits that provide direct services and tangible goods also may serve in a community advocacy role, we examine civic associations with the primary goal of community advocacy and representation.

The implications of these differences within the broader nonprofit and voluntary sector are crucial for understanding organizational effectiveness. With the influence of new public management, evaluation studies within the nonprofit sector have focused on organizations that produce measurable outputs and outcomes (Dart, 2010). In contrast, voluntary associations often focus on facilitating democratic processes, which have more difficult outcomes to evaluate. Some civic association outputs can be quantified, such as the ability of associations to increase opportunities for voice; however, other outputs and outcomes are much more difficult to quantify, such as the effectiveness of associations in influencing policy processes. Without as many concrete outputs to consider compared with nonprofits that provide goods and services, evaluations of voluntary civic association effectiveness inevitably involve more elements of perceptual judgments, which add to the complexities of evaluation.

In the plethora of organizational effectiveness studies, only a handful of studies have examined voluntary associations (Andrews et al., 2010; Torres, Zey, & McIntosh, 1991; Webb, 1974). We argue that the multiple constituency approach, which has been used consistently in evaluating service-oriented nonprofits that also involve some elements of subjective evaluation, particularly is useful for examining the effectiveness of voluntary civic associations. A central concern is how and by whom effectiveness is defined and evaluated as these organizations involve multiple stakeholders (Herman, 1992; Martin, 1980). As in other nonprofits, the internal and external stakeholders of civic associations that provide an advisory role to government are important not only because they are involved in and affected by the work of these associations but also because the legitimacy of these associations depends on them.

In this study, we explore how civic association effectiveness is evaluated differently by different stakeholders based on their respective goals and interests; thus, our main contention is that a multiple constituency approach to organizational effectiveness provides a systematic framework to elucidate these differences particularly in situations where perceptual judgments are used in evaluations. Using the multiple constituency approach, our study demonstrates how effectiveness evaluations by multiple constituencies diverge on different dimensions of effectiveness. Rather than merely disagreement on ratings, the findings suggest that the disparate goals and interests of multiple constituencies shape their understandings of effectiveness and thus effectiveness evaluations. The framework discussed in this study will be important for public and nonprofit managers, funders and evaluators as they examine specific areas where advisory civic associations can improve their practices. This article uses Los Angeles neighborhood councils (NCs)—grassroots self-organizing entities formed by voluntary community members with semiformal ties to the city—as the empirical case. Next, we review and extend previous theoretical and empirical works on organizational effectiveness by distinguishing approaches to evaluating effectiveness and the types of nonprofits evaluated.

Evolution of Organizational Effectiveness as a Construct

Since the 1950s, understanding organizational effectiveness has remained a theoretical and empirical problem in organizational theory. Various theoretical efforts have attempted to resolve these conceptual complexities by taking different approaches toward organizational effectiveness. Early literature on organizational effectiveness can be distinguished by unidimensional and framework-based approaches. Unidimensional approaches focus on one dimension of effectiveness based on a particular theory. In contrast, framework-based approaches, such as the multiple constituency or multidimensional approaches, recognize that organizational effectiveness is a multifaceted construct. For example, in the first generation of studies, the debate among unidimensional approaches to effectiveness was between goal attainment and system resource theories. The goal attainment approach defines effectiveness as achieving a set of stated goals, and efforts have been made to identify objective measures (Georgopoulos & Tannenbaum, 1957; Price, 1972). In contrast, Yuchtman and Seashore (1967) proposed a system resource approach, where organizational effectiveness is understood as resource acquisition.

The second-generation studies made the important contribution of recognizing organizational effectiveness as a multidimensional construct; thus, framework-based approaches attempted to resolve the debate between the goal attainment and system resource perspectives (Cameron, 1986; Cameron & Whetten, 1983; Quinn & Rohrbaugh, 1981, 1983). Although it understands organizational effectiveness as a multifaceted construct, the multidimensional framework fails to recognize that a diversity of perspectives regarding effectiveness can emerge from an organization’s multiple stakeholders. Thus, a variant approach to understanding the multiple dimensions of effectiveness is the multiple constituency framework (Connolly, Conlon, & Deutsch, 1980; Tsui, 1990; Zammuto, 1984). Connolly et al. (1980) advocate for a multiple constituency framework, which “allows multiple evaluations from multiple constituencies” (p. 212) as these multiple constituencies have different understandings of organizational effectiveness based on their respective interests (Boschken, 1994; Hitt, 1988; Zammuto, 1984).

Some have suggested that tackling the multidimensionality of organizational effectiveness may be less of a problem compared with using a single perspective to evaluate an organization as more than one effectiveness assessment can exist based on different stakeholder views (Dart, 2010; Tsui, 1990). Thus, understanding organizational effectiveness involves “human judgments about the desirability of the outcomes of organizational performance from the vantage point of the varied constituencies directly and indirectly affected by the organization” (Zammuto, 1984, p. 614). Tsui (1990) advocates using this approach for organizations “that have some sort of constituency relationships, when the entity’s effectiveness can only be meaningfully measured by subjective opinions of some referent groups, and/or where consensus does not exist on the relative significance of objective performance measures” (p. 480). Therefore, the multiple constituency framework is suitable for constituency-focused organizations.

Theoretical and Empirical Approaches to Nonprofit Organizational Effectiveness

The nonprofit effectiveness literature has drawn concepts from the general organizational effectiveness literature discussed above; however, a review of the literature demonstrates that empirical studies on nonprofit effectiveness have been tilted toward organizations that produce tangible goods and services, and limited research exists on the effectiveness of voluntary civic associations. We differentiate between the direct service provision and advocacy functions of nonprofits. Our focus is on the broadened notion of advocacy, which includes activities that influence policy decision-making and implementation processes and enhance civic engagement while doing so; these activities also aim to change resource allocation priorities (Schmid et al., 2008). The expanded definition includes rights-oriented traditional advocacy groups and advisory community-based associations that represent various community interests. Nonprofit organizations that provide tangible goods and services also may play a community advocacy role based on this enlarged definition (Balassiano & Chandler, 2010; Boris & Mosher-Williams, 1998; Schmid et al., 2008); however, the main focus in this article is advisory associations that have the core function of community advocacy and representation rather than service provision.

In Table 1, empirical studies on nonprofit effectiveness are classified using a functional typology of organizations on one dimension (service delivery vs. advocacy) and approaches to organizational effectiveness on the other dimension (unidimensional vs. framework-based). Some observations can be made from this classification. First, there has been continued interest in adopting a framework-based approach to examining the effectiveness of nonprofit and voluntary organizations through multiple dimensions or constituencies as opposed to a unidimensional one. The first-generation studies that take a unidimensional approach tend to focus on the goal attainment aspect of organizational effectiveness rather than the resource acquisition perspective; this is more evident for studies that focus on service-oriented nonprofits (Byington, Martin, DiNitto, & Maxwell, 1991; Lillis & Shaffer, 1977). A handful of studies, however, rely on a system resource approach to assess organizational effectiveness for both types of nonprofits (Crittenden, Crittenden, & Hunt, 1988; Provan, 1980). Currently, the dominant approach is the multidimensional framework (Dart, 2010), which identifies the main domains pertinent to nonprofit effectiveness. Several empirical studies have focused on identifying the different dimensions of organizational effectiveness particular to the organization under study (Andrews et al., 2010; Shilbury & Moore, 2006; Sowa et al., 2004).¹

Table 1.

Empirical Studies on Organizational Effectiveness of Nonprofit and Voluntary Sector

		Core functions
Approaches to effectiveness		Service provision	Advocacy
Unidimensional	Goal attainment	Lillis and Shaffer (1977) Byington, Martin, DiNitto, and Maxwell (1991) Torres, Zey, and McIntosh (1991)	Webb (1974) Mesch and Schwirian (1996)
	System resource	Provan (1980)	Crittenden, Crittenden, and Hunt (1988)
Framework-based	Multidimensional	Cameron (1981) Frisby (1986) Chelladurai and Haggerty (1991) Kushner and Poole (1996) Selden and Sowa (2004) Shilbury and Moore (2006)	Wilson and Zhou (1992) Goodlad et al. (1999) Andrews, Ganz, Baggetta, Han, and Lim (2010)
	Multiconstituency	Martin (1980) Fried and Worthington (1995) Herman and Renz (1997, 2004) Tassie, Murray, and Cutt (1998) Papadimitriou and Taylor (2000) Balser and McClusky (2005)	Worth (2003)

These dominant approaches to organizational effectiveness are limited for two reasons. First, unidimensional approaches tacitly assume that consensus exists regarding the organizational goals and evaluation criteria being used. The multidimensional approach attempts to overcome this limitation; however, it does not explicitly consider that key internal and external stakeholders might hold different objectives and viewpoints that shape evaluations on one or multiple dimensions of effectiveness. Thus, effectiveness evaluations inevitably favor one stakeholder group’s view over others. We argue that this is a limited approach not only for service-oriented nonprofits but also for voluntary civic associations and propose a multiple constituency framework to understand organizational effectiveness in such settings.

A Multiple Constituency Framework of Civic Association Effectiveness

We view the multiple constituency approach to organizational effectiveness as a modified goal attainment model, where different stakeholder perspectives exist in evaluation processes (Herman & Renz 1997). We argue that the multiple constituency framework adds necessary complexity to understanding organizational effectiveness, which particularly is appropriate for evaluating voluntary civic associations that serve internal and external stakeholder interests (Kanter & Summers, 1987). The multiple constituency approach enables a systematic way of understanding how different stakeholders assess organizational effectiveness from their respective standpoints rather than assuming shared agreement on evaluation perspectives.

As seen in Table 1, various empirical studies have used a multiple constituency framework but mostly focus on service-specific nonprofits (Balser & McClusky, 2005; Fried & Worthington, 1995; Herman & Renz, 1997; Martin, 1980; Papadimitriou & Taylor, 2000; Tassie, Murray, & Cutt, 1998). The multiple constituency framework has been applied usefully to nonprofits with a service delivery orientation as these organizations also have goals and outcomes that are difficult to evaluate using objective measures. Thus, a multiple constituency approach is conceptually appropriate as it provides a more comprehensive picture of organizational effectiveness.

Our research proposes that the multiple constituency framework also can be adopted as a structured way of understanding the evaluation of voluntary civic association effectiveness. Smith (1999a) characterizes community grassroots organizations as “generally focusing on local advocacy for local institutional changes and on better services for community or neighborhood residents” (p. 104) and also argues that more research should be conducted in studying the effectiveness of these voluntary civic associations, where the core function is not service provision (Smith, 1997). Unlike other empirical studies on service-specific nonprofits, few studies have used the multiple constituency framework to study these broadly defined advocacy organizations. One relevant study is Worth’s (2003) research on the effectiveness of rights-oriented environmental organizations in Western Australia. His study confirms that “there is no one common construction of the organizational or political effectiveness of social advocacy organizations” (p. 95). No studies systematically investigate the effectiveness of voluntary advisory associations, which differ from social movement organizations.

Using a multiple constituency framework to examine how evaluations vary among constituencies in voluntary association is crucial for several reasons. First, voluntary associations have outputs and outcomes, that are comparatively intangible and indirect in nature, such as having an impact on policy processes (Fried & Worthington, 1995). Like other types of organizations, some civic association outcomes may be quantifiable, whereas others may be more difficult to operationalize. For example, whether a voluntary civic association promotes diverse community interests in the policy process is much more difficult to evaluate than whether an association increases community outreach efforts.² Worth (2003) aptly points out,

the question of organizational and management effectiveness is very difficult to assess, as their main goal is to assist in changing a societal value or governmental policy. Even if the goal is achieved, it may be difficult to link the particular actions and strategies of an individual advocacy organization with any subsequent policy shift. (p. 85)

Therefore, understanding organizational effectiveness in an objectively measurable way is problematic and leads to the use of perceptual judgments of effectiveness, which the multiple constituency framework uses to improve evaluation processes and their interpretation.

Second, unlike bureaucratic nonprofits that mainly are run by professionals or paid staffs, active volunteers conduct the core activities of voluntary associations (Smith, 1999b), which complicates the understanding of organizational effectiveness. From the goal-attainment perspective, understanding organizational effectiveness becomes more complex because volunteers do not depend as heavily on these organizations as paid staffs (Pearce, 1993).³ Indeed, all organizations have issues of goal ambiguity (Chun & Rainey, 2005) and the multiplicity of objectives (Kanter & Summers, 1987); however, we argue that the degree to which this occurs is greater for volunteer-run associations. For instance, paid staffs, as part of their job responsibilities, must carry out the organization’s mission and official goals, whereas volunteers comparatively have more liberty to bring their own preferences and priorities into the organization in the absence of strong managerial control. Because of the lack of strong internal and external control mechanisms, volunteers active in advisory associations may have varying perspectives about organizational goals. Volunteer self-evaluations also tend to be more favorable than evaluations from external stakeholders. Hence, it is important to understand how volunteers evaluate organizational goals as it is these values that bind them to the organization and indicate their perspectives on what the organization should accomplish.

Finally, advisory civic associations, which often are required for consultation in public decision-making processes, involve multiple external stakeholders in addition to volunteers, such as administrators and elected officials who work closely with associations and whose work is affected by association activities and input. Although there may be some universally agreed on goals for advisory civic associations, the interests of multiple stakeholders may diverge, which in turn affects organizational effectiveness evaluations (Martin, 1980). Specifically, it is expected that the organizational interests of external stakeholders, such as local elected officials and street-level bureaucrats that interact frequently with advisory civic associations, will differ with those of association members and thus with overall effectiveness evaluations. In addition, conflicting role expectations of multiple constituencies may lead to divergence in effectiveness evaluations (Fried & Worthington, 1995). With organizations that address multiple goals and interests, privileging one perspective over another may be problematic; therefore, our study explores whether the effectiveness ratings of stakeholder groups of voluntary civic associations differ, and if they differ, how they differ.

Application of the Multiple Constituency Framework

This study enacts a research strategy for evaluating the organizational effectiveness of voluntary civic associations. The objectives are twofold: first, to assess whether there are group differences in the perceptual evaluations of organizational effectiveness of voluntary civic associations and, second, to describe such differences in the effectiveness ratings among constituency groups. The first research goal is addressed by using multivariate analysis of variance (MANOVA), which analyzes whether group differences exist in effectiveness ratings (Tsui & Ohlott, 1988). Second, to understand how the effectiveness ratings differ among groups, we conduct post hoc tests. We first proceed with explaining our empirical case, Los Angeles NCs and the organizational goals of multiple stakeholders.

Empirical Case: The Los Angeles NC System

The case that we investigate is Los Angeles NCs, advisory associations created as a result of City Charter reform in 1999 to make government more responsive to community needs (Box & Musso, 2004). The city outlines the NC system in its City Charter and requires NCs to adhere to open meeting laws and ethics standards, which reflects the public nature of NCs. Nevertheless, NCs are foremost self-organizing, grassroots associations with elected board members in their respective communities, which have specific boundaries. They are run entirely by volunteers and have significant latitude in determining their bylaws and activities on which they focus. To date, there are 90 certified NCs with each NC representing 38,000 residents on average (Department of Neighborhood Empowerment, 2011).

The Los Angeles City Charter and the city’s Plan for a Citywide System of NCs outline the official goals of the citizen participation system. The language in these documents represents a broadly interpreted consensus among constituency groups regarding the aim of the citywide system. For example, as described in the City Charter,

To promote more citizen participation in government and make government more responsive to local needs, a citywide system of neighborhood councils and a Department of Neighborhood Empowerment is created. Neighborhood councils shall include representatives of the many diverse interests in communities and shall have an advisory role on issues of concern to the neighborhood. (City of Los Angeles, 2000, Article IX, Sec. 900)

In addition, the City Charter mandates that NCs “monitor the delivery of City services” and have “periodic meetings with responsible officials of City departments” (City of Los Angeles, 2000, Article IX, Sec. 910). Thus, NCs aim not only to provide greater voice to citizens in local government affairs but also to improve the government delivery of public services and increase the responsiveness of government administrators and elected officials, who in addition to volunteer board members could be viewed as stakeholders in the system as they communicate and work with NCs. In other words, as outlined in the City Charter, volunteers in NCs involve “activism” (Musick & Wilson, 2008) intended to change the work of city government by communicating community preferences and other local information. Thus, the primary functions of NCs relate to increasing voice in local decision making.

To assess whether there are group differences in perceptual evaluations of organizational effectiveness of NCs, we employ survey data collected in 2005-2006 from three constituency groups: neighborhood council volunteer board members (NC), Department of Neighborhood Empowerment (DONE) project coordinators (PC) and city council (CC) staff. These three groups are identified as major stakeholders to NCs because they work most closely within and with these organizations. In particular, the 1999 City Charter reform created the DONE to oversee and assist NC activities, and thus PCs from DONE are identified as one major constituency group that provides a street-level administrators’ view of effectiveness. City Council staff members provide the perspective of local elected officials as they work as liaisons between elected officials and NCs. They usually attend NC meetings regularly to obtain community sentiments on important issues, communicate the positions of elected officials and provide information regarding legislative activities that concern the community. Interviews with CC staff demonstrate that the directives of elected officials shape their interactions with and attitudes toward NCs, thus making CC staff perspectives a reasonable proxy for the perspectives of elected officials.

Disparate Organizational Goals of Multiple Constituencies

Grounded on previous studies and our fieldwork, we expect that discrepancies in effectiveness ratings from the three constituency groups will be based on the respective organizational goals of the different evaluators outlined in Table 2 as the three constituency groups likely view effectiveness from different perspectives and with different motivations. For example, because they are entirely run by volunteers, NCs may not view their official goals with similar importance or will consider some objectives more attractive to their community and individual needs than others. According to open-ended survey responses, NC members commonly view their goals as improving their community and having a voice in the decisions that shape their community. Thus, NCs may have a perspective of organizational effectiveness that reflects whether particular conceptions of community improvement are achieved. Moreover, NCs likely desire more rather than less community influence over local decision making.

Table 2.

Differences in Goals in the Citywide NC System

Evaluators	Organizational goals
Neighborhood council board member	Improving the community Increasing voice in decision-making processes
City council staff	Promoting interests of their constituencies, which would include multiple NCs and other stakeholders in their district
Project coordinator	Assisting NCs in their development Regulating NCs when rule violations occur in the operation of NCs

Note: NC = neighborhood council.

In contrast, for elected officials, there may be differences in role expectations of NCs because their main goals pertain to serving the interests of their entire district or constituencies and not necessarily one particular NC as conveyed in CC staff interviews. To most elected officials, NCs are viewed as strictly advisory in nature rather than as direct decision makers. Therefore, the goals of an NC may conflict with the goals of the local CC office, which may influence CC staff evaluations of NCs. For example, one CC staff member interviewed described the goal of the NC system as establishing a strong mechanism for obtaining community input from the advisory NCs to be relayed to the CC, which serves as the policy-making body (personal interview, June 8, 2006).

Last, for the PCs, their main organizational goal is the successful development of the NC system for the respective communities in which they oversee. They are concerned about the development of the NC system as a whole and play a role in guarding against violations of any rule in the internal operations of NCs, such as the City Charter’s specific directive that NCs represent the diverse interests of the community. Thus, PCs may be more critical of NCs in this respect than other stakeholders based on the City Charter’s mandate.

Data and Measures

To explore these potential differences, we analyze indicators from three surveys that measure the organizational effectiveness of NCs and that are derived from language in the City Charter and Plan for NCs that directs NCs to (a) represent their communities, (b) deliberate on service needs, and (c) act as a communication channel between community stakeholders and department officials. In the survey, PCs evaluated the NCs with which they worked; CC staff evaluated only the NCs located in their own district; and NC board members provided self-evaluations of their organizations. Using a 4-point Likert-type scale ranging from 1 = poor to 4 = excellent, six survey items are included in this study regarding how effective NCs are in (a) promoting more citizen participation in government, (b) working to solve problems in the neighborhood, (c) including representatives of the many diverse interests in the neighborhood, (d) advising the city on citywide policies, (e) advising the city on local service needs, and (f) advising the city on land use. Overall, 710 board members from 82 NCs responded to the survey with a response rate of 47%; due to missing data, our analysis includes 530 observations from this survey. Nineteen CC staff members from 12 of the city’s 15 council districts evaluated 62 NCs. Last, the PCs from DONE evaluated 63 to 69 NC organizations depending on the evaluation item. Fourteen of the city’s 15 PCs at that time participated in the evaluations.

Statistical Analyses

In this study, we use MANOVA and post hoc tests in tandem to determine whether perceptual effectiveness ratings differed on average across the three constituent groups. Using these statistical procedures, the first research purpose is to investigate the differences among effectiveness ratings by the three groups. Multivariate analysis of variance is a method to evaluate whether population means on a set of dependent variables vary across different factors, or in this case groups, and thus addresses the question of whether evaluations from multiple constituencies differ. The MANOVA procedure extends the concept of univariate analysis of variance (ANOVA) and is appropriate for situations where several dependent variables are considered simultaneously (Field, 2009; Huberty & Olejnik, 2006; Norušis, 2008). The null hypothesis in MANOVA is the equality of population means among groups on each dependent variable and also equality among groups on linear combinations of these dependent variables. The main justification for using multivariate analysis is that the relationships among multiple dependent variables are taken into account, whereas a series of univariate analyses cannot address intercorrelations among dependent variables (Haase & Ellis, 1987). As Table 3 shows, the six effectiveness indicators are moderately correlated.

Table 3.

Bivariate Correlations Coefficients

Effectiveness items	1	2	3	4	5	6
1. Promoting more citizen participation in government	1
2. Advising the city on citywide policies	.654**	1
3. Including representatives of the many diverse interest	.741**	.571**	1
4. Working to solve problems in the neighborhood	.755**	.734**	.720**	1
5. Advising the city on local service needs	.676**	.723**	.659**	.826**	1
6. Advising the city on land use	.542**	.674**	.527**	.715**	.788**	1

Note: **p < .01

Once a significant difference between groups is found through MANOVA, post hoc tests—Tukey’s honestly significant difference (HSD) test and descriptive discriminant analysis (DDA)—are conducted to describe obtained group differences. As post hoc comparisons, Tukey’s HSD test followed by univariate ANOVAs compares all possible group means in their differences; however, DDA is considered a superior method to univariate comparisons on each of the dependent variables because it reduces the dependent variables to theoretical dimensions and further allows one to understand group differences (Field, 2009). Simply put, DDA is the reverse of MANOVA, sharing the same assumptions and test statistics but providing a follow-up measure to explain group differences. In sum, MANOVA focuses on whether there is a difference between groups, and DDA focuses on finding the linear combinations of the dependent variables or the dimensions that can maximally separate the groups (Field, 2009).

Whether Group Differences Exist in Effectiveness Ratings?

Descriptive statistics on the effectiveness ratings for the three groups are reported in Table 4.⁴ Overall, the self-evaluations of NCs by their board members generally tend to be higher than the other two group evaluations. In particular, greater discrepancy exists between NC and PC evaluations than between NC and CC evaluations. The evaluations from NCs and CCs tend to overlap. In a multivariate setting, we conduct a MANOVA to assess whether statistically significant group differences exist in the six effectiveness ratings.⁵ The result from MANOVA on the linear combination of six dependent variables indicates a significant group difference, Wilks’s Λ = .726, F(12, 380) = 5.511, p < .001. We, therefore, conclude that the three constituent groups evaluated the various dimensions of organizational effectiveness of NCs dissimilarly.⁶

Table 4.

Effectiveness Mean Scores for Three Constituency Groups

	NC (n = 82)		PC (n = 55)		CC (n = 61)		ANOVA	Tukey’s HSD
Effectiveness items	M	SD	M	SD	M	SD	F	Comparison
Promoting more citizen participation in government	2.412	0.493	1.836	0.877	2.548	0.901	14.721	CC, NC > PC
Advising the city on citywide policies	2.449	0.474	1.982	0.850	2.518	0.961	8.620	CC, NC > PC
Including representatives of the many diverse interests	2.647	0.479	2.000	0.943	2.354	0.956	11.080	NC, CC > PC
Working to solve problems in the neighborhood	2.815	0.511	2.127	0.862	2.597	0.873	14.311	NC, CC > PC
Advising the city on local service needs	2.751	0.502	2.145	0.931	2.684	0.948	10.766	NC, CC > PC
Advising the city on land use	2.730	0.672	2.364	0.950	2.790	0.946	4.367	CC, NC > PC

Note: Scales from 1 = poor, 2 = fair, 3 = good, 4 = excellent.

How the Effectiveness Ratings Separate Constituency Groups?

The goal of the post hoc analyses is to determine whether these linear combinations of effectiveness measures can be used to discriminate among the three constituency groups. That is, the question of interest is to describe how the combinations of the six effectiveness ratings separate NC, CC, and PC evaluations. First, as follow-up tests, multiple univariate ANOVAs with Tukey’s HSD tests were conducted to compare all possible group means in their differences. As reported in Table 4, univariate comparisons indicate the NC and CC ratings differ from PC ratings, but the differences between NCs and CCs are statistically insignificant. To understand the group differences in a multivariate setting, we also conduct DDA as a post hoc test. With our three groups, two linear discriminant functions are obtained where the first function maximally discriminates the groups better than the second one.⁷

The DDA results in Table 5 show that two dimensions describe group differences in the NC effectiveness ratings. Table 5 reports the discriminant loadings of the structure matrix, which represent the correlation of each observed variable (i.e., the six effectiveness measures) with each unobserved discriminant function (i.e., two dimensions). These correlations are analogous to factor loadings in factor analysis and can be used to assign meaningful labels to the discriminant functions. The two effectiveness measures—(a) working to solve problems in the neighborhood and (b) including representatives of the many diverse interests—load higher on the first dimension; and this function can be considered as an internal dimension of organizational effectiveness, which discriminates the groups better than the second dimension. The second dimension (function) correlates with (a) promoting more citizen participation in government, (b) advising the city on citywide policies, (c) advising the city on local service needs, and (d) advising the city on land use. The common feature of the second dimension is focusing on the effectiveness of the external roles of NCs in creating vertical ties with city government.

Table 5.

Structure Matrix: Correlation of Each Variable to the Discriminant Function

	Function
Effectiveness items	1	2
Working to solve problems in the neighborhood	.760^a	.506
Including representatives of the many diverse interests	.727^a	.335
Promoting more citizen participation in government	.413	.844^a
Advising the city on citywide policies	.359	.622^a
Advising the city on local service needs	.548	.578^a
Advising the city on land use	.250	.445^a

Largest absolute correlation between each variable and any discriminant function.

To better describe the group differences in these two dimensions, linear discriminant function group centroids, representing the average discriminant score for participants in the three groups (i.e., effectiveness measures in this case) in a single point, are shown in Figure 1. One can observe that NC board evaluations are separated from PC and CC evaluations within the first dimension, which is focused on the internal aspects of the organizational effectiveness of NCs. Self-evaluation of NC board ratings were the highest followed by CC and PC staff. Although not as clearly as in the first dimension, the three constituency groups also are separated within the second linear discriminant function—the external dimension of organizational effectiveness—where the group means of CC evaluations is the highest followed by NC and PC evaluations.

Figure 1.

Linear discriminant functions plot of group centroids

Discussion

We find divergence in the effectiveness ratings of NCs among different stakeholder groups, which confirms findings from previous multiple constituency framework studies that a common construct of organizational effectiveness is not found. We also find differences in how effectiveness ratings differ with discrepancies emerging along different dimensions of effectiveness. These conclusions from our exploratory analysis directly point to the difficulties in analyzing organizational effectiveness. It reflects the practical challenges that arise in evaluation processes and confronts the dilemma in answering the question of effectiveness for whom.

Ultimately, we subscribe to the use of the multiple constituency framework as used by Connolly et al. (1980) as an empirical technique for the collection of organizational effectiveness information and to their belief that it is “arbitrary to label one of these perspectives a priori as the correct one” (p. 212). We do not argue that the multiple constituency approach allows evaluators to judge which stakeholder group provides the most valid evaluation perspective. Rather as an evaluation strategy, the multiple constituency framework allows researchers to discover differences in evaluations by various internal and external stakeholder groups, which assists in understanding why these differences occur, in uncovering potential biases in the perceptual understandings of organizational goals and effectiveness, and in resolving or creating mutual understanding about these differences.

For instance, our findings indicate that different constituency groups evaluate NCs differently, which suggests that discrepancies exist in the goals and interests on which the three constituency groups concentrate. In other words, the discrepancies between responses may occur because evaluators view organizational effectiveness from different perspectives and with different motivations, which implies that stakeholder groups may not share similar objectives as presented in Table 2. Each of the judgments on effectiveness is situated inevitably in the evaluator’s context, which is a crucial point to remember in organizational effectiveness evaluations. Rather than privileging one perspective over another, in our view, each constituency group provides a partial and yet important perspective of organizational effectiveness of voluntary associations.

Specifically, NC board members’ self-evaluations on the internal dimension of effectiveness—which highly correlates with neighborhood problem solving and increasing the diversity of representatives and substantive interests in participation processes—diverge from PC and CC evaluations. Board member self-evaluations rate this dimension the highest compared with the other two groups. On one hand, this finding may be unsurprising as board members tend to self-evaluate their activities more positively; on the other hand, they have intimate knowledge of the day-to-day efforts volunteer board members contribute to internal organizational activities. Alternatively, from an organizational perspective, the internal activities of NCs serve the particular concerns of active board members—e.g., concerns with neighborhood beautification or land use issues—and thus board members may evaluate their activities more favorably as they themselves shape the nature of activities to meet their particular needs.

The question remains, however, whether NC activities serve not only the particular interests of board members but also the broader community; thus, in this respect, other stakeholders will perceive their effectiveness differently. Considering evidence from our fieldwork on NCs where outreach has been a consistent problem for NC organizing activities, we believe the evaluations by PC and CC staff provide alternative perspectives on the effectiveness of NCs for this internal dimension of collaboration and participation within the community. Previous studies of descriptive and substantive representation of NCs found an overall lack of representative legitimacy of the boards (Guo & Musso, 2007; Jun, 2005). Reflecting this concern, PC evaluations are more critical compared with other groups on this internal effectiveness dimension. This is because DONE’s main organizational goal is the successful development of the NC system. As street-level administrators, the role of PCs is to assist and regulate NCs in their daily operations and to uphold the NC mandates in the City Charter and the Plan. Owing to their role and goals, PCs observe NC effectiveness in this first dimension as it relates to the goals of the NC system outlined in the City Charter, which is to include the city’s diverse interests.

Although the externally oriented dimension, which involves participating in policy-formation processes in local governance and collaborating with city government, explains group differences less powerfully than the internal dimension, we find that the group means of the external dimension for CC staff ratings were the highest followed by the ratings of NC and PC evaluators. In this case, we reiterate that the evaluator’s particular interests and perspectives (in Table 2) shape their evaluations. This external dimension of effectiveness is of greater interest to CC staff than the internal dimension. To explain, the impetus for 1999 charter reform that created the NC system was to open policy processes to include more participation from local communities. The creation of the citywide NC system in effect precipitated the involvement of more stakeholder groups, and this reform has affected the work of CC staff. From fieldwork observations, CC offices increasingly have sent their staff to NC meetings in their district. Subsequently, this reallocation of CC staff time devoted toward NCs lead to favorable perceptions of NC effectiveness as CC offices often are connected to and participate in NC activities. In other words, CC staff members attend NC meetings and gather community perspectives on policy issues; thus, CC staffs perceive NCs to facilitate greater participation in local governance processes. In interviews, several CC staff members said that NCs in some ways assisted their work as a formal venue to hear community concerns and share information.

Arguably, from the perspective of the CC staff, the higher effectiveness ratings in this external dimension suggest the success of charter reform in that it satisfies the need for more participation in urban governance. Indeed, NCs have become involved in city policy and governance on several levels ranging from the parochial to citywide issues, such as policies regarding emergency responses to burglar alarms, the city’s water rate and CC term limits. For example, in 2004, NCs effectively opposed an 18% rate hike proposed by the city’s Department of Water and Power. This mobilization of opposition is considered one of the most influential actions of NCs since their implementation in 2001.

Different expectations and interests regarding the role of NCs, however, exist that may explain discrepancies in ratings of effectiveness between CC staff and NC board members regarding this external dimension. To explain, the main goals of local elected officials pertain to serving the interests of their entire district and not necessarily one particular NC. According to interviews with CC staff, NCs are often considered as an additional stakeholder group with which the CC must work rather than as special advisory bodies accorded special privileges. In contrast, NC board members have advocated for a greater role in decision-making processes beyond an informal advisory capacity. Self-evaluations of NCs for this external dimension are lower than the ratings of CC staff, which reflects this desire from NC board members for an expanded role in agenda-setting processes. For instance, between 2005 and 2006, NCs played the most proactive role in the agenda-setting process by seeking specific authority to propose items for CC consideration through noncommunity impact statements (Los Angeles City Clerk, 2005). If passed, this initiative would have enabled NCs to submit proposals directly for CC consideration. This initiative reflects increased notions of political efficacy that NCs gained over the course of their development despite the failure of this effort. The divergent evaluations of board members and CC staff reflect differences that emerge due to the particular goals and interests of each stakeholder group and thus different definitions of effectiveness.

The major implication from this analysis is that multiple constituencies involved in volunteer-run associations may have different operative goals. These differences will be reflected in divergent evaluations because organizational effectiveness is measured with reference to such objectives. Thus, our study implications provide a crucial corrective to previous evaluation studies that assume agreement on organizational objectives and attribute divergence in evaluations to rating disagreements. Instead, we argue that the evaluation of the effectiveness of civic associations needs to overtly consider the specific goals and priorities of major stakeholders. This is all the more critical for voluntary associations as these organizations are shaped by the goals of multiple stakeholders in the absence of strong internal and external control mechanisms. Moreover, these findings apply to evaluations of not only voluntary civic associations but also other types of organizations with stakeholder groups with divergent interests.

One of the limitations of this analysis is that we did not include the relative importance of goals among the effectiveness items due to limitations in our survey data. From our results, we can, however, infer that the formal objectives outlined in the charter are not equally important for all stakeholder groups. In addition, our analysis does not include how individual volunteers differ in the goals and motivations they bring into the organizations. Future studies should examine how differences in objectives among volunteers lead to different perceptions of effectiveness.

A remaining question is whether voluntary civic associations can and should have universally agreed on notions of organizational effectiveness. While Connolly et al. (1980) assert that no single conclusion on organizational effectiveness can be drawn, Herman and Renz (1999) argue that multiple constituencies likely differ in their criteria regarding organizational effectiveness but congruence on effectiveness judgments may result from social processes, such as ongoing dialogue among stakeholders. Our conclusion is not to suggest that this is impossible but that if in such situations where views on effectiveness differ, managers or evaluators involved could devise interactive social processes that potentially may lead to a mutual understanding of such dissimilarities (Herman & Renz, 1997). These social processes may serve to resolve and overcome differences or identify potential biases in views of effectiveness. To fully understand these social processes in evaluation, however, involves incorporating a time dimension in empirical research. In this article, we only focus on a cross-sectional evaluation and do not address whether different evaluations converge over time. Future research should examine particularly whether and how divergent perspectives on organizational effectiveness might be reconciled over time. Further theoretical and empirical examination of civic associations that primarily play an advisory and advocacy role in the community will assist in understanding the value of such organizations for improved democratic processes in urban governance.

Conclusion

The main goal of this article is to propose a framework to understand the organizational effectiveness of voluntary civic associations, which often require evaluations using multiple perspectives. In this research, we underscore the distinction between voluntary civic associations that serve in an advisory role to local governments and bureaucratically managed organizations that provide tangible goods or services (Andrews et al., 2010) and point to the limited use of the multiple constituency framework in evaluation studies of civic associations that play an advisory and representative role in the community. Our research indicates that because of the greater divergence of objectives held by multiple constituencies in voluntary advisory associations, perceptions of effectiveness also diverge significantly. That is, external stakeholders of volunteer-run organizations have less power in imposing their own objectives and definitions of effectiveness because volunteers are tied to the organization by their own values in contrast to, for example, firms and professional nonprofits, where members may adhere more closely to defined organizational interests because of employee–wage relationships. Thus, the findings from this research have theoretical and practical importance for studying the organizational effectiveness of advisory civic associations from a multiple constituency approach.

To theoretically advance our understanding of why certain civic associations are more effective than others, namely, to understand the determinants of civic association effectiveness, we first need to explore the ways in which we can measure their effectiveness more accurately. Current developments in the organizational effectiveness literature, especially for nonprofit and voluntary organizations, should direct more attention to understanding how we can appropriately measure organizational effectiveness before proceeding to understand its determinants. A dearth of studies seriously attempts to assess civic association effectiveness, and a reliance on a single stakeholder group for providing effectiveness ratings is incomplete; rather, a complete picture of organizational effectiveness requires multiple perspectives. Based on these notions, a multiple constituency approach provides a systematic and structured way to understand the multifaceted and complex concept of voluntary civic association effectiveness. The multiple constituency framework illuminates important differences in perspectives on organizational effectiveness that may arise from stakeholder differences regarding organizational objectives. It provides a crucial starting point for researchers and practitioners for understanding why these differences emerge and the implications of these differences. This understanding is important for future work that explores whether and how stakeholders with different objectives in the same organization can accomplish organizational goals that serve their diverse interests.

Footnotes

Acknowledgements

The authors would like to express their gratitude to the anonymous reviewers for their valuable comments on earlier drafts of this manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

Bios

Kyu-Nahm Jun is an assistant professor at Wayne State University, Detroit, Michigan. Her research focuses in assessing the democratic legitimacy of place-based associations that represent geographically bounded urban communities. Other research interests include the role of information technology between citizen and government agencies in urban governance.

Ellen Shiau is a PhD candidate in public policy at the University of Southern California, Los Angeles. Her research interests are broadly on civic participation and urban political processes, and community and economic development.

References

Alexander

Brudney

J. L.

Yang

(2010). Introduction to the symposium: Accountability and performance measurement: The evolving role of nonprofits in the hollow state. Nonprofit and Voluntary Sector Quarterly, 39, 565-570.

Andrews

K. T.

Ganz

Baggetta

Han

Lim

(2010). Leadership, membership, and voice: Civic associations that work. American Journal of Sociology, 115, 1191-1242.

Balassiano

Chandler

S. M.

(2010). The emerging role of nonprofit associations in advocacy and public policy: Trends, issues, and prospects. Nonprofit and Voluntary Sector Quarterly, 39, 946-955.

Balser

McClusky

(2005). Managing stakeholder relationships and nonprofit organization effectiveness. Nonprofit Management and Leadership, 15, 295-315.

Berry

J. M.

Portney

K. E.

Thomson

(1993). The rebirth of urban democracy. Washington, DC: Brookings Institution.

Bluedorn

A. C.

(1980). Cutting the Gordian knot: A critique of the effectiveness tradition in organizational research. Sociology and Social Research, 64, 477-496.

Boris

Mosher-Williams

(1998). Nonprofit advocacy organizations: Assessing the definitions, classifications, and data. Nonprofit and Voluntary Sector Quarterly, 27, 488-506.

Boschken

H. L.

(1994). Organizational performance and multiple constituencies. Public Administration Review, 54, 308-312.

Box

R. C.

Musso

J. A.

(2004). Experiments with local federalism: Secession and the neighborhood council movement in Los Angeles. American Review of Public Administration, 34, 259-276.

10.

Byington

D. B.

Martin

P. Y.

DiNitto

D. M.

Maxwell

M. S.

(1991). Organizational affiliation and effectiveness: The case of rape crisis centers. Administration in Social Work, 15(3), 83-103.

11.

Cameron

K. S.

(1981). Domains of organizational effectiveness in colleges and universities. Academy of Management Journal, 24(1), 25-47.

12.

Cameron

K. S.

(1986). Effectiveness as paradox: Consensus and conflict in conceptions of organizational effectiveness. Management Science, 32, 539-553.

13.

Cameron

K. S.

Whetten

D. A.

(1983). Organizational effectiveness: A comparison of multiple models. New York, NY: Academic Press.

14.

Chelladurai

Haggerty

T. R.

(1991). Measures of organizational effectiveness of Canadian national sport organizations. Canadian Journal of Sport Science, 16(2), 126-133.

15.

Chun

Y. H.

Rainey

H. G.

(2005). Goal ambiguity in U.S. federal agencies. Journal of Public Administration Research and Theory, 15(1), 1-30.

16.

City of Los Angeles. (2000). Los Angeles charter and administrative code. Retrieved from http://www.amlegal.com/nxt/gateway.dll?f=templates&fn=default.htm&vid=amlegal:laac_ca

17.

Connolly

Conlon

E. J.

Deutsch

S. J.

(1980). Organizational effectiveness: A multiple-constituency approach. Academy of Management Review, 5, 211-217.

18.

Crittenden

W. F.

Crittenden

V. L.

Hunt

T. G.

(1988). Planning and stakeholder satisfaction in religious organizations. Journal of Voluntary Action Research, 17(2), 60-73.

19.

Dart

(2010). A grounded qualitative study of the meanings of effectiveness in Canadian “results-focused” environmental organizations. Voluntas: International Journal of Voluntary and Nonprofit Organizations, 21, 202-219.

20.

Department of Neighborhood Empowerment. (2011). What are neighborhood councils? Retrieved from http://done.lacity.org/dnn/Default.aspx?tabid=97

21.

Field

(2009). Discovering statistics using SPSS. Thousand Oaks, CA: SAGE.

22.

Forbes

D. P.

(1998). Measuring the unmeasurable: Empirical studies of nonprofit organization effectiveness from 1977 to 1997. Nonprofit and Voluntary Sector Quarterly, 27, 183-202.

23.

Fried

B. J.

Worthington

(1995). The multiconstitutency approach to organizational performance assessment: A Canadian mental health system application. Community Mental Health Journal, 31(1), 11-24.

24.

Frisby

(1986). The organizational structure and effectiveness of voluntary organizations: The case of Canadian national sport governing bodies. Journal of Park and Recreation Administration, 4(3), 61-74.

25.

Garson

G. D.

(2010). Multivariate GLM, MANOVA, and MANCOVA. Statnotes: Topics in multivariate analysis. Retrieved from http://faculty.chass.ncsu.edu/garson/PA765/manova.htm

26.

Georgopoulos

B. S.

Tannenbaum

A. S.

(1957). A study of organizational effectiveness. American Sociological Review, 22, 534-540.

27.

Goodlad

Flint

Kearns

Keoghan

Paddison

Raco

(1999). The role and effectiveness of community councils with regard to community consultation. Edinburgh, UK: The Commission on Local Government and the Scottish Parliament.

28.

Goodman

P. S.

Atkin

R. S.

Schoorman

F. D.

(1983). On the demise of organizational effectiveness studies. In Cameron

K. S.

Whetten

D. A.

(Eds.), Organizational effectiveness: A comparison of multiple models (pp. 163-183). New York: Academic Press.

29.

Guo

Musso

J. A.

(2007). Representation in nonprofit and voluntary organizations: A conceptual framework. Nonprofit and Voluntary Sector Quarterly, 36, 308-326.

30.

Haase

R. F.

Ellis

M. V.

(1987). Multivariate analysis of variance. Journal of Counseling Psychology, 34, 404-413.

31.

Herman

R. D.

(1992). Nonprofit organization effectiveness: At what, for whom, according to whom? Nonprofit and Voluntary Sector Quarterly, 21, 411-415.

32.

Herman

R. D.

Renz

D. O.

(1997). Multiple constituencies and the social construction of nonprofit organization effectiveness. Nonprofit and Voluntary Sector Quarterly, 26, 185-206.

33.

Herman

R. D.

Renz

D. O.

(1999). Theses on nonprofit organizational effectiveness. Nonprofit and Voluntary Sector Quarterly, 28(2), 107-126.

34.

Herman

R. D.

Renz

D. O.

(2004). Doing things right: Effectiveness in local nonprofit organizations: A panel study. Public Administration Review, 64, 694-704.

35.

Hitt

M. A.

(1988). The measuring of organizational effectiveness: Multiple domains and constituencies. Management International Review, 28(2), 28-40.

36.

Huberty

C. J.

Olejnik

(2006). Applied MANOVA and discriminant analysis (2nd ed.). New York, NY: Wiley-Interscience.

37.

Jun

K.-N.

(2005). The democratic legitimacy of community associations: Los Angeles neighborhood councils in context (Unpublished doctoral dissertation). University of Southern California, Los Angeles.

38.

Kanter

R. M.

Summers

(1987). Doing well while doing good: Dilemmas of performance measurement in nonprofit organizations and the need for a multiple-constituency approach. In Powell

W. W.

(Ed.), The nonprofit sector: A research handbook (1st ed., pp. 154-166). New Haven, CT: Yale University Press.

39.

Kushner

R. J.

Poole

P. P.

(1996). Exploring structure–effectiveness relationships in nonprofit arts organizations. Nonprofit Management and Leadership, 7(2), 119-136.

40.

Lillis

C. M.

Shaffer

P. L.

(1977). Economic output as an organizational effectiveness measure for universities. Academy of Management Journal, 20, 476-482.

41.

Los Angeles City Clerk. (2005). Council File: 05-0575. Retrieved from http://cityclerk.lacity.org/CFI/displayOnlinedocument.cfm?SRT=D1&cfnum=05-0575

42.

Martin

P. Y.

(1980). Multiple constituencies, differential power, and the question of effectiveness in human service organizations. Journal of Sociology and Social Welfare, 7, 801-816.

43.

Mesch

G. S.

Schwirian

K. P.

(1996). The effectiveness of neighborhood collective action. Social Problems, 43, 467.

44.

Murray

(2010). Evaluating the effectiveness of nonprofit organizations. In Renz

Herman

R. D.

(Eds.), The Jossey-Bass handbook of nonprofit leadership and management (pp. 431-458). San Francisco, CA: John Wiley.

45.

Musick

M. A.

Wilson

(2008). Volunteers: A social profile. Bloomington, IN: Indiana University Press.

46.

Musso

J. A.

Weare

Bryer

Cooper

T. L.

(2011). Toward “strong democracy” in global cities? Social capital building, theory-driven reform, and the Los Angeles neighborhood council experience. Public Administration Review, 71(1), 102-111.

47.

Norman

(2010). Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Sciences Education, 15, 625-632.

48.

Norušis

(2008). SPSS 16.0 statistical procedures companion. Upper Saddle River, NJ: Prentice Hall.

49.

Papadimitriou

Taylor

(2000). Organisational effectiveness of Hellenic national sports organisations: A multiple constituency approach. Sport Management Review, 3(1), 23-46.

50.

Pearce

J. L.

(1993). Volunteers: The organizational behavior of unpaid workers. London, UK: Routledge.

51.

Price

J. L.

(1972). The study of organizational effectiveness. Sociological Quarterly, 13(1), 3-15.

52.

Provan

K. G.

(1980). Board power and organizational effectiveness among human service agencies. Academy of Management Journal, 23, 221-236.

53.

Putnam

R. D.

(2000). Bowling alone: The collapse and revival of American community. New York, NY: Simon & Schuster.

54.

Quinn

R. E.

Rohrbaugh

(1981). A competing values approach to organizational effectiveness. Public Productivity Review, 122-140.

55.

Quinn

R. E.

Rohrbaugh

(1983). A spatial model of effectiveness criteria: Towards a competing values approach to organizational analysis. Management Science, 29, 363-377.

56.

Schmid

Bar

Nirel

(2008). Advocacy activities in nonprofit human service organizations. Nonprofit and Voluntary Sector Quarterly, 37, 581-602.

57.

Selden

S. C.

Sowa

J. E.

(2004). Testing a multi-dimensional model of organizational performance: Prospects and problems. Journal of Public Administration Research and Theory, 14, 395-416.

58.

Shilbury

Moore

K. A.

(2006). A study of organizational effectiveness for National Olympic Sporting Organizations. Nonprofit and Voluntary Sector Quarterly, 35(1), 5-38.

59.

Smith

D. H.

(1997). Grassroots associations are important: Some theory and a review of the impact literature. Nonprofit and Voluntary Sector Quarterly, 26, 269-306.

60.

Smith

D. H.

(1999a). The effective grassroots association, Part II: Organizational factors that produce external impact. Nonprofit Management and Leadership, 10(1), 103-116.

61.

Smith

D. H.

(1999b). The effective grassroots association, Part I. Nonprofit Management and Leadership, 9, 443-456.

62.

Sowa

J. E.

Selden

S. C.

Sandfort

J. R.

(2004). No longer unmeasurable? A multidimensional integrated model of nonprofit organizational effectiveness. Nonprofit and Voluntary Sector Quarterly, 33, 711-728.

63.

Stevens

(2002). Applied multivariate statistics for the social sciences. Mahwah, NJ: Lawrence Erlbaum.

64.

Tabachnick

B. G.

Fidell

L. S.

(2006). Using multivariate statistics. Boston, MA: Pearson.

65.

Tassie

Murray

Cutt

(1998). Evaluating social service agencies: Fuzzy pictures of organizational effectiveness. Voluntas: International Journal of Voluntary and Nonprofit Organizations, 9(1), 59-79.

66.

Torres

C. C.

Zey

McIntosh

W. A.

(1991). Effectiveness in voluntary organizations: An empirical assessment. Sociological Focus, 24(3), 157-174.

67.

Tsui

A. S.

(1990). A multiple-constituency model of effectiveness: An empirical examination at the human resource subunit level. Administrative Science Quarterly, 35, 458-483.

68.

Tsui

A. S.

Ohlott

(1988). Multiple assessment of managerial effectiveness: Interrater agreement and consensus in effectiveness models. Personnel Psychology, 41, 779-803.

69.

Webb

R. J.

(1974). Organizational effectiveness and the voluntary organization. Academy of Management Journal, 17, 663-677.

70.

Wilson

C. D.

Zhou

Y. H. A.

(1992). Measuring effectiveness in citizens’ participation organizations: An exploratory study. Urban Review, 24(1), 23-37.

71.

Worth

(2003). The WA forest conflict: The construction of the political effectiveness of advocacy organisations. Journal of Australian Studies, 27(78), 83-96.

72.

Yuchtman

Seashore

S. E.

(1967). A system resource approach to organizational effectiveness. American Sociological Review, 32, 891-903.

73.

Zammuto

R. F.

(1984). A comparison of multiple constituency models of organizational effectiveness. Academy of Management Review, 9, 606-616.