Different Ways to Disagree

Abstract

Organizational communication processes are complex, but all too often, researchers oversimplify the study of these processes by relying on a single method. Particularly when scholars and practitioners partner together to solve organizational problems, meaningful results require methodological flexibility and diversity. As an exemplar of the fit between mixed methods research and engaged scholarship, the present project used surveys, observations, and interviews to study the process of dissent in two organizations. Results emphasized that each organization’s mission was a key factor in how employees expressed disagreement and how others responded. Emergent research questions and sequential data collection revealed insights that would have been missed in single method studies and provided more complete insights for the nonacademic practitioners involved in the study.

Keywords

mixed methods research sequential studies engaged scholarship organizational dissent

There has been a growing emphasis placed on not just applying academic results to business contexts but also on engaging with business leaders in identifying important questions to be answered (Barge & Shockley-Zalabak, 2008; Van de Ven & Johnson, 2006; Wensley, 2011). This type of partnership does not always fit neatly in paradigmatic ideals to which scholars have become accustomed. Rather, nonacademic practitioners may pose questions or issues that are more complex than can be addressed from a single research tradition. Mixed methods research is better poised to address complex problems (Hesse-Biber & Johnson, 2013). Partnerships between academic researchers and nonacademic practitioners demonstrate the value of mixed methods research, because such activity affords scholars the tools necessary to understand the complexity in organizational communication processes.

One area in which to explore the value of mixed methods for engaged scholarship is organizational dissent. Research has demonstrated that dissent is associated with increased creativity (De Dreu & West, 2001; Mitchell, Nicholas, & Boyle, 2009) and decreased groupthink (Janis, 1982). However, most studies of dissent have exclusively used self-report surveys of dissenters (Garner, in press), a method that may be particularly susceptible to bias given the sensitive nature of dissent (Jehn & Jonsen, 2010). Because of the traditional reliance on a single method of analysis, organizational dissent represents an ideal arena in which to examine the advantages of mixed methods research for engaged scholarship. The present study combines a quantitative self-report survey with observations and follow-up interviews to provide a richer understanding of dissent. In so doing, these results offer an exemplar of the benefits of mixed methods research for engaging with organizational leaders and demonstrate the importance of serendipitous findings that would have been missed by single method studies.

Mixed Methods Research, Pragmatism, and Engaged Scholarship

While some scholars may label themselves as positivists or interpretivists, many mixed methods researchers use pragmatism to define their metatheoretical assumptions (Teddlie & Tashakkori, 2009, 2010). Though not synonymous with mixed methods research, pragmatism is a good platform for mixed methods (Biesta, 2010). This perspective focuses on a problem to be resolved rather than tightly held positions on realism and knowledge (Creswell & Plano Clark, 2011; Tashakkori & Teddlie, 1998), which allows a researcher to avoid the dichotomy of positivism and interpretivism (Feilzer, 2010).

Engaged scholarship is one expression of a pragmatist position. This type of scholarship describes a partnership between academic and nonacademic stakeholders in order to use the knowledge and abilities of each to define solutions to problems (Van de Ven & Johnson, 2006). These partnerships are built on the idea that collaboration between academic researchers and nonacademic practitioners can lead to better questions and better answers (Eschenfelder, 2011; Markides, 2007; McGahan, 2007; Simpson & Seibold, 2008). Whereas scholars have noted a number of ways in which academic studies can be translated into practice, engaged scholarship goes beyond this type of applied research. Applied research might best be described as a “knowledge-transfer problem” (Van de Ven & Johnson, 2006), where academic scholars work to translate their research into nonacademic language and present their findings in forums that might be visible to practitioners. On the other hand, engaged scholarship involves collaborative inquiry, where academic and nonacademic constituencies work together to examine big questions that are important to both. As such, engaged scholarship is synonymous with scholarship at the interface of researcher and practitioner (Saunders, 2011). Waldron (2007) summarized applied research as a scholar studying an academically interesting issue and then presenting results and applications to practitioners. He described engaged scholarship in terms of trust and interwoven goals between academic and nonacademic individuals.

This type of partnership is difficult to pursue if the researcher is limited to a single method. Antonacopoulou (2011) argued that debate about academic pursuits and nonacademic praxis have “highlighted the need for management research to be more applicable to management practice and . . . highlighted the need for rigorous investigation supported by sophisticated methods for data collection and analysis” (p. 314). The problems that concern nonacademic audiences may not fit neatly into that particular method. Indeed, issues experienced by practitioners may be complex to the degree that they cannot be addressed by any single method. Mixed methods research provides the flexibility necessary to respond to business leaders’ needs as they describe issues that are important in their organizations and the ability to comprehensively study those issues from multiple perspectives.

Mixed Methods Research in Management Literature

A number of scholars have written about the advantages of mixed methods research in general (i.e., Creswell & Plano Clark, 2011; Johnson & Onwuegbuzie, 2004). Molina-Azorin (2011) focused specifically on organizational research and found that mixed methods studies had more value-added to management scholarship than did single method studies as measured by article citations. Bryman and Bell (2007) provided one potential reason for that value—although quantitative methods are appropriate for comparing variables, qualitative methods are better suited for examining organizational processes. Similarly, Myers (2010) argued that mixed methods research was important in organizational communication scholarship because one method could be designed to overcome the limitations of another method used to address the same research question. That is particularly important when examining a sensitive issue or problem (Jehn & Jonsen, 2010). Jehn and Jonsen (2010) argued that self-report methods by themselves were incapable of presenting a complete picture of some research foci because individuals would be concerned with portraying themselves favorably. Attribution theory suggests that other-report methods would also be subject to bias because others might overemphasize personal attributions and underemphasize contextual constraints (Ross, 1977). Jehn and Jonsen contended that mixed methods research is able to overcome these limitations by using each perspective to counter the bias in the other perspective. The focus of the present study was organizational dissent, and the sensitive nature of dissent emphasized the importance of mixed methods research for this study.

Organizational Dissent

Organizational dissent describes interactions where one or more employees explicitly disagree with current practices or policies (Garner, 2012; Kassing, 2011). Kassing (2011) related dissent to employee voice but explained that the two were not always synonymous (dissent would not include complaints about customers, for example). Previous scholars have demonstrated that dissent could be focused on the good of the organization (Graham’s, 1986, “principled dissent”) or focused on gaining a personal advantage (Hegstrom, 1999). The defining element of dissent is that the dissenter is challenging the status quo in a way that what he or she perceives is counter to managerial expectations.

That might seem to indicate that managers should do everything possible to silence dissent as decisions are being made, but nothing could be further from the truth. The absence of dissent in decision making is the essence of groupthink, where groups value harmony and cohesiveness at the expense of effective decision making (Janis, 1982). Dissent and debate enable more creative problem solving (De Dreu & West, 2001; Edmondson & Munchus, 2007; Mitchell et al., 2009), even when the dissenter is incorrect (Schulz-Hardt, Brodbeck, Mojzisch, Kerschreiter, & Frey, 2006). Dissent is necessary in decision making, because those people who make the decisions are often not the implementers of the decisions (Landier, Sraer, & Thesmar, 2009). Landier et al. explained that the separation between decision making and implementation means that implementers may have information necessary to effective decisions and that information is expressed as organizational dissent. Dissent also be a corrective to unethical decisions (Teo & Caspersz, 2011). The ability to express dissent has also been linked to increased job satisfaction (Lutgen-Sandvik, Riforgiate, & Fletcher, 2011), increased perceptions of procedural justice (Korsgaard & Roberson, 1995), decreased burnout (Avtgis, Thomas-Maddox, Taylor, & Patterson, 2007), and decreased turnover (Spencer, 1986).

One of the more important directions in this stream of literature is the nature of the dissent conversation, where an employee expresses a contrary opinion and another person responds to that expression. The benefits of dissent for organizations and for employees are directly related to the outcomes of that dissent conversation. Dissent that is rejected without consideration by a manager is unlikely to help that manager make effective decisions. Conversations in which the dissenter is ignored and/or punished by a manager are unlikely to lead to perceptions of procedural justice or to increased job satisfaction. Additionally, employees base their decisions of whether to express dissent on the outcomes of past dissent conversations (Milliken, Morrison, & Hewlin, 2003), which means that dissent conversations in which the employee is rejected will likely lead others to withhold their ideas, resulting in less dissent and more myopic decision making.

Previous research has examined conversational outcomes and has emphasized the importance of what the dissenter says. Author (2012) found that dissenters reported solution presentation, circumvention, and repetition as more likely to be effective than other types of messages. Kassing (2007) found that dissenters reported circumvention to be a strategy that was likely to lead to instrumental change even as it often led to relational deterioration. Kassing (2005) examined others’ perceptions of dissent and found that solution presentation and direct-factual appeals were perceived as competent communication.

Two things are missing from this research. First, many of these studies have relied on dissenters’ self-reports to examine dissent. A number of scholars have critiqued self-report data more broadly, but Jehn and Jonsen (2010) noted that self-report data are particularly suspect when examining sensitive organizational issues. Thus, studies examining the effectiveness of dissent but relying on the self-reports of dissenters may be missing key components of the larger dissent process. Second, although several studies have used other-reports to understand employee dissent (i.e., Garner, in press; Kassing, 2005), these studies seem to focus solely on others’ perceptions while neglecting how dissenters’ see the conversation. In essence, the pendulum swings too far away from self-report. One type of research focuses on the dissenter while neglecting the context around him/her, and the other type focuses on aspects of context while neglecting the dissenter. Though there are a number of ways to examine multiple perspectives, mixed methods research is ideally suited to address these deficiencies, because each method can be designed to compensate for any blind spots in other methods. This combination approach provides a more complete view of the outcomes from dissent conversations.

The study began as a deductive study examining organizational dissent. The research questions originally guiding the study included the following:

Research Question 1: How is dissent expressed in these organizations?

Research Question 2: What is the relationship between what is said and the outcome of dissent conversations?

However, these research questions were ultimately sharpened through conversations with staff and management at each organization. Although Tushman and O’Reilly (2007) discussed educational concerns, their admonition is relevant for the present study:

Our doctoral programs should pivot on the choice of research question—and then train our students in the use of appropriate, discipline-based theoretical and methodological tools. Whereas dissertations are now often driven by disciplinary concerns, we believe that the source of research questions should be the phenomena, not the traditions or constraints of a given disciplinary point of view. (p. 772)

Thus, the research questions hinged on my engagement with the organization. Consequently, as my partnership with each participating organization deepened, the approach shifted to more of an inductive stance. Along those lines, Pierce (1903) noted that new insights often come from a combination of deduction and induction, what he called “abduction” (as cited in Suddaby, 2006). Similarly, mixed methods research scholars have noted that research questions could be emergent with mixed methods research (Christ, 2007; Plano Clark & Badiee, 2010), and this is addressed in the Discussion section. The research questions were subsumed under a more comprehensive question about the nature of dissent in these organizations. Thus, the final results address how dissent was expressed and the factors related to conversational outcomes, but the results are broader than would have been achieved with only the initial study.

The following paragraphs describe this project sequentially with the methods and results of one phase presented, followed by the methods and results of the next phase. Phase 1 consisted of a quantitative survey that included two qualitative questions. Those questions directed my attention during the second phase, which consisted of observations of management and staff meetings. Finally, a series of interviews were used to aid in interpreting those observations. Figure 1 provides a visualization of this sequence. Using notation similar to Morse’s (2010), these phases are best presented as QUAN + qual − QUAL − QUAL.

Figure 1.

Data collection timeline.

Participating Organizations

Two organizations participated in this study. I was already engaged with one organization as a volunteer when I began to plan a project with one of the executive officers to examine organizational dissent among the staff. Riverside Services is a faith-based nonprofit organization with approximately 30 employees. Riverside provides assistance for people in need, including food, clothing, job training, and medical services. Following the engaged scholarship model, one reason that Riverside was an appropriate sample for this project was my partnership with the organization. Another reason that this organization was appealing for this study was its smaller size. Although I knew that might limit the power of quantitative analyses, the smaller size enabled me to better integrate observations and interviews with survey results. Additionally, the value of using a nonprofit organization as the sample became evident when the role of mission surfaced as an important element of dissent.

I met with the executive director of Downtown Health Assistance as part of another project and explained that I study the way that managers and employees communicate in organizations. Downtown is a clinic, pharmacy, and education center for low-income patients with 37 employees, and the executive director said that he would be interested in learning more about how the staff could better communicate. This organization provided an ideal opportunity because it was similar to Riverside, but the differences in mission (faith-based vs. secular, different clientele) made it possible to tease out how mission affected organizational dissent, which emerged as important as the project progressed. Data were collected over the course of 10 months.

Phase 1

Survey Administration

For both organizations, a survey was emailed to employees via Qualtrics. Employees were given 2 weeks to complete the survey, at which time a reminder was sent. The survey was closed 2 weeks after the reminder. A total of 40 (19 from Riverside, 21 from Downtown) employees participated in the survey (59.70% response rate). Participants were between 24 and 67 years old (M = 44.00, SD = 14.96) and had been with their organization between 7 months and 20 years (M = 4.43, SD = 4.51). Participants included 24 women and 8 employees of non-European descent.

The survey instructions directed participants to recall a conversation where they expressed dissent regarding a policy or practice at work and to focus on this conversation as they responded to survey items. Garner’s (2009) Dissent Messages Scale measured what participants said in those conversations. This scale asked “to what degree did what you said resemble these statements” and then presented participants with 32 items that described 11 possible dissent messages. For example, the statement “suggest a way to fix this situation” was one of four items that measured solution presentation. The statement “use some type of humor or jokes to express dissatisfaction” was one of three items that measured humor. Other types of dissent messages included venting, direct-factual appeals, inspirational appeals, circumvention, coalition-building, repetition, ingratiation, exchange, and pressure tactics. Participants responded on 5-point Likert-type scales that were anchored by “not at all like what I said” and “extremely similar to what I said.” Canary and Spitzberg’s (1987) Conversational Effectiveness and Appropriateness Scales measured the degree to which participants perceived that they were effective and/or appropriate in this dissent conversation. Sample items included “I achieved what I wanted to achieve in the conversation” and “My conversation was very suitable to the situation.” Table 1 includes the means, standard deviations, reliabilities, and correlations of all variables in the study. Factor analysis loadings were consistent with prior research and are available from the author.

Table 1.

Means, Standard Deviations, Reliabilities, and Correlations of All Variables.

	Mean	Standard Error	Standard Deviation	α	1	2	3	4	5	6	7	8	9	10	11	12
1. Conversational Effectiveness	3.26	0.16	0.94	0.92	—
2. Conversational Appropriateness	4.04	0.08	0.46	0.70	.508***	—
3. Venting	3.66_a	0.14	0.83	0.78	0.073	0.140	—
4. Direct-Factual Appeal	3.64_a	0.14	0.80	0.67	−0.002	0.134	0.538***	—
5. Solution Presentation	3.49_a,b	0.14	0.80	0.81	0.036	0.133	−0.150	0.151	—
6. Inspiration	2.93_b,c	0.19	1.09	0.86	−0.064	−0.206	0.296*	0.318*	−0.168	—
7. Circumvention	2.23_c,d	0.19	1.11	0.79	−0.169	−0.461***	0.090	0.406**	−0.030	0.457***	—
8. Coalitions	2.13_d,e	0.18	1.05	0.86	0.082	−0.068	−0.108	0.291*	0.186	0.315*	0.335*	—
9. Repetition	1.76d_e,f	0.15	0.90	0.78	0.222	−0.223	−0.100	−0.109	−0.174	0.358**	0.301*	0.256	—
10. Ingratiation	1.76d_e,f	0.17	0.98	0.82	−0.100	−0.366**	0.020	0.217	−0.266	0.258	0.485***	0.375**	0.383**	—
11. Exchange	1.53_e,f	0.13	0.73	0.72	−0.141	−0.539***	0.004	0.156	−0.085	0.269	0.549***	0.412**	0.424**	0.747***	—
12. Humor	1.53_f	0.12	0.69	0.69	−0.122	−0.406**	−0.223	−0.022	−0.104	0.138	0.364**	0.207	0.573***	0.600***	0.604***	—
13. Pressure	1.31_f	0.09	0.54	0.83	−0.151	−0.305*	−0.171	0.003	0.133	−0.059	0.074	0.173	0.311*	0.314*	0.356**	0.252

Note. Different subscripts indicate significant differences between means.

***

p < .01. **p < .05. *p < .10.

Two open-ended questions were also included in the survey. The survey began with the question, “What’s something about communication where you work that is really working?” The survey ended with the question “If you could change one thing about communication in this organization, what would that be?” Some participants skipped these open-ended questions, but the available responses (n = 29; 17 from Riverside, 12 from Downtown) were analyzed using a constant comparative method (Glaser & Strauss, 1967). Each statement was compared with previous statements. When a statement was similar to a previous response, those responses were grouped together. When a statement was not similar to any previous responses, it was left separate. Responses were examined in this way to define categories that best captured these responses. After an initial pass through the data in this way, I revisited any responses that did not fit into categories to determine how these responses might relate to the categories previously identified. In some cases, a category could be defined more broadly to include a response. In other cases, responses were isolated and were dropped from further analysis.

Results

The first step in data analysis was to identify differences between organizations, and a multivariate analysis of variance was used for this test (Wilks’s Λ = .713, p = .92). All of the individual ANOVAs were nonsignificant, indicating that there were no significant differences between organizations on any variable. Therefore, the remaining analyses aggregated both samples, and the results of those analyses are presented in Table 1. As previously mentioned, Garner (2009) examined the tactics used to express dissent (i.e., presenting a solution, pressure, and humor) and identified 11 dissent message types. To better understand which of these types were used more frequently, I calculated confidence intervals of two standard errors around each mean. Nonoverlapping confidence intervals indicated significant differences in the use of those types of dissent messages. Finally, correlations were used to test for significant associations between dissent messages and conversational effectiveness and appropriateness. Significant correlations between messages and conversational outcomes could indicate that participants experienced that outcome when using that message. For example, the negative correlation between circumvention and conversational appropriateness indicates a negative relationship between circumvention as a dissent strategy and the appropriateness of the dissent conversation.

With such a small sample size, I expected that many relationships between variables would be nonsignificant because of the inability to detect small and medium effect sizes. Nevertheless, it is interesting to note several features in the data. The most common ways of expressing dissent were venting, direct-factual appeals, and presenting solutions to problems. That these messages were not significantly related to conversational effectiveness indicates that either the influence of messages on effectiveness in these organizations is too small to be detected with this sample size or that there is no relationship between these types of dissent messages and conversational effectiveness. In either case, these results indicate that something else plays a substantial role in the effectiveness of participants’ dissent conversations. Part of that “something else” might be the appropriateness of what participants said. The results of later phases point to another factor.

Inspiration messages of dissent were noteworthy in this context because of the importance of mission in nonprofit organizations. According to Garner (2009), inspiration included appeals to “what’s right.” It seemed reasonable to assume that these messages would be related to conversational effectiveness and appropriateness since that might indicate that aligning one’s dissent with the organizational mission might increase the chances of a successful conversation. That was not the case. In fact, inspirational appeals were negatively related to conversational effectiveness and appropriateness, though not in statistically significant ways. On the other hand, it is worth pointing out that inspiration was significantly and positively related to circumvention and repetition. These are both second-order dissent strategies in that they subsume other messages. Whereas “inspiration” describes the content of the dissent message, circumvention and repetition describe the audience and the number of times that dissent is expressed, respectively. Thus, a significant correlation might indicate that participants expressed dissent using inspirational appeals when circumventing their supervisor and when they were repeating their dissent.

From the open-ended questions, several themes developed. Participants from both organizations talked about mission and the value of clients, both in terms of what the organization did well and what could be improved. One participant at Downtown commented that “one positive thing is that we all know of the great need that our clients have and that our basic effort is to help the clients.” At the same time, another participant said, “It often feels like we are more concerned with how the building ‘looks’ than we are with making our clients feel comfortable and welcomed.” An employee at Riverside (whose mission is faith-based) noted that the mission affected the way that leadership and staff interacted: “Because we do have godly leadership that depends on God to guide and direct situations, I feel that we don’t experience the strife that maybe other organizations contend with.”

Responses from both organizations focused on how the organization used meetings, as well. Participants from Riverside were generally positive regarding meetings at their organization. One participant noted, “[We] generally don’t just meet for meetings sake, mostly productive,” whereas another stated, “I think the regularly scheduled directors meeting and monthly staff meetings help to bring everyone together.” In contrast, someone from Downtown commented that “most director meetings are ‘tell me what is going on in your department’ type of meetings . . . [these meetings] are not conducive to our work or what we believe needs to be changed.”

A final theme focused on how upper management related to middle management and other employees. One employee at Downtown complained, “At the moment, I believe that communication is severely lacking from the top down.” Another said, “There is a disconnect between upper and middle management regarding communication. [Middle management personnel] are not listened to or generally dismissed.” At Riverside, a middle manager stated that “my direct supervisor communicates very little with his management staff . . . [middle managers] are treated like children.”

Even at this point in the project, the use of mixed methods provided a better picture of organizational dissent than either the quantitative or qualitative results alone could offer. That no dissent messages were related to conversational effectiveness and that several were inversely related to conversational appropriateness in the quantitative results is somewhat in line with the qualitative finding of employees’ dissatisfaction with communication. Employees perceive that most forms of dissent are inappropriate in their organization, which may be part of the issue that they report in the open-ended responses. The correlation between inspirational messages and circumvention and repetition alludes to the organization’s mission playing a role in dissent, but the nature of that role began to be clearer in the qualitative results. A second phase was conducted to further understand these concepts.

My partnership with Riverside included observing meetings, and those observations began even before I presented the survey results to my liaison in that organization. Such an arrangement was not originally part of my agreement with Downtown. However, when the executive director at Downtown saw the survey results, particularly the summary of open-ended responses, he invited me to a series of their regularly scheduled meetings so that I could better understand communication in that organization. My observations of both organizations formed the basis of Phase 2 in this research.

Phase 2

Meeting Observations

I observed meetings at both Riverside and Downtown. Riverside observations were partially concurrent with survey distribution, whereas Downtown observations were subsequent to the surveys. However, the relationship between these two methods was more complicated than simply concurrent or sequential. Because I was primarily an observer in the meetings, those data were collected independently from the survey data. However, meeting data were analyzed after survey data, so that my interpretations of observations were made in light of the survey results.

Riverside Services had management-level meetings every 2 weeks. I observed four such meetings. At Downtown Health Assistance, I observed one executive board committee meeting (which meets once a month), one staff meeting (which is convened monthly), and one “quality management” meeting (which included management level staff minus the executive director and was supposed to be monthly). At the first meeting at Riverside and at each meeting at Downtown, I was introduced as a researcher who was helping the organization with its communication. After obtaining informed consent, I took detailed notes during the meetings and recorded field notes and reflections immediately following each meeting. I then revisited these data to identify examples of dissent, which were examined in detail for the present analysis. I also observed four staff meetings at Riverside, but those meetings were primarily announcements and reports. Therefore, those observations helped inform my understanding of the organization but were not directly analyzed as part of this project. All told, I observed 13½ hours of meetings as part of this project.

The units of analysis for the present study were examples of dissent in these meetings. Those units were analyzed in a modified constant comparative analysis (Glaser & Strauss, 1967). Although I was no doubt influenced by the quantitative results, I tried to approach these observations separately from those analyses to allow ideas about dissent in these organizations to emerge inductively. I read and reread each example of dissent and reflected on the context of the meeting in which the dissent occurred until I felt that I understood what the dissenter was saying and how people were responding to him or her. At that point, I tried to capture the conversation in a description that included what was said and others’ responses. I then compared each instance of dissent with other examples and looked for similarities to use as I condensed the data into cogent themes. The next section presents those themes.

Results

“That’s All I Have to Say”

Some of the more interesting occurrences in meetings were reactions to my presence. On two occasions at Downtown Health when I was present but the executive director was not, there was a reference to the director’s lack of openness to employee dissent as meeting participants felt the need to explain communication. During the executive board meeting, I was asked to explain the quantitative portion of the study. As I did so, one director responded, “All I know is there’s a huge variance of communication style among executive directors in nonprofit organizations. Huge. Period. That’s all I have to say.” Several people chuckled in acknowledgement after his comment. The tone seemed to imply that the director was not a good communicator. Several weeks later, I attended a meeting of the senior staff minus the executive director. This regularly scheduled meeting was run by the associate executive director with the charge of increasing the quality of services provided by the organization. It seemed as if it intentionally excluded the executive director so that attendees could be candid with ways they saw to improve quality. As the meeting began, the associate executive director turned to me and said, “I have to give you some background. We were not allowed to have this meeting for a while. Any meeting.” The implication of this theme was that participants wanted me to know that the executive director’s communication was poor and that he was closed to employees’ feedback.

Director’s Blind Spot

In another meeting, I directly witnessed this lack of openness. The meeting began as the executive director, reacting to my report of the quantitative results, described a new initiative to invite employee feedback at meetings. He stated that meetings would be more “inclusive” rather than just upper management giving information to employees. One of the major items of business in this meeting was a presentation of remodeling plans for the organization’s building. As the executive director was going through these plans and what the changes would mean for staff and for clients, one employee pointed out that the present waiting room made it difficult to protect clients’ confidentiality and wondered if the remodel would address that. The executive director commented that this issue was not included in the remodel. Other employees looked at the remodel plans and suggested alternative areas so that clients who were there for sensitive treatment could be kept separate from other clients, thus protecting confidentiality. The executive director responded with, “Those areas are not waiting areas. I know what you’re saying . . . but those are not waiting areas.” When the discussion continued, the executive director grew anxious and kept glancing in my direction (perhaps because I was there and he was trying to be “open”). He attempted to end comments by saying, “Certainly your ideas and your comments are well taken. If you send me an email about it, I will certainly address them the best that I can.” The person who originally expressed dissent summarized her dissatisfaction by saying,

This goes back to what I was trying to say earlier. Maybe whenever we were having this discussion how the building was going to be moved around or how it’s going to affect the clients, maybe it would have been good to have staff that’s actually working with the clients to be involved and give input. . . . It’s not so much the building. It’s that you don’t know our clients feel that way.

There was silence after this statement. Then the executive director continued explaining the remodel plans as if nothing had been said.

This example was telling in part because there was no consideration of including those staff members who had direct contact with clients in the decision making. The conversation also demonstrated that, although the executive director claimed to want openness, he did not know how to respond when employees were allowed to challenge his ideas.

Mission

The third theme in these meetings focused on the role of mission in dissent and in responses to dissent. In the examples from Downtown of the executive director’s blind spot, employees used serving clients as a rationale for their positions. The employee that questioned the remodel plans did so because she saw a client need (the desire for confidentiality) that was not being met.

At Riverside, the mission served a different purpose. During a discussion of the organization’s strategic plan, the associate executive director was inviting feedback from middle managers regarding the organization’s vision. The executive director, who was also in attendance, took the opportunity to emphasize his perspective, based on his interpretation of the mission statement (which he had developed):

Let me just tell you that, on that mission statement, the Lord has laid on my heart in one of my morning prayer times to break [the mission statement] down for our board Thursday night into three categories, and explain it in detail for a scriptural perspective. Just the way you’ve got it broken down [here] . . . and it really clarifies for me how we’re really on target with our mission statement.

The effect of this monologue was to silence any creative ideas about vision. In effect, the mission reference was a preemptive shot at any employee who might challenge the executive director’s vision.

The results of Phase 2 further clarified the quantitative and qualitative findings from Phase 1. Even before the final phase, these results indicate how mixed methods research is a useful approach for engaging with nonacademic practitioners. The leaders of both organizations wanted insights that would help them improve communication. The quantitative results provided some ideas, but the qualitative steps unpacked those ideas by surrounding the survey results with additional context. It is also worth noting that the executive director at Downtown specifically requested mixed methods research by asking me to observe meetings after receiving survey results. Even if only intuitively, he recognized the value in examining the problem using multiple lenses. After those observations, I wanted to better understand more specifically how employees made sense of mission and openness in ways that anonymous responses to surveys and passively observing meetings did not allow. The executive director at Riverside, although not asking for further results, did not hesitate to allow me additional access to employees when I suggested that I would like to further explore communication and mission. The next section describes this final phase of research.

Phase 3

Interviews

As a final stage of data collection, I conducted interviews with a number of staff members at Riverside Services. Interview participants were selected so as to sample from each level of the organization, to get a variety of departments represented, and based on recommendations from previous interview participants. I spoke with 11 staff members which were recorded (out of a total of 30 employees). Eight of the interview participants were women and seven were Caucasian (both consistent with the demographic makeup of the staff). Participants included an executive officer, four participants who were upper middle management, two participants who were lower middle management, and four participants who were nonmanagement. Later interviewees provided similar responses to earlier ones, indicating that I was reaching theoretical saturation. As I analyzed responses, data seemed to fit into “whole” categories, furthering my confidence that I had sufficient data.

I offered to meet each person in a conference room at the organization or at a McDonald’s less than a mile away (two participants asked to meet at this McDonald’s). In addition to asking questions about strengths and weaknesses of communication at Riverside, I also asked how open people were to hearing new ideas and examples of times when employees had expressed dissent. Because these interviews were conducted after the survey and observations, I also asked participants if I had missed anything while collecting data regarding communication through surveys and observations.

The interviews were recorded and transcribed (66 single-spaced pages). These interviews were analyzed in a manner similar to the observations of Phase 2. I identified descriptions of dissent conversations in the interview transcripts and summarized what was said and how others responded. I then considered each datum and compared it with examples in other interviews. Doing so was an iterative process, and it resulted in two themes that emerged from participants’ responses. Those themes are presented in the next section.

Results

Bounded Openness

One of the dominant themes in the interviews was that supervisors were generally open to new ideas. However, there were at least two caveats attached to that openness. First, supervisors expected employees’ ideas to be solution-oriented rather than problem-oriented. One person stated that “people are open to listening to the idea . . . if you’re going to suggest that we do something, you need to back it up, you have to be willing to present a solution to it.” Second, the solution needed to be cost-effective. When I asked about how supervisors responded to employees’ dissent, an employee responded by saying,

They’d think about it, pray about it. Research it. And say, “If you think this is really going to work and if it doesn’t cost a lot of extra money”—the money thing is always an issue—“then try it.”

When I asked for examples of this openness, several employees described a recent change where one of the organization’s core functions was moved from one department to another department and finally to a third department, each time in response to employees’ dissent. I interviewed the employee who first initiated this change, and he emphasized his desire to help clients get what they needed with dignity. Just like the other interviews, his dissent, as well as the dissent that moved the function to the third department, were solution-oriented and did not require money.

Mission Determines Response

In that last example, the employee who first expressed dissent was focused on helping Riverside’s clients. Several interview participants described this conversation, and all of them explained that upper management approved the change because it seemed to be in the best interest of helping clients. Just as people were generally open to new ideas, the mission of the organization seemed to be a uniting factor. One person best captured this idea saying:

I think the way we share the same mission and what we do here and why we do it. I found that sometimes I would be at odds with another department but then when I understand where they’re coming from and what their goal is in that department, it kind of softens my heart why they’re so passionate in getting something accomplished.

A middle manager described how this organization’s faith-based mission affected day-to-day interactions,

We allow our faith to kind of dictate how we interact and work with one another and handle situations. We’re not perfect at it, and it doesn’t always work out the way that it should, but our whole goal is to . . . make [the mission] be the focus and the end all to what we do.

When I asked this middle manager for an example of that, she described a conflict and explained, “Had we not thought about [the mission], it could have been worse.”

Results Integration

No mixed methods study is complete without an integration of what was found through each method, because the point of using mixed methods in a single project is to find something that could not be found using single methods in isolation. This section, then, integrates the results from each phase of the study.

Observing the meetings at each organization, the differences between the climates was striking, which makes it surprising that there were not statistically significant differences in terms of conversational effectiveness or other variables. Given the descriptions in interviews of Riverside’s openness to dissent and a climate that seemed less hostile between upper management, I would have expected participants from Riverside to report higher levels of conversational effectiveness. However, I did not observe any dissent at Riverside in the meetings that I attended. Thus, the quantitative results may suggest that, whereas the climate at each organization was different, participants did not feel effective in either context.

The role of mission in the first phase of the study was conspicuous (specifically in the open-ended responses), and thus, mission was a significant part of the subsequent phases. Mission shaped both the expression of dissent as well as responses to those conversations. Mission prompted employees to dissent as they saw areas in which clients’ interests were not being addressed. In the Downtown staff meeting that I observed, an employee challenged the ways in which their building was being remodeled because the executive director had not taken staff concerns about clients into consideration. The executive director seemed unable to respond to that dissent and instead ignored it, perhaps because using mission as a basis for dissent would have equated rejecting the dissent with rejecting the mission. At Riverside, interview participants identified mission as a primary guide in how they interacted with each other, including how they expressed dissent. In light of the importance of mission, it is interesting that employees in both organizations did not report inspirational appeals (which can be related to mission) as being conversationally effective. On the one hand, that is surprising given that participants from both organizations seemed to use mission as a basis for dissenting. On the other hand, the employee in the Downtown meeting was not successful in getting the plans revised. The executive director at Downtown balked as an employee used mission to challenge his plans, and the executive director at Riverside used a mission reference to preempt dissent. It could be that, in these ways, mission is seen as a way for managers to justify their decisions rather than as a basis for employees to share input in those decisions.

Discussion

Implications for Mixed Methods Research

The most important contribution of this article is an exemplar of mixed methods research in engaged scholarship. A number of scholars have advocated involving practitioners in the development of research aims (Barge & Shockley-Zalabak, 2008; Markides, 2007; McGahan, 2007; Van de Ven & Johnson, 2006), and managers in both organizations shaped the research focus at the beginning of the study and then suggested alternative avenues of examination as the study progressed. A single method approach might make sense when pursuing an issue from a particular theoretical tradition. Involving nonacademic audiences in the study design rather than as passive receivers of results required considering the phenomenon using mixed methods and resulted in a deeper study. Antonacopoulou (2011) argued that research that addressed complex problems required sophisticated methods. Certainly there is engaged scholarship that is not mixed methods research, and certainly there is mixed methods research that is not engaged scholarship. Nevertheless, the present study demonstrates the value added by using mixed methods research to address the questions that were important to practitioners involved in this research. Gulati (2007) advocated moving beyond methodological parochialism and simultaneously moving beyond a false theory/practice dichotomy, and the present study, by using mixed methods that fit research questions with theoretical and practical value, demonstrates one way to advance knowledge in both ways. Similarly, Tushman and O’Reilly (2007) emphasized the role of research questions guiding methodological concerns, and the research presented here demonstrates why that can be important. Although theoretically important questions could have revealed some knowledge, the interplay between theory and praxis in combination with methods appropriate to the emerging research questions provided a better understanding of the phenomenon under study and better insights into how the organizations might improve communication than would have been revealed with a single method.

These results demonstrate the importance of emergent research questions in mixed methods research. Although Yin (2006) critiqued emergent research questions and argued that a study’s focus should always be determined a priori, Plano Clark and Badiee (2010) acknowledged that the ability to approach a study with either predetermined or emergent research questions was a strength of mixed methods research. The present study illustrates how that might happen. In the end, the results of Phases 2 and 3 spoke to the original research questions, but the answers that later phases provided went beyond the ways in which I had approached each question in Phase 1. These extensions provided conclusions that were more meaningful academically and practically than would have been seen if the research questions had been rigidly maintained.

These results also demonstrate the value of sequential data collection. Castro, Kellison, Boyd, and Kopak (2010) critiqued sequential studies, saying that the results were often poorly integrated, but the present study is an example of the value of following up on early results with additional research. The qualitative portion of the survey indicated that missions and meetings were important elements in these organizations, which led me to focus on how dissent was tied to the organization’s mission as expressed in meetings. This attention was not part of the original direction of the study, but it revealed important insights. Christ (2007) stated that longitudinal studies with sequential designs could uncover insights that concurrent studies might miss, and these results confirm that argument. The first phase demonstrated the need for future phases, and later phases provided context for earlier ones.

Organizational Dissent

Perhaps the strongest results regarding organizational dissent from the present study focused on the connection between mission and the role of dissent in organizational decisions. Previous research has found that mission attachment is a factor, albeit a weak one, in employee retention (Brown & Yoshioka, 2003). The results presented here go beyond that to suggest that mission is deeply involved in interactions within the organization itself, not just in whether an employee chooses to stay or leave. The organization’s mission was embedded in the ways in which employees expressed dissent with their supervisors’ decisions as evidenced in observations and in interviews. In one organization, mission was emphasized as a basis for responding to dissent. In the other organization, it seemed as if the executive director was unable to respond to employees’ dissent because that dissent was based on the organization’s mission. The role of mission seems underexplored in organizational dissent literature and to a certain extent, in management literature as well, but these results suggest that mission is vital to organizational processes.

At the same time that mission was related to how employees expressed dissent and how others responded, mission may not have been strongly related to the effectiveness of those dissent conversations. None of the message types in Phase 1 were related to effectiveness, including mission-focused appeals. In the meeting in which I observed employees using mission in their dissent, their conversation seemed ineffective. One exception was the example of dissent given by several interview participants at Riverside, where an employee changed one of the core functions of that organization based on his vision of meeting clients’ needs. It seemed as if mission was a necessary but not sufficient factor in effective dissent. Employees also had to offer realistic solutions to problems. Previous research has connected solution presentation to effective dissent (Garner, 2012, in press; Kassing, 2005). Although Phase 1 did not find a significant relationship between solution presentation and effectiveness, it is possible that the interaction between mission and a solution orientation could be a determining factor in whether employees’ dissent is considered in organizational decisions, particularly with a larger sample. Further research is needed to explore that possibility.

A final insight from these data concerns the difficulty in being open to dissent. Garner (in press) found that most managers wanted to be open to employees’ ideas but could not provide examples of accepting positions that disagreed with the status quo. The executive director at Downtown provided the clearest evidence of that phenomenon, where he announced at the beginning of the meeting that he would be more open to dissent, but by the end of the meeting, he was ignoring dissent and trying to move on to the next topic. The executive director at Riverside also demonstrated this difficulty as he used the mission statement and his vision to preempt disagreement. Despite knowing the benefits of dissent and despite voicing openness, it was difficult for these leaders to accept ideas that were different from their own.

Limitations and Conclusion

Certainly, the limitations of this study should be recognized. The survey sample was small. There was not enough power to discount most of the nonsignificant results, and further research is needed to explore these ideas in the nonprofit arena. I was limited in the number of meetings that I could observe by the practical considerations of the organization. Further data might have fleshed out the themes that were found or might have added nuances that I missed with this sample. Additionally, all results came from only two organizations. As such, these results should not be generalized beyond the organizations from which they were drawn. Conceptually, the role of mission in organizational dissent was not directly examined in the first phase as this variable emerged as important only through the analysis of Phase 1 results. Future research might replicate this phase with a larger sample and might include mission-related variables to explore this connection quantitatively.

Those limitations and unanswered questions notwithstanding, these results demonstrate the potential importance of mission in expressing dissent. While having a mission-focus was not sufficient for success, it did seem to be an important dimension in dissent conversations. Additionally, these results demonstrated nuances in managers’ attempts to be open. Despite wanting to be open to employees’ dissent, the two managers in the present study showed discomfort and personal bias that interfered with employees’ ability to express disagreement with policies or practices.

More important, this project demonstrates how multiple methods used together can contribute insights that would be missed with single method studies. In the present project, the results by themselves would be important, but their real value comes when taken collectively. For example, the importance of mission was seen in Phase 1, potential connections between mission and dissent were seen in Phase 2, and nuances in those connections were fleshed out in Phase 3. At another level, this project demonstrates the necessity of complex methods when addressing problems or questions raised in partnership with nonacademic practitioners. These practitioners are often not predisposed to a methodological stance and are instead focused on problems to be solved or questions to be answered. Mixed methods research provides a robust way in which to address those problems and questions.

Footnotes

Acknowledgements

The author would like to thank his liaisons and the employees at each of the organizations that helped shape this project.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a Research and Creative Activity Fund grant from Texas Christian University.

References

Antonacopoulou

E. P.

(2011). Advancing practice relevant scholarship. In Cassell

Lee

(Eds.), Challenges and controversies in management research (pp. 314-331). New York, NY: Routledge.

Avtgis

T. A.

Thomas-Maddox

Taylor

Patterson

B. R.

(2007). The influence of employee burnout syndrome on the expression of organizational dissent. Communication Research Reports, 24, 97-102.

Barge

J. K.

Shockley-Zalabak

(2008). Engaged scholarship and the creation of useful organizational knowledge. Journal of Applied Communication Research, 36, 251-265.

Biesta

(2010). Pragmatism and the philosophical foundations of mixed methods research. In Tashakkori

Teddlie

(Eds.), Sage handbook of mixed methods in social and behavior research (2nd ed., pp. 95-118). Thousand Oaks, CA: Sage.

Brown

W. A.

Yoshioka

C. F.

(2003). Mission attachment and satisfaction as factors in employee retention. Nonprofit Management & Leadership, 14, 5-18.

Bryman

Bell

(2007). Business research methods (2nd ed.). Oxford, UK: Oxford University Press.

Canary

D. J.

Spitzberg

B. H.

(1987). Appropriateness and effectiveness perceptions of conflict strategies. Human Communication Research, 14, 93-118.

Castro

F. G.

Kellison

J. G.

Boyd

S. J.

Kopak

(2010). A methodology for conducting integrative mixed methods research and data analyses. Journal of Mixed Methods Research, 4(4), 342-360.

Christ

T. W.

(2007). A recursive approach to mixed methods research in a longitudinal study of postsecondary education disability support services. Journal of Mixed Methods Research, 1(3), 226-241.

10.

Creswell

J. W.

Plano Clark

V. L.

(2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.

11.

De Dreu

C. K. W.

West

M. A.

(2001). Minority dissent and team innovation: The importance of participation in decision making. Journal of Applied Psychology, 86, 1191-1201.

12.

Edmondson

V. C.

Munchus

(2007). Managing the unwanted truth: A framework for dissent strategy. Journal of Organizational Change Management, 20, 747-760.

13.

Eschenfelder

(2011). Lessons about engaged communication scholarship: I heard it through the grapevine. Florida Communication Journal, 39, 37-54.

14.

Feilzer

M. Y.

(2010). Doing mixed methods research pragmatically: Implications for the rediscovery of pragmatism as a research paradigm. Journal of Mixed Methods Research, 4(1), 6-16.

15.

Garner

J. T.

(2009). When things go wrong at work: An exploration of organizational dissent messages. Communication Studies, 60, 197-218.

16.

Garner

J. T.

(2012). Making waves at work: Perceived effectiveness and appropriateness of organizational dissent messages. Management Communication Quarterly, 26, 224-241.

17.

Garner

J. T.

(in press). Open Doors and Iron Cages: Management Responses to Employee Dissent. Journal of Business Communication.

18.

Glaser

B. G.

Strauss

A. L.

(1967). The discovery of grounded theory. New York, NY: Aldine de Gruyter.

19.

Graham

J. W.

(1986). Principled organizational dissent: A theoretical essay. In Staw

B. M.

Cummings

L. L.

(Eds.), Research in organizational behavior (Vol. 8, pp. 1-52). Greenwich, CT: JAI.

20.

Gulati

(2007). Tent poles, tribalism, and boundary spanning: The rigor-relevance debate in management research. Academy of Management Journal, 50, 775-782.

21.

Hegstrom

T. G.

(1999). Reasons for “rocking the boat”: Principles and personal problems. In Geissner

H. K.

Herbig

A. F.

Wessela

(Eds.), Business communication in Europe (pp. 179-194). Tostedt, Germany: Attikon Verlag.

22.

Hesse-Biber

Johnson

R. B.

(2013). Coming at things differently: Future directions of possible engagement with mixed methods research. Journal of Mixed Methods Research, 7(2), 103-109.

23.

Janis

I. L.

(1982). Groupthink (2nd ed.). Boston, MA: Houghton Mifflin.

24.

Jehn

K. A.

Jonsen

(2010). A multimethod approach to the study of sensitive organizational issues. Journal of Mixed Methods Research, 4(4), 313-341.

25.

Johnson

R. B.

Onwuegbuzie

A. J.

(2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33, 14-26.

26.

Kassing

J. W.

(2005). Speaking up competently: A comparison of perceived competence in upward dissent strategies. Communication Research Reports, 22, 227-234.

27.

Kassing

J. W.

(2007). Going around the boss: Exploring the consequences of circumvention. Management Communication Quarterly, 21, 55-74.

28.

Kassing

J. W.

(2011). Dissent in organizations. Cambridge, England: Polity.

29.

Korsgaard

M. A.

Roberson

(1995). Procedural justice in performance evaluation: The role of instrumental and non-instrumental voice in performance appraisal discussions. Journal of Management, 21, 657-669.

30.

Landier

Sraer

Thesmar

(2009). Optimal dissent in organizations. Review of Economic Studies, 76, 761-794.

31.

Lutgen-Sandvik

Riforgiate

Fletcher

(2011). Work as a source of positive emotional experiences and the discourses informing positive assessment. Western Journal of Communication, 75, 2-27.

32.

Markides

(2007). In search of ambidextrous professors. Academy of Management Journal, 50, 762-768.

33.

McGahan

A. M.

(2007). Academic research that matters to managers: On zebras, dogs, lemmings, hammers, and turnips. Academy of Management Journal, 50, 748-753.

34.

Milliken

F. J.

Morrison

E. W.

Hewlin

P. F.

(2003). An exploratory study of employee silence: Issues that employees don’t communicate upward and why. Journal of Management Studies, 40, 1453-1476.

35.

Mitchell

Nicholas

Boyle

(2009). The role of openness to cognitive diversity and group processes in knowledge creation. Small Group Research, 40, 535-554.

36.

Molina-Azorin

J. F.

(2011). The use and added value of mixed methods in management research. Journal of Mixed Methods Research, 5(1), 7-24.

37.

Morse

J. M.

(2010). Procedures and practice of mixed method design: Maintaining control, rigor, and complexity. In Tashakkori

Teddlie

(Eds.), Sage handbook of mixed methods in social and behavior research (2nd ed., pp. 339-352). Thousand Oaks, CA: Sage.

38.

Myers

(2010). Mixed methods. In Miller

V. D.

Poole

M. S.

Seibold

D. R.

, and Associates, Advancing research in organizational communication through quantitative methodology, Management Communication Quarterly, 25(1), 4-58.

39.

Plano Clark

V. L.

Badiee

(2010). Research questions in mixed methods research. In Tashakkori

Teddlie

(Eds.), Sage handbook of mixed methods in social and behavior research (2nd ed., pp. 275-304). Thousand Oaks, CA: Sage.

40.

Ross

(1977). The intuitive psychologist and his shortcomings: Distortions in the attribution process. In Berkowitz

(Ed.), Advances in experimental social psychology (Vol. 10, pp. 174-220). New York, NY: Academic Press.

41.

Saunders

M. N. K.

(2011). The management researcher as practitioner: Issues from the interface. In Cassell

Lee

(Eds.), Challenges and controversies in management research (pp. 243-257). New York, NY: Routledge.

42.

Schulz-Hardt

Brodbeck

F. C.

Mojzisch

Kerschreiter

Frey

(2006). Group decision making in hidden profile situations: Dissent as a facilitator for decision quality. Journal of Personality and Social Psychology, 91, 1080-1093.

43.

Simpson

J. L.

Seibold

D. R.

(2008). Practical engagements and co-created research. Journal of Applied Communication Research, 36, 266-280.

44.

Spencer

D. G.

(1986). Employee voice and employee retention. Academy of Management Journal, 29, 488-502.

45.

Suddaby

(2006). From the editors: What grounded theory is not. Academy of Management Journal, 49, 633-642.

46.

Tashakkori

Teddlie

(1998). Mixed methodology: Combining qualitative and quantitative approaches (Applied Social Research Methods Series, No. 46). Thousand Oaks, CA: Sage.

47.

Teddlie

Tashakkori

(2009). Foundations of mixed methods research: Integrating quantitative and qualitative techniques in the social and behavioral sciences. Thousand Oaks, CA: Sage.

48.

Teddlie

Tashakkori

(2010). Overview of contemporary issues in mixed methods research. In Tashakkori

Teddlie

(Eds.), Sage handbook of mixed methods in social and behavior research (2nd ed., pp. 1-44). Thousand Oaks, CA: Sage.

49.

Teo

Caspersz

(2011). Dissenting discourse: Exploring alternatives to the whistleblowing/silence dichotomy. Journal of Business Ethics, 104, 237-249.

50.

Tushman

O’Reilly

(2007). Research and relevance: Implications of Pasteur’s quadrant for doctoral programs and faculty development. Academy of Management Journal, 50, 769-774.

51.

Van de Ven

A. H.

Johnson

P. E.

(2006). Knowledge for theory and practice. Academy of Management Review, 31, 802-821.

52.

Waldron

V. R.

(2007). Public scholarship, relational practice: A reflection on “virtuous” partnerships. Management Communication Quarterly, 21, 118-125.

53.

Wensley

(2011). Seeking relevance in management research. In Cassell

Lee

(Eds.), Challenges and controversies in management research (pp. 258-274). New York, NY: Routledge.

54.

Yin

(2006, April). Mixed methods research: Are methods genuinely integrated or merely parallel? Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.