Facilitating collaboration with collaborative technology: Prerequisites and evidence from an explanatory mixed-methods study

Technology use and its effectiveness

Even before the advent of the Internet in the late 1980s, public organizations actively leveraged information technology to enhance operational efficiency and improve internal communication (Bozeman & Bretschneider, 1986; Bretschneider, 1990; Ho, 2002; K. L. Kraemer & Dedrick, 1997). By the late 1990s and early 21st century, technological advancements and the widespread use of computers fueled a rapid increase in ICT adoption within public organizations across the whole world.

Many studies during this period adopted a celebratory tone, highlighting technology’s potential to revolutionize public service delivery, enhance managerial effectiveness, and promote democratic values. Predictive models and frameworks emerged, such as staged models designed to describe, predict, and guide the evolution of technology in public organizations (Layne & Lee, 2001; Mergel & Bretschneider, 2013; Moon, 2002). Such overemphasis on technological changes resulted in a significant overlook of the organizational and administrative contexts of digital governance practices. Subsequent empirical studies later revealed that the actual progression through e-government stages, if such stages exist, is “neither as accelerated nor as simple as the models posit.” (Coursey & Norris, 2008, p. 532).

Meanwhile, information technology has predominantly been employed to reinforce existing administrative and political arrangements instead of defining a political system or leading the organizational transition process (Biancone et al., 2023; Hinkley, 2022; Richardson & Emerson, 2018). Similarly, when discussing the role of e-government in promoting citizen participation, scholars worldwide found that despite ICT’s explosive adoption, only a few local governments had adopted e-democracy initiatives, among which few have achieved their intended goals, and even fewer local governments planned to adopt e-democracy shortly (Federici et al., 2015; Gupta & Biswas, 2021; Norris & Reddick, 2013). For example, Zou et al. (2023) argued that “technocraticism prevails in the research and practice of e-government. . . the use of ICT does not automatically foster greater participation, nor does it lead to good governance.” (p. 3). It becomes clear that the process and impacts of ICTs on the public sector have been more of a gradual evolution rather than a revolutionary transformation (Lindquist & Huse, 2017; Norris, 2010; Stratu-Strelet et al., 2021).

Despite the practical backlash, technology adoption in government and its potential effectiveness has long been a shared interest of many scholars and practitioners (Maclean & Titah, 2021; Mas & Gómez, 2021). As of June 13, 2024, the E-Government Reference Library v20.0 (EGRL) contains 19,932 references on digital government, digital governance, and digital democracy (Scholl, 2024). A search of the EGRL v20.0 using “performance” as the keyword revealed a diversity of topics, including website quality and usability, e-governance performance, e-service effectiveness, e-project efficiency, public value creation and integration, public crisis management, success rates of e-project implementation, and the performance of intelligent decision-making (Nam, 2019; Mao et al., 2021; Panagiotopoulos et al., 2019; Schuppan, 2009; Sharma et al., 2021; Twizeyimana & Andersson, 2019). These studies, unsurprisingly, indicated a broad set of factors practically impacting technology effectiveness on government operation, transformation, and value creation. Examples of these influential factors are users’ demographic conditions and IT experience, their perceived technology usefulness and risks, motivation of employees, cultural and political situation, leadership, IT governance, organizational process and decision making, resources, task-technology fit, collaboration within or across organizations, among others.

Collaboration at different levels and potentials of technology

Collaboration generally refers to two or more individuals or organizations working together to accomplish common goals (Batory & Svensson, 2019). Governance describes the decision-making process that implements new initiatives and resolves issues (Biancone et al., 2023). Collaborative governance is thus a process involving multiple individuals or organizations working on collective decision-making and problem-solving.

Extensive public administration research has explored the mechanisms, challenges, and effectiveness of collaborative governance, particularly in addressing emerging issues such as emergency management, healthcare, environmental protection, and immigration, which demand cross-boundary cooperation and collective problem-solving (Biancone et al., 2023; McNaught, 2024; Susha & Gil-Garcia, 2019). These studies primarily concentrated on interorganizational collaborative efforts spanning organizational boundaries, supported by frameworks and models that illustrate the dynamics and processes of collaboration. For example, Emerson et al. (2012) referred the collaboration process and dynamics to principled engagement, shared motivation, and capacity-building for joint action. According to Ansell and Gash (2008), influential factors of the collaboration process include policy and organizational agreement, levels of conflicts or trust, shared understanding, communication, leadership, external demands, resources, interdependence, and incentives.

Another common collaboration scenario involves individuals within the same department or across several departments within an organization working together to complete a task (Mac McCullough et al., 2020; Reeb, 2023). This intraorganizational collaboration environment differs from interorganizational conditions in several ways, such as the presence of a baseline level of trust among members, the greater likelihood and frequency of face-to-face interactions, and a shared understanding of the organization’s mission, goals, culture, and plans. For instance, collaboration at the individual and group level is less hindered by conflicting missions or values and a lack of commitment. Instead, it allows for more focused planning and substantive communication among participants. Insights from conflict resolution and psychology literature are particularly relevant at this intraorganizational level, whereas organizational ecology and contingency theory offer greater applicability to collaboration in interorganizational settings.

No matter what level it is, measuring collaboration performance could be challenging given the conceptual and methodological complexity and the normative appeal of using collaboration to solve problems (Douglas & Ansell, 2021; O’Leary & Vij, 2012). Also, the inconsistency in operationalizing and measuring key aspects and effectiveness of collaboration, the difficulty in observing and evaluating the evolving collaboration performance over time, the biased self-reported perceptions on collaboration performance, and the diverging perspectives of different collaboration participants and organizations are all challenging the practice of collaboration performance measurement (Emerson & Nabatchi, 2015; Moynihan et al., 2020; Poocharoen & Wong, 2016).

Another challenge in collaborative governance is the relationship between governance processes and technology adoption, particularly with advancing collaborative technologies (Barandiarán et al., 2019; Bolívar, 2018). Studies on this topic have investigated areas such as healthcare, water management, and smart city development, with the common assumption being that collaborative technologies can engage potential collaborators, enhance transparency, accessibility, and data-driven decision-making in collaborative governance (Batory & Svensson 2019; Gilman, 2017; Hinkley, 2022). Collaborative technologies are information communication tools that enable sharing, exchanging, and integrating data across boundaries among individuals and organizations (Sun et al., 2022). They also facilitate interactions by accelerating the speed and flow of information, both synchronously and asynchronously. On the other side, collaboration programs call for technologies to meet the growing need for information and knowledge exchange and integration. Collaboration managers are expected to capture the attention of individuals and organizations, organize information flow, and promote knowledge sharing through both formal and informal communication channels in addition to traditional face-to-face interactions (Biancone et al., 2023; Hinkley, 2022). This demand becomes a key driver for adopting collaborative technologies, which offer significant benefits in breaking down boundaries and speeding up information exchanges.

The COVID-19 pandemic accelerated technology adoption across the public sector, highlighting the potential of digital tools to facilitate intraorganizational and interorganizational collaboration and remote service delivery during crises. While these tools present opportunities to enhance collaborative governance, the successful integration of technology into such processes requires a deeper understanding of the dynamics and relationships between technology use and the collaboration process and performance (Biancone et al., 2023; Chen & Lee, 2018; Mitchell et al., 2015).

Consequently, this study explores the relationship between the use of collaborative technology and the collaboration process and performance within the context of intraorganizational collaboration. After examining this relationship, the study will further investigate the conditions critical to effectively using collaborative technology. As this case study seeks to provide empirical evidence of these relationships, it intentionally refrains from proposing conceptual frameworks or hypotheses at the outset. However, a more robust framework, grounded in the study’s findings, is expected to emerge by the conclusion of the paper.

Data collection and analysis

The research site is an interlocal government technology commission in the Midwest of the U.S. that centrally provides technical support and consulting to approximately 70 governmental entities and over 5,000 dedicated government workers across 120 locations. As a nationally recognized leader in government IT management, the organization is dedicated to providing high-quality, cost-effective technology services and facilitating technology innovations across local government departments. The organization was selected to participate in this study primarily due to its innovative organizational culture and committed leadership in technological advancements, such as the use of collaborative technologies.

The study began with face-to-face conversations with the organization’s CIO, program manager, and division directors to establish rapport, understand the context, explain the study’s purpose, and seek cooperation. This phase lasted about 8 months, culminating in a precise study procedure and timeframe, including participating groups, intervention, and instruments. Figure 1 illustrates the detailed research design and data collection process. Specifically, the organization aimed to enhance performance by transitioning from simple communication tools (i.e. emails, messages, and Google Docs) to a unified collaborative project management tool (Quip), which was identified as the intervention in this study. Among the several divisions in the organization, two (the Administrative Division and Technical Division) were recommended by the organizational leadership to participate in this study. Specifically, the Administrative Division manages projects, processes, and resources that ensure smooth department operations. Employees in this division rely on strong organizational, managerial, and interpersonal communication skills to carry out their jobs, which emphasize long-term planning and maintaining organizational cohesion. Technical Division, on the other hand, specializes in tasks that directly contribute to the organization’s core products, services, or technical infrastructure. Employees in this division focus on innovation and technical problem-solving. Their work requires expertise in specific technical fields and often involves collaboration with other teams to develop solutions or complete projects. Unlike the Administrative Division, the Technical Division is more dynamic and innovation-driven, requiring more leadership and technological support.

OPEN IN VIEWER

Figure 1.

Design of the case study.

Under each division, two departments were randomly assigned to the treatment and control group by tossing a coin. This procedure ended with four participating departments: the PMO department and Cloud department were treatment groups (11 participants), and the Finance department and Oracle department were control groups (9 participants). All four departments were asked to do a departmental-level strategic planning task within 5 weeks. Back in the days before 2018, this department-level task was done biennially by each department head alone. Starting in 2018, the organization sought to reform its strategic planning process, shifting it from a task traditionally assigned solely to department heads to one that requires the active engagement of all department employees. To achieve this goal, participants in this study were asked to work collaboratively with colleagues from the same department to develop their departmental-level strategic planning reports.

At the quantitative stage, the focus is to explore How the new collaborative technology affects the collaboration process and performance? All groups participated in the pretest Qualtrics survey before the intervention. Web survey tools such as Qualtrics, compared with mail and telephone survey tools, have lower costs, are more efficient, and create more complex questionnaire designs (Newcomer & Triplett, 2015). After the pretest, participants in treatment groups were given a new collaboration software, Quip, to do a departmental-level strategic planning task. Like other collaborative project management tools, Quip is designed to facilitate teamwork, streamline project workflows, and enhance communication among team members. Quip combines documents, spreadsheets, checklists, and chat into one platform, providing a real-time communication solution that promotes accelerated, connected, and transparent team collaboration.

Participants in control groups were also invited to do the same task without receiving the new software Quip. Instead, they continued to utilize emails and Google Docs, the tools that had been used in the past for the task. Table 1 demonstrates that Quip offers several advantages over Google Docs in areas like integrated communication, task management, real-time updates, and a unified workspace for project management and team collaboration.

OPEN IN VIEWER

Table 1.

Comparing Quip to Google docs.

Aspect	Quip	Google Docs
Integrated communication	Combines documents, spreadsheets, and team chat in one platform, facilitating seamless communication.	Separate tools for documents and chat (Google Docs and Google Chat), requiring switching between apps.
Task management	Built-in task lists and project management features allow tracking tasks and deadlines within the same interface.	No built-in task management; relies on integration with Google Tasks or other third-party tools.
Real-time collaboration	Real-time updates and notifications within documents and chat, keeping team members informed instantly.	Real-time collaboration within documents, but notifications are managed separately through email or other apps.
Unified workspace	Single interface, reducing the need to switch between multiple tools.	Requires using multiple Google Workspace apps (Docs, Sheets, Chat, etc.) for a unified experience.

During the experiment, treatment group participants received tutorial videos introducing the features and functions of the new collaborative technology Quip. Since Quip is easy to use, standardized training sessions were not provided for treatment group participants. Instead, they were asked to watch tutorial videos at their own pace for self-training. After 5 weeks, all groups participated in the posttest Qualtrics survey to collect their perceptions of the departmental and technological environment and the role of technology they used to accomplish the task. Appendix 1 shows detailed measures built on insights from previous collaboration literature and public information management literature. Specifically, the interaction level of the collaboration process was measured using indicators such as communication, common understanding, shared knowledge, and interpersonal relationships. In addition, collaboration performance was evaluated on trust, accountability, time-saving, and contribution to achieving collaboration goals. The 10-level Likert Scale was used in the above measurement. Appendix 2 displays the specific survey questions investigated in the experiment.

The pretest and posttest surveys indicated how participants felt about using technology in interpersonal collaboration, such as the strategic planning task. However, they failed to demonstrate how participants utilized technology tools and why they perceived that way. To further explain the “how” and “why” questions, follow-up semi-structured interviews were conducted with all 20 survey participants in September and October 2018. Semi-structured interviews allowed for “a dialogue whereby initial questions are modified in the light of the participants’ responses and the investigator can probe interesting and important areas which arise” (Smith & Osborn, 2008, p. 57). At the beginning of each face-to-face interview, the interviewee was informed orally about the nature and purpose of the interview, the expected amount of time to complete the interview, and the investigators’ contact information. Detailed interview protocols can be found in Appendix 3. All interviews were recorded and transcribed except one with a treatment group participant. The average interview length was about 24 min; the shortest took 17 min and the longest took 43 min. Results of the interviews contributed to explaining the experimental results collected from the pretest and posttest surveys.

In analyzing the pretest and posttest survey data, the Wilcoxon Signed-rank Test and Paired-samples T-Test were conducted to see the overtime trend in treatment and control groups. When comparing treatment groups with control groups, the tests were Independent Sample T-test and Mann-Whitney U Test, depending on the normality of the data. Specifically, all normally distributed data were tested using the Independent Samples T-test. Mann-Whitney U Test does not assume normally distributed data, but it does assume that the distributions are non-normal in a similar manner, which is a homogeneity of variance assumption and has been tested using Levene’s Test when needed.

All interview recordings were transcribed and then open coded in MAXQDA12, one of the most popular qualitative data analysis tools. Open coding ensures codes and themes emerge directly from the raw data, thus increasing the work’s validity.

Validity, reliability, and ethical issues

For surveys, all questions were first reviewed and discussed with the organization’s CIO and program manager to ensure the survey questions’ content validity and the measurements’ accuracy. The resulting questionnaires were then tested by the organization’s CIO, program manager, and division directors. Based on their comments and suggestions, modifications were made to the wording and question structure, and the end products became the questionnaires used in the pretest and posttest surveys. The surveys were deemed reliable as the pretest results revealed identical demographics (e.g. age, gender, and job experience) and comparable initial performance levels across all participating groups. Meanwhile, an analysis of the overtime trend showed that the groups’ overall perceptions of collaboration and the organizational and technological environment remained essentially unchanged following the intervention.

For interviews, several measures were taken to address the main validity issues in qualitative research (Rubin & Rubin, 2012). For example, to ensure the interviewees were knowledgeable about the research problem, all interviewees were participants in the pretest and posttest surveys in which they reported their experiences and perceptions before and after the intervention. To ensure they were speaking from firsthand experience, at the beginning of each interview, they were asked about their personal experiences of doing the planning task in the past and 2018. They also recalled working with/without the new technology tool. To ensure they were not just showing their best faces, interviewees were encouraged to discuss problems or challenges and make improvement suggestions. All interviews were conducted privately in an organizational meeting room between an interviewee and the interviewer(s).

This study also emphasized ethical issues such as voluntary involvement, informed/implied consent, confidentiality, and data security. At the beginning of each survey and interview, participants were notified literally or orally about the nature and purpose of this study. The perceived usefulness of the study became the primary incentive for employees to participate as they expected to see issues with the use of collaborative technology, how to improve working conditions, and the ultimate collaboration performance in their departments. Participants’ signatures in surveys and their completed interviews served the purpose of implied consent. To protect participants’ privacy and anonymity, it was guaranteed that the raw data from surveys and interviews would not be provided/reported to their department heads or organization managers. Meanwhile, information about individual participants’ names, departments, and job positions were hidden in writing the results in the next section. All the data were secured in a password-required folder.

Negative experiences with the new collaborative technology

Descriptive statistics in Table 2 and comparison in Figure 2 displayed the results before and after the intervention. When looking at the overtime trend, control groups in the year of the study reported a significantly (p < .05) increased level of trust in using Google Docs (z = 2.333) and an improved feeling of fitness between Google Docs and task needs (t = 2.630). Treatment groups, however, had a significantly (p < .05) decreased level of communication (t = −2.785) and shared knowledge over time (t = −2.345). The new collaboration software, Quip, was reported to be significantly (p < .05) less easy to use (t = −2.204), incompatible with existing systems (t = −2.463), and could not fit well with the collaboration task needs (t = −2.410). It was statistically less needed in doing the assigned task (t = −2.807), supported by department leadership (t = −2.609), and ineffective in saving time (t = −3.086) and achieving task goals (t = −2.915).

OPEN IN VIEWER

Table 2.

Mean values and standard deviation of key variables.

Variables	Pretest				Posttest
	Control groups (N = 9)		Treatment groups (N = 11)		Control groups (N = 9)		Treatment groups (N = 11)
	Mean	Std	Mean	Std	Mean	Std	Mean	Std
Communication	8.33	1.27	7.73	1.27	8.78	1.72	5.09	3.00
Common understanding	8.56	1.33	7.00	1.55	9.00	1.32	5.27	3.23
Joint actions	8.33	1.41	7.18	1.40	8.89	1.36	5.27	3.17
Interpersonal relationships	8.33	1.00	7.73	2.05	8.67	1.87	5.73	2.90
Trust	8.11	1.45	7.27	2.24	8.89	1.45	5.82	3.49
Accountability	8.44	1.13	7.64	1.78	8.78	1.72	6.00	3.58
Timesaving	7.67	1.50	6.36	2.01	8.56	0.88	4.00	2.72
Goal achievement	8.00	1.32	6.18	2.27	8.22	1.72	4.00	2.79
Technology complexity	8.44	1.13	6.36	1.75	8.56	0.88	4.36	2.54
Technology compatibility	7.78	0.97	6.27	1.74	8.33	1.09	4.64	2.29
Technology fitness	7.56	1.33	6.82	1.94	8.78	0.97	4.27	2.87
Incentives	9.33	0.71	7.91	2.30	8.67	1.00	3.09	2.81
Leadership	9.11	1.27	7.27	1.85	9.00	0.71	4.64	2.38
Resources	6.78	2.22	6.09	2.30	8.89	0.78	4.64	2.54

OPEN IN VIEWER

Figure 2.

Treatment groups and control groups in pretest and posttest.

When comparing the treatment groups to the control groups, descriptive statistics in Table 2 showed that numerically, control groups scored higher than treatment groups regarding the collaboration process and performance, both before and after the treatment. The Independent Samples T-test and Mann-Whitney U Test were conducted to test if these differences are statistically significant. Specifically, the Independent Samples T-test showed that treatment groups had a significantly lower rating of common understanding before the intervention, F(1, 18) = 0.461, t = −2.375, p < .05. Also, treatment group participants reported an afterward negative evaluation of “the new collaborative technology could save collaboration time,” F(1, 18) = 4.647, t = −4.801, p < .01.

The homogeneity of variance test of non-normally distributed data showed that posttest trust, communication, common understanding, and shared knowledge had a p-value less than 0.05, which implies the null hypothesis of equal distributions between the two groups has been rejected. Therefore, rather than using the Mann-Whitney U Test, the analysis used an Independent Samples T-test with bootstrapping (based on 1,000 bootstrap samples) at a 95% confidence interval as the estimation technique. The results in Table 3 showed that treatment groups had a significantly lower level of posttest trust (t = 2.652), communication (t = 3.212), common understanding (t = 3.489), and shared knowledge (t = 3.421) than control groups.

OPEN IN VIEWER

Table 3.

Independent samples T-test with bootstrapping.

Variables	Levene’s test for equality of variances		T-test for equality of means		Bootstrap
Posttest:	F	Sig.	t	Sig. (two-tailed)	Sig. (two -tailed)
Trust	9.992	0.005	2.652	0.019	0.022
Communication	7.314	0.015	3.212	0.006	0.011
Common understanding	8.437	0.009	3.489	0.004	0.006
Shared knowledge	7.274	0.015	3.421	0.004	0.006

Note. N = 20. Bootstrap results are based on 1,000 bootstrap samples.

All other measures of collaboration performance and process were tested using the Mann-Whitney U Test. Treatment groups reported significantly worse feelings about the after-treatment interpersonal relationship (z = −2.870, p < .01) and whether the new collaborative technology contributed to achieving collaboration task goals (z = −2.389, p < .05).

To conclude, treatment groups had a significantly lower rating of common understanding of the collaboration task before the test. After the intervention, compared to emails and Google Docs used by control groups, the new collaboration software Quip used by the treatment group negatively affected the trust, communication, common understanding, shared knowledge, and interpersonal relationships among employees. It was also less effective in saving collaboration time and contributing to achieving collaboration goals. Figure 2 helps us understand these comparison results between treatment groups and control groups.

Is there a significant difference between the technological and organizational environments of control and treatment groups? The Independent Samples T-test results in Table 4 showed that in doing strategic planning in the past, treatment groups felt the technologies they used were significantly less easy to use and incompatible with existing systems. Unfortunately, the new collaboration software Quip did not change their perceptions. Also, it failed to fit the collaboration task needs well, not to mention the concerns with the funding. What’s more, the Independent Samples T-test with bootstrapping showed that the treatment group had a significantly lower level of posttest technologies’ ease to use (F(1, 18) = 7.471, t = 5.109, p < .01) and leadership support (F(1, 18) = 8.369, t = 5.782, p < .01) than the control group.

OPEN IN VIEWER

Table 4.

Independent samples T-test.

Variables	Levene’s test for equality of variances		T-test for equality of means
Pretest:	F	Sig.	t	Sig. (two-tailed)
Ease-to-use	1.227	0.283	3.076	0.007
Compatibility	2.437	0.136	2.313	0.033
Posttest:
Compatibility	2.721	0.116	4.485	0.000
Task-technology fit	8.077	0.011	4.489	0.000
Funding	6.053	0.024	4.817	0.000

Note. N = 20.

All other measures of the technological and organizational environment were tested using the Mann-Whitney U Test. Treatment groups reported a significant negative perception of leadership in using technologies for strategic planning before the intervention. After using the new collaboration software Quip, they felt it was less needed to do the task.

To summarize, in doing strategic planning in the past, the treatment group reported less supportive leadership and the technologies they used were significantly less easy to use and incompatible with existing systems. Such pretest survey results underscored the treatment groups’ hope for more decisive leadership and user-friendly technology tools. Unfortunately, the condition did not change much even with the new collaborative software Quip, which they believed was even less easy to use, incompatible with existing systems, incapable of fitting the task needs well, and not needed and supported by department leadership. Also, the potential funding issue caused a big concern among treatment group participants.

Why and how?

The above experiment results inspired the research interest in understanding Why the use of new collaborative technology failed to improve the interpersonal collaboration process and performance? How might such a result be affected by the organizational and technological context? Face-to-face semi-structured interviews with all experiment participants were then administered to explore the above “why” and “how” questions.

Surprisingly, employee participants in treatment groups complained that they did not see any collaboration in doing the task by saying, “I don’t have an opportunity to really sit down and plan.” “I was not involved in the planning of it.” This resulted in limited experience in using the new collaborative technology. For example, some of them said that, “organizationally, we just didn’t come together to use it (Quip).” “Never used it.” “For the documents that were in Quip, I reviewed them, but I don’t know if I put any edits within the documents themselves.”

In contrast, control group participants were glad to see encouraged engagement and inclusive participation in accomplishing the assigned strategic planning task this year. They shared that “we had a meeting, in person first. . .put it out for everybody and then took feedback.” Then, “we just did it in our normal. . .team meeting. . .We reworded a few things and switched a few things around.”

Several factors were identified as roadblocks to using the new collaborative technology in treatment groups. For example, treatment group interviewees mentioned that they do not have time to learn it, “I had a difficult time finding time to understand how to use this.” This might be because they did not see the needs or benefits of using it, “it just wasn’t enough advantage with Quip to make it worth that effort.” Another reason might be a lack of commitment and support they saw from the department leadership side. “[G]iven our time constraints, our manager was not actively soliciting that information.” Reasons like no time to learn, no need to learn, and lack of leadership support helped explain why they reported limited knowledge about the new technology, “I can learn something, but I don’t know enough now.” Together, these factors brought in a possible misunderstanding of the features and values of the new collaborative technology, while some insisted that “it didn’t match our business need.”

In fact, participants who did have certain experiences with the new collaborative technology spoke highly of it. “Yeah, it was pretty easy from what I looked at, and saw. . .it was pretty compatible with what I saw, and what they were really using it for. . .it did everything I would want it to do.” “From a collaboration and communications standpoint, I liked it, if I can recall back when I used it. So, easy to access, easy functionality to get around, easy to download stuff.”

People without much experience with the new technology tended to avoid giving an evaluation regarding its effects. One interviewee said, “in my belief, it had little impact, just because we didn’t use it. Not that it couldn’t have, but we just didn’t.” Others were less willing to assess as they said, “I don’t know that I’d looked at it in enough detail to really come to and give you a truly honest assessment to say I see or did not see a weakness.” “I didn’t use it enough to be able to make a fair judgment.” This finding helped interpret the previously reported decreased level of overtime perceptions of treatment groups on the role of new collaborative technology in the collaboration process and performance. More importantly, it demonstrated a lack of interpersonal collaboration within treatment groups. This unexpected condition could impair the interests and incentives of participants; it also discouraged them from spending time on and using the new technology for collaboration purposes.

Looking forward, interviewees in both groups shared thoughts on ensuring an effective transition from old technology to a new one. The four most commented suggestions were stakeholder involvement, leadership, mechanisms of facilitating learning, and planning. Interviewees were interested in participating, “I think, if anything, getting everybody involved in the initial discussion was a good step.” From the participation, they could get “a little more ownership and a little more accountability, and a little more pride.” Interviewees also expressed strong expectations of sustained and supportive leadership throughout the collaboration project. “Leadership. Yeah, I think we didn’t have. And that wasn’t defined very well.” “[T]he manager’s insistence. . .So it’s both the expectation and follow-through from management to use the product. . .sustainable leadership.”

What’s more, they emphasized the importance of learning either via departmental training or self-learning, by saying “we need an additional understanding of the functionality” “[T]rain people. . .So we would need to identify what are our current needs, and what are our current requirements, and what of those requirements and needs are our current toolset not capable of meeting?” Besides, there is a need for a project plan that defines expectations, goals, roles, processes, and timeframe. “It’s just like having a project plan. You say: how are we going to communicate? How often are we going to do it? What’re our goals? What do we need to accomplish? Who’s responsible for what?”

Additionally, interviewees highlighted the significance of having face-to-face conversations, in addition to online communication. “[S]ome internal conversations that could’ve helped encourage a different level of participation. At least give it a shot. I don’t think we took those preparatory steps.” “We still expected to have a personal, just talking in the hallway type conversation. . .And then he edited within Quip.”