Abstract
Hybrid brainstorming is ecologically more valid than all-interactive or all-noninteractive brainstorming, yet understudied. Although ideational benefits of hybrid groups have been found, studies have rarely focused on its affective/motivational contributions or ability to select ideas. In a randomized experiment, noninteractive-then-interactive (hybrid) groups perceived (1) higher goal clarity, engagement, and task attractiveness, and (2) chose more quality ideas than all-noninteractive groups. Additionally, (3) given the instruction for both hybrid and all-noninteractive conditions to be critical in idea selection, participants individually selected ideas that were more useful, thus overall higher quality, than the nonselected.
Keywords
A series of lab experiments (Meadow & Parnes, 1959; Meadow et al., 1959; Parnes & Meadow, 1959) support that “a brainstorming session, when properly conducted, can produce more good ideas than a conventional conference” (Sutton & Hargadon, 1996, p. 686). However, due to production blocking, social loafing, and evaluation apprehension, brainstorming groups that interact tend to produce ideas of lower quantity and quality than individuals brainstorming separately (Stroebe et al., 2010). Without measures such as using a trained facilitator, electronic media, or groups with heterogeneous knowledge bases (see DeRosa et al., 2007; Kramer et al., 2001; Stroebe & Diehl, 1994), creative performance of all-interactive groups would remain controversial.
Generating Creative Solutions in a Hybrid
While noninteractive-versus-interactive brainstorming has long dominated the discussion on the utility of brainstorming (Korde & Paulus, 2017), available evidence refers us to the benefits of a hybrid of noninteractive and interactive brainstorming. First, brainstorming practitioners have consistently implied that for productive interactive brainstorming, time between each interaction should be packed with individuals’ exhaustive efforts to generate ideas on their own (Brown, 2009; Kelley, 2001; Osborn, 1953). Korde and Paulus (2017), using face-to-face (FtF) brainwriting, accordingly found that hybrid sessions produce more ideas than those who repeatedly generate ideas either only noninteractively or only interactively (see also Ocker et al., 1998). More related to the present study, Rotter and Portugal (1969) used FtF brainstorming without a facilitator to compare across all-interactive, all-noninteractive, noninteractive-then-interactive (hybrid), and interactive-then-noninteractive (hybrid) groups. Both of their hybrid groups produced more ideas than all-interactive groups (see also Girotra et al., 2010) but fewer than all-noninteractive groups.
These findings suggest the potential as well as limitations of hybrid brainstorming, and encourage us to further investigate its productivity, given the paucity of data. Particularly, whether FtF hybrid brainstorming with a trained facilitator would perform better than all-noninteractive brainstorming without a facilitator remains untested. Since interactive brainstorming is typically led by a facilitator in managerial/organizational settings (Kelley, 2001; Osborn, 1953), a facilitator may add ecological validity (Kramer et al., 2001; Offner et al., 1996)—though at the expense of some internal validity (see Limitations).
Making Creative Decisions in a Hybrid
Second, although many studies did not find evidence that either an all-noninteractive or all-interactive group can select better ideas than the other, nor that either group can discern the best ideas above chance level (Faure, 2004; Putman & Paulus, 2009; Rietzschel et al., 2006; though not without controversy, e.g., see discussion in Larey & Paulus, 1999), there are reasons for hybrid groups’ potential to select better creative ideas. For example, pre-evaluation of ideas, which is highly compatible with or even spontaneous in a hybrid setting, has been found to later help select ideas with slightly higher overall quality (Rietzschel et al., 2010). Consistent with this pre-evaluation hypothesis, idea evaluation by hybrid (noninteractive-interactive) groups was positively, though very weakly, correlated to independent coders’ evaluation, while there was no such correlation for all-interactive groups (Girotra et al., 2010). The necessity for selecting the best idea is assumed in brainstorming (Ahn & Van Swol, 2021). However, noninteractive-then-interactive groups’ ability to select better ideas, particularly in comparison to all-noninteractive groups, has not been reported.
Non-Idea Benefits
Third, interactive brainstorming generates various non-idea benefits. Kramer et al. (1997) found that, compared to groups without training in brainstorming techniques, both all-interactive and all-noninteractive brainstorming groups perceived communication to be more satisfying, decision making more effective, and the group process more equitable. However, they did not find differences between all-interactive and all-noninteractive groups. Henningsen and Henningsen (2013) showed that all-interactive brainstorming groups experienced greater cohesiveness than all-noninteractive groups. This was consistent with a survey of managers and professionals by Dennis and Reinicke (2004), who showed that compared to either all-noninteractive or electronic brainstorming, all-interactive FtF brainstorming is perceived as more capable of supporting group well-being (group cohesiveness and relationships) and providing member support (individual growth and network building). Research on different types of hybrid brainstorming has also tested for non-idea benefits. Groups using a combination of FtF and asynchronous electronic brainstorming were more satisfied with their outcome ideas, though not process itself, than all-FtF groups or all-asynchronous electronic brainstorming groups (Ocker et al., 1998). De Vreede et al. (2010) found no differences in satisfaction with outcomes or process between multiple all-noninteractive groups versus multiple subgroups building on previous subgroups. However, we know of no research on non-idea benefits of noninteractive-then-interactive FtF hybrid brainstorming (see Hypotheses for reasons for this design).
The illusion of group productivity during interactive brainstorming is likely associated with various non-idea benefits like greater process satisfaction (Paulus et al., 1993, 1995). Nonetheless, as long as such non-idea benefits (1) are valuable in and of themselves in social/organizational settings (Dennis & Reinicke, 2004; Kramer et al., 1997), and (2) could feed into group performance in the long run (e.g., Rodríguez-Sánchez et al., 2017), this illusion of group productivity may not be problematic.
Motivation from interaction
Sutton and Hargadon (1996) performed ethnographic research of highly innovative professionals at a leading design consulting firm. They reported that interaction, which is part of hybrid brainstorming, motivates each employee in that organization, imbuing them with lasting enthusiasm and vigor for their own projects after the meeting was finished (Sutton & Hargadon, 1996). To our knowledge, no quantitative study has focused exclusively on comparing motivation from all-noninteractive with hybrid brainstorming. Note as well that motivation is qualitatively different from previously tested non-idea benefits (e.g., satisfaction, cohesiveness, perceived increased efficacy). While satisfaction or cohesiveness may not be a necessary component for creativity (the componential model of creativity; Amabile, 1988), motivation is—together with domain expertise and creative processes. Specifically, intrinsic motivation, which is described by a deep “commitment to” (task engagement) or being “attracted by” the work itself (Amabile, 1988, p. 133), is critical for creativity as opposed to extrinsic, controlling pressures or motivators such as expected evaluation, contracted-for reward, or external directives. These extrinsic factors tend to fail to motivate (Amabile & Pratt, 2016). Notwithstanding their emphasis on intrinsic motivation, Amabile and Pratt (2016) recently updated their componential model by specifying that certain sources of influence external to an individual (e.g., funding for successful work or any social/organizational environment that helps focus on the given goal) may result in “synergistic extrinsic motivation.” As long as a synergistic the extrinsic motivator does not attempt to control or override individuals’ self-autonomy, it enhances intrinsic motivation. Given that any social environment that enables individuals’ deeper involvement with a task can be a synergistic extrinsic motivator (Amabile & Pillemer, 2012), we propose that hybrid brainstorming qualifies as one if it enhances perceived engagement and task attractiveness. Furthermore, clarity of an organizational or project “goal”—one of the foremost catalysts to create intrinsic motivation (Amabile & Pratt, 2016, p. 169)—is likely enhanced during interaction in hybrid brainstorming, because as communication can foster a shared mental map (Weick, 1993).
Hypotheses
The present study compares a hybrid of noninteractive-then-interactive brainstorming and all-noninteractive brainstorming using the FtF paradigm, for several reasons. First, both are arguably the most ecologically valid, and thus basic, two modes of brainstorming, at least in FtF settings (Sutton & Hargadon, 1996). Noninteractive brainstorming was considered the default and essential (Osborn, 1953). Interactive brainstorming was conceived as an adjunct to noninteractive brainstorming when individuals need help from others (Osborn, 1953; Paulus et al., 1995), so all-interactive brainstorming would be the least in sync with organizational practice. Interactive-then-noninteractive brainstorming, although useful, would be an exception when members lack basic knowledge of the topic (e.g., when kicking off a new project; Dugosh & Paulus, 2005; Sutton & Hargadon, 1996). Second, the noninteractive-then-interactive brainstorming sequence has been found to be more productive than the interactive-then-noninteractive sequence (Baruah & Paulus, 2008; cf. Paulus et al., 1995), and it has been widely known that all-noninteractive groups outperform all-interactive groups (Stroebe et al., 2010). Thus, it seems more interesting to compare the two (noninteractive-then-interactive groups and all-noninteractive groups) that have been found as more productive. Third, current work builds partly on the FtF brainstorming study by Girotra et al. (2010), which compared noninteractive-then-interactive groups with all-interactive groups in idea generation and selection. For these reasons, we pit hybrid of noninteractive brainstorming followed by interactive brainstorming against all-noninteractive brainstorming.
Goal Clarity
Shared vision, or goal orientation, in a group promotes the collective pursuit of creative solutions (García-Morales et al., 2006). In a task environment, people rely on a mental model, an organized knowledge structure unique to each individual (Mathieu et al., 2000). For team functioning, ongoing communication is pivotal to translate individual mental models into a shared one (April, 1999) and encourage a superordinate goal (Weick, 1993). Thus, hybrid group members who communicate with each other can better reflect on the common goal to reach in comparison to all-noninteractive groups, who do not benefit from such interaction.
H1: Members of a hybrid group will perceive their goal with greater clarity in comparison to members of an all-noninteractive group.
Perceived Engagement
Maslach et al. (2001) define engagement 1 as marked by vigor and absorption. Vigor is a high level of energy and mental resilience. Absorption is being fully focused and engrossed in work and is likened to flow (Schaufeli et al., 2002), which involves centering of attention, loss of self-consciousness, and deep sense of enjoyment (Csikszentmihalyi, 1990). Engagement in creative collaboration generates intense momentum (Osborn, 1953) and instigates a back-and-forth dynamic of enthusiasm (Brown, 2009), which carries it through “the darkest and most pressure-tinged stages of a project” (Kelley, 2001, p. 56). Without it, a team would not be able to stay on course or reach solutions to tough problems (Kelley, 2001). These outcomes of brainstorming are likely to be unique to a hybrid group.
H2-a: Hybrid group members will have greater vigor than all-noninteractive group members.
H2-b: Hybrid group members will experience deeper absorption in the task compared to all-noninteractive group members.
Task Attractiveness
A tendency to join, leave, or stay with a brainstorming group would be affected by how much a member perceives the task of brainstorming attractive to perform. This is distinct from the social attractiveness of a group, which exists when members are attracted to a group because of, for example, sharing similar values (Carron et al., 1985). Innovative processes are iterative and nonlinear (Brown, 2009), so an organization that wants to innovate ought to have its employees engage and re-engage in the ideation process often and consistently. Moreover, creative thinking is physically and mentally demanding (Csikszentmihalyi, 1988). If idea generation per se is unattractive and daunting as a task, without associated processual rewards, members would not desire to re-engage in the activity. Enjoyable idea generation would, by contrast, attract members to reengage. In an ethnographic study (Sutton & Hargadon, 1996), employees described group brainstorming as “most invigorating” (p. 700) and said that they “had to go” (p. 697) and brainstorm despite their busy schedule. A number of quantitative studies supported this conclusion (e.g., Paulus et al., 1993; Stroebe et al., 1992). We extend this result to hybrid brainstorming:
H3: Hybrid group members will be more attracted to the given creativity task compared to all-noninteractive group members.
Generation of Ideas
By stating that “the creative power of the individual still counts most,” Osborn (1953, p. 139) clarified the relationship between interactive and noninteractive idea generation as complementary rather than competitive, though this idea has not been fully reflected in the literature (Korde & Paulus, 2017). Further, there are likely other reasons than ideas alone (e.g., goal clarity, engagement, and task attractiveness) for group brainstorming (Sutton & Hargadon, 1996). Thus, extending previous findings that all-noninteractive groups outperform all-interactive groups in idea quantity and quality in FtF brainstorming (though in FtF brainwriting and electronic brainstorming it has been shown that groups can outperform individuals DeRosa et al., 2007; Korde & Paulus, 2017) to the comparison between FtF all-noninteractive and hybrid groups would not lead us to simplistically conclude that hybrid groups are less useful than all-noninteractive groups. In line with H1 to H3, it would serve to examine if noninteractive and interactive brainstorming provide distinctly complementary benefits. A key assumption behind this interpretation would be that individuals are generally situated within social contexts where they alternate between interactive and noninteractive brainstorming to generate both motivation (Brown, 2009; Kelley, 2001) and creative solutions, particularly when one who has run out of new ideas wants to get unstuck (Sutton & Hargadon, 1996; see Dunbar, 1997). However, our design does not operationalize individual needs for a brainstorming meeting nor do our participants go through several alternating interactive and noninteractive sessions (see Korde & Paulus, 2017). Instead, we simply question if all-noninteractive brainstorming would outperform hybrid brainstorming, while being clear about what we cannot extrapolate from our results. Past research has largely supported the superior productivity of all-noninteractive brainstorming over all-interactive brainstorming (Stroebe et al., 2010). Additionally, the fact that all-noninteractive groups do not need time to share written ideas with others, whereas hybrid groups do at the start of and throughout the interactive session, increases the possibility that all-noninteractive groups will produce more ideas. However, it is possible hybrid brainstorming will outperform noninteractive brainstorming given the presence of facilitators during interaction, which tends to enhance productivity (Kramer et al., 2001; Offner et al., 1996; Oxley et al., 1996). Further, hybrid groups members might generate a sufficiently large number of ideas in solitude during the first few minutes (when momentum is strong, Kohn & Smith, 2011), which precedes the interactive session. It is only with the nuanced understanding of the social contexts we have discussed that we ask:
RQ1: Will all-noninteractive groups outperform hybrid groups in quantity of ideas?
Studies revealed that idea quantity is positively associated with idea quality (Stroebe et al., 2010). Given that either all-interactive or all-noninteractive groups could outperform the other in idea quantity (RQ1) and that we determine idea idea quality by the number (instead of ratio) of ideas rated “good” on usefulness, originality, and both usefulness and originality (overall quality), we ask:
RQ2: Will all-noninteractive groups outperform hybrid groups in the number of non-selected ideas that are good on (a) usefulness, (b) originality, and (c) overall quality?
Selection of Ideas
Girotra et al. (2010) compared noninteractive-then-interactive (hybrid) brainstorming groups against all-interactive brainstorming groups. Their hybrid brainstorming groups first individually evaluated ideas at the end of the noninteractive phase and then interactively evaluated ideas after the interactive phase. This led the hybrid brainstorming groups to select ideas that were better than ideas selected by all-interactive groups, who evaluated ideas only once as a group after the interactive session. This finding is encouraging given the dearth of empirical research on idea selection (Paulus et al., 2019). There is also some realism in explicitly asking participants to evaluate their ideas before joining the group, because members of a group or organization would often be asked or expected to bring good ideas to the meeting. Yet, it is possible that (1) the explicit request for the hybrid group members to rank ideas by the end of their first (noninteractive) phase, and (2) the difference in whether decision making was done alone or as a group may have affected results. Considering these points, in the present study, we test if hybrid groups will select better ideas than all-noninteractive groups when there is the same single request for both conditions to individually evaluate ideas at the end of the entire idea generation session.
Hybrid groups may excel in decision making for several reasons. One reason is that they are likely to voluntarily process their own ideas at least one more time than all-noninteractive groups. When the noninteractive phase ends and interaction starts, hybrid participants would likely share their better ideas earlier than lower-quality ideas. This would be due to them trying to appear competent to their peers. These “‘prestige’ or ‘status’ auctions” naturally take place during interactive brainstorming (Sutton & Hargadon, 1996, p. 705). This implies that members, before sharing or during the short transition from the alone to group session, process their ideas on their qualities to decide which to share first. This, as an eliminative process key to reducing cognitive burdens (Simon, 1955), would make the next round of idea evaluation cognitively less overwhelming and more effective (Haught-Tromp, 2017; Iyengar & Lepper, 2000).
H4: Hybrid group members will outperform all-noninteractive group members in the number of individually selected quality ideas evaluated for their (a) usefulness, (b) originality, and (c) overall quality.
In some previous studies (e.g., Putman & Paulus, 2009; Rietzschel et al., 2006), both all-interactive and all-noninteractive groups were not able to select ideas above chance level. However, Rietzschel et al. (2010, 2014) showed that, with explicit instruction to select “creative” ideas, groups tend to select more original but no less feasible ideas. These researchers noted that clearly communicating creativity to participants as the selection criterion for them to use contrasted with previous studies that asked participants to simply select best (e.g., Putman & Paulus, 2009) or favorite (e.g., Faure, 2004; Rietzschel et al., 2006) ideas. Asking participants to select best or favorite ideas may not have been sufficient to override what Mueller et al. (2012) call an implicit bias people have toward practicality and against creativity.
We extend these findings on the relationship between clearly communicated selection criteria and selection quality (Rietzschel et al., 2010, 2014) to the opposite direction toward usefulness. One reason for this direction is that to be creative is to produce something both original and useful (Stein, 1953). Thus, usefulness poses an equally important question as originality. With the clear and emphatic communication to think critically during idea selection, we expect that the proportion of useful ideas will be higher among the selected ideas than among the nonselected ideas. Another reason for our idea selection criterion is that, rather than emphasis on originality, emphasis on usefulness might better align with the goal of idea selection in industrial contexts (see Cooper, 1990), on which our task is based. Even if we find the expected results, however, they will not necessarily suggest that selection of more useful ideas is due to the specific evaluative criterion to be critical, because the evaluative criterion does not vary across conditions. Nonselected ideas will be compared against only the individually selected ideas (see Method), given discussion for group decision making will be only in the hybrid condition (see H6).
H5: In both hybrid groups and all-noninteractive groups, proportion of quality ideas will be higher for individually selected than nonselected ideas evaluated for their (a) usefulness, which in turn will positively affect (b) overall quality.
Finally, we anticipate hybrid groups will select higher-quality ideas during the subsequent group discussion. After the individual selection, hybrid group members will be asked to freely discuss and debate to select as a group their top ideas out of the individually selected ideas. The rationale is that because there will be a smaller number of ideas to consider (only nine ideas—three from each of the three members), it will likely be less cognitively demanding than having to choose from a larger number of options (Rietzschel et al., 2014). With fewer options, more cognitive resources will be available for careful consideration of each (Schwartz, 2004).
H6: After the initial individual idea selection, hybrid group members will select ideas of higher (a) usefulness, (b) originality, and (c) overall quality through free group discussion.
Method
Participants
The study was approved by the university IRB. Participants were 161 undergraduate students from a large Midwestern university in the United States. They received extra credit in a communication class for their participation. Eleven participants who did not follow instructions were excluded, resulting in a final sample of 150. Participants were randomly assigned to hybrid (36 groups, n = 108) or all-noninteractive groups (14 groups, n = 42). Hybrid groups followed seven brainstorming rules (“Defer judgment,” “Encourage wild ideas,” “Build on the ideas of others,” “Stay focused on the topic,” “One conversation at a time,” “Be visual,” and “Go for quantity,” Brown, 2009; Kelley, 2001; Osborn, 1953; Putman & Paulus, 2009) 2 ; all-noninteractive groups followed the same rules modified for them, which resulted in eliminating the “One conversation at a time” rule. For hybrid groups, three participants entered the lab and interacted throughout the session. For all-noninteractive groups, there were only one or two participants in the lab. The three-person all-noninteractive groups were determined and aggregated in the order that these separate individuals’ data were collected.
Materials and Procedures
The presence of a trained facilitator and use of semiotic resources (a whiteboard, post-it notes, and markers) were the default (Brown, 2009; Kelley, 2001; Offner et al., 1996) in both conditions.
Seven facilitators took turns running the experimental sessions: Six were undergraduate research assistants selected by a communication professor to assist in lab operations. The facilitation was counter-balanced and randomly assigned to sessions. The first author facilitated four hybrid groups when research assistants were unavailable. Facilitators trained by engaging in several pilot sessions. While leading, the facilitator had a guideline packet at hand explaining the rule enforcement with examples of what to say. Part of these explanations followed the example of Putman and Paulus (2009). The first author watched the pilot sessions and provided detailed feedback, so that level and style of facilitation could be largely equivalent. This study constituted one-half of a larger laboratory session that involved two studies.
Hybrid groups
Upon entering the lab, participants completed informed consent, after which they were randomly assigned to either hybrid or all-noninteractive condition. They sat at a table in front of a whiteboard and received a marker, post-it notes of a certain color (each participant per session was given a different color), and a rule sheet. After viewing a two-minute video illustration of group brainstorming (Cheung, 2017), they completed a two-minute warm-up exercise with a different topic. After this warm-up, the facilitator presented the main problem: Create the best backpack in the world for college students like you! You are a member of a product development team at a huge sportswear company. The sales are down 50%. The future of your company hinges on your creative ideas!
The topic was carefully selected based on pilot testing and consideration of a proper difficulty level given the undergraduate participants’ familiarity with the object. For the main brainstorming task, participants in the hybrid condition generated ideas individually for 6 minutes, and then brainstormed together for 12 minutes. This ensured that members in the hybrid condition started interacting only after having generated ideas individually. For hybrid groups, the facilitator was outside the lab only during noninteractive brainstorming (6 minutes). During interaction, participants freely took turns sharing their individually generated ideas, while at the same time generating more ideas. Six minutes was selected, as (1) Kohn and Smith (2011) demonstrated that idea generation slows down considerably within 5 minutes, which we also observed during the pilot sessions, (2) the duration of each session (6 or 12 minutes) was pilot-tested several times to ensure that they were adequate in length, and (3) we adopted the same 1:2 ratio from Girotra et al. (2010), who used a hybrid of a 10-minute individual brainstorming session followed by a 20-minute interactive brainstorming session.
When the time was up, participants were asked to be completely critical and select their individual top three ideas in solitude. Then participants in this hybrid condition were asked to come together as a group to freely debate and select the top three ideas of the group. During this discussion within hybrid groups, there was no emphasis on criticalness. After completing the survey, participants were given debriefing forms and thanked.
All-noninteractive groups
All-noninteractive groups were the same as the hybrid condition, except for a few adjustments. First, any interactive elements and facilitation by the facilitator were excluded. All-noninteractive group members were either alone or did not interact when there was another participant in the room. These participants used post-it notes and markers to record their ideas but did not use the whiteboard. The facilitator was outside the room during the entire noninteractive brainstorming (18 minutes). Second, two rules were modified (“Welcome wild ideas” instead of “Encourage wild ideas” and “Build on previous ideas” in place of “Build on the ideas of others.”) Finally, all-noninteractive participants did not have a group discussion after the critical individual selection of their top three ideas.
Measures
To account for non-independence among members in hybrid groups, multilevel modeling with group ID as a random factor with correlated error variance was used for analysis of level-1 outcome variables (i.e., the survey data, idea quantity and quality). Fitting models with the following equations, we specify condition (hybrid or all-noninteractive group) as the level-2 predictor (RQ1, RQ2, and H1–H4),
where i = individual, and j = group, respectively; individual selection (or nonselection) as the level-1 predictor (H5),
where i = phase (before or after individual selection), j = group; and having an additional group discussion (or not) within hybrid groups as the level-1 predictor (H6),
where i = phase (before or after group discussion), and j = group, respectively.
Motivational Benefits
Goal clarity
Three items (e.g., “Our group had one clear goal”) measured goal clarity on a scale of 1 (strongly disagree) to 7 (strongly agree). The items were adopted from a vision scale by García-Morales et al. (2006). Questions were modified for all-noninteractive groups (e.g., “I had a clear goal during the task”). Reliability was strong (Rwg = 0.77; for interpretation, see LeBreton & Senter, 2008). See the Appendix for the survey items.
Engagement
Engagement was comprised of 7 items (Rwg = 0.61) measuring vigor and absorption. Three items (e.g., “During the task, my group was full of energy”) were adopted from Salanova et al.’s (2003) vigor scale, and measured on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree). For noninteractive groups, questions were modified (e.g., “During the task, I was full of energy”). Reliability was moderate (Rwg = 0.54). Absorption was comprised of four items (e.g., “My group was immersed in the task”) adopted from the absorption scale (Salanova et al., 2003) and measured on the same 7-point scale as vigor. Items were modified (e.g., “I was immersed in the task”) for all-noninteractive groups. Reliability was moderate (Rwg = 0.60).
Task attractiveness
Four items (e.g., “I liked the guidelines for brainstorming in this group”) adopted and modified from the task attractiveness scale (Estabrooks & Carron, 2000) were measured on a scale of 1 (strongly disagree) to 7 (strongly agree). Questions were modified for all-noninteractive groups (e.g., “I liked the guidelines for brainstorming”). Reliability was strong (Rwg = 0.74).
Quantity
Number of ideas generated by each participant was used to measure idea quantity. Unlike hybrid groups, all-noninteractive group participants would generate a few duplicate ideas because they were not aware of the ideas that others in the same noninteracting group generated. Only one of a duplicate idea was counted.
Quality
The number of ideas out of all nonselected ideas that were rated as “good” on usefulness was taken as the usefulness score for nonselected ideas. The number of ideas out of the three individually selected ideas that were “good” on usefulness was taken as the usefulness score for individually selected ideas. To compare the level of usefulness between nonselected and selected ideas (H4), the proportion (not the total number) of nonselected ideas “good” on usefulness was compared against the proportion of selected ideas “good” on usefulness. Measurement and comparisons of originality and overall quality were done in the same manner.
Coding scheme
A total of 3,346 ideas by 150 participants were rated on usefulness and originality (Litchfield et al., 2011) using a scale from 1 (very bad) to 4 (very good). An idea with a score of 3 or above was good and 2 or below bad for each of the two criteria. Usefulness was content-analyzed by three independent coders and originality 3 by another three independent coders. The first round involving 250 ideas did not meet inter-coder reliability on usefulness (KALPHA = .668) and originality (KALPHA = .591). The next round with another 250 ideas had a KALPHA = .909 and .830, respectively. The coders then rated the remaining 2,846 ideas. Only ideas rated good on both usefulness and originality were counted as ideas good on overall quality. 4
Results
Motivational Benefits
Goal clarity
We hypothesized (H1) hybrid group members (M = 6.32) would perceive their goal with greater clarity than all-noninteractive members (M = 5.93). H1 was supported, γ01 = 0.40, df = 48, t = 2.54, p = .015. Intraclass correlation (ICC) 5 was .07, indicating that approximately 7% of total variance in perceived goal clarity was attributable to group differences. The effect size was an increase of 0.50 standard deviation (SD) 6 (Tables 1 and 2).
Descriptive statistics (Means and Standard Deviations) of Outcome Variables.
HLM Estimates of the Hybrid-Group Effects (Level-2 Predictor) on Non-Idea Benefits in Comparison to All-Noninteractive Groups (H1–H3).
p < .05; **p < .01; ***p < .001 (two-tailed).
Engagement
We hypothesized that hybrid members would have higher vigor (H2-a) and absorption (H2-b) than all-noninteractive members. Vigor was higher for hybrid members (M = 5.38) than for all-noninteractive members (M = 4.88), γ01 = 0.50, df = 48, t = 2.08, p = .043, ICC = .23, and the effect was a 0.46 SD increase. Thus, H2-a was supported. Absorption was higher for hybrid members (M = 5.50) than for all-noninteractive members (M = 4.84), γ01 = 0.67, df = 48, t = 3.61, p < .001, ICC = .10, and the effect size was 0.72 SD favoring hybrid members. Thus, H2-b was supported.
Task attractiveness
H3 stated that hybrid members (M = 6.18) would perceive stronger attraction to the task compared to all-noninteractive members (M = 5.70). H3 was supported, γ01 = 0.48, df = 48, t = 2.88, p = .006, ICC = .08, with an effect size of 0.56 SD increase.
Generation of ideas
Quantity
Quantity of ideas all-noninteractive members generated (M = 22.86) was significantly greater than hybrid members (M = 16.85), γ01 = 6.01, df = 48, t = 3.42, p = .001, ICC = .12, with an effect size of 0.69 SD increase. The answer to RQ1 (if all-noninteractive brainstorming would generate ideas of greater quantity) was positive (Table 3).
HLM Estimates of the Hybrid-Group Effects (Level-2 Predictor) on Idea Generation in Comparison to All-Noninteractive Groups (RQ1 and RQ2).
p < .05; **p < .01; ***p < .001 (two-tailed).
Quality of non-selected ideas
Quality of ideas was evaluated on usefulness, originality, and overall quality. The number of non-selected ideas rated good on usefulness was greater for all-noninteractive members (M = 11.36) than for hybrid members (M = 7.07), γ01 = 4.28, df = 148, t = 4.56, p < .001, ICC < .01; the effect was a 0.83 SD increase. The answer to RQ2-a was positive. The number of non-selected ideas rated good on originality was greater for all-noninteractive members (M = 12.60) than for hybrid members (M = 9.28), γ01 = 3.32, df = 148, t = 3.30, p = .001, ICC < .01; the effect size was an increase of 0.61 SD. The answer to RQ2-b (if all-noninteractive groups would generate more original ideas) was positive. The number of non-selected ideas rated as good on overall quality (good on both criteria) was greater for all-noninteractive members (M = 8.21) than for hybrid members (M = 6.67), γ01 = 1.55, df = 148, t = 1.91, p = .058, ICC = .12, with an effect size of 0.35 SD increase. The answer to RQ2-c (if all-noninteractive groups would generate more overall high-quality ideas) was marginally positive.
Selection of Ideas
Quality of individually selected ideas was evaluated on usefulness, originality, and overall quality to compare between the hybrid and all-noninteractive conditions (H4). The number of individually selected ideas rated good on usefulness for hybrid members (M = 2.20) was greater than for all-noninteractive members (M = 1.74), γ01 = 0.47, df = 148, t = 3.09, p = .002, ICC < .01, and the effect was a 0.57 SD increase; H4-a was supported (Table 4). An average hybrid member did not individually select more ideas good on originality (M = 1.59) than an average all-noninteractive member (M = 1.76), γ01 = –0.17, df = 48, t = –1.02, p = .311, ICC = .08; H4b was unsupported. There were no more individually selected ideas rated good on overall quality for hybrid (M = 1.29) than all-noninteractive members (M = 1.21), γ01 = 0.08, df = 148, t = 0.51, p = .614, ICC < .01; H4-c was unsupported.
HLM Estimates of the Hybrid-Group Effects (Level-2 Predictor) on Individual Idea Selection in Comparison to All-Noninteractive Groups (H4).
p < .05; **p < .01; ***p < .001 (two-tailed).
The usefulness and overall quality criteria were used to compare the quality of individually-selected versus nonselected ideas (H5). The comparisons were done within either hybrid or all-noninteractive groups, respectively. In hybrid groups, the proportion of useful ideas among individually selected ideas (M = .74) was higher than nonselected ideas (M = .44), γ01 = .30, df = 179, t = 10.34, p < .001, ICC = .08. The effect was a 1.30 SD increase; H5-a was supported for hybrid groups (Table 5). In hybrid groups, the proportion of ideas good on overall quality among individually selected ideas (M = .43) was higher than nonselected ideas (M = .36), γ01 = .07, df = 179, t = 2.45, p = .015, ICC = .01, and the effect was a 0.32 SD increase; H5-b was supported for hybrid groups. The better overall quality of the ideas individually selected by an average hybrid group member (H5-b) seems to be due to them selecting more useful (H5-a) yet no less original ideas. In all-noninteractive groups, the proportion of useful ideas among individually selected ideas (M = .58) was not significantly higher than nonselected ideas (M = .49), although there was a trend in the positive direction, γ01 = .09, df = 82, t = 1.67, p = .098, ICC < .01, and the effect was a 0.35 SD increase; H5-a was not supported for all-noninteractive groups. In all-noninteractive groups, the proportion of ideas good on overall quality among individually selected ideas (M = .40) was not higher than nonselected ideas (M = .34), γ01 = .07, df = 82, t = 1.195, p < 0.235, ICC < .01, and the effect was a 0.28 SD increase; H5-b was unsupported for all-noninteractive groups.
HLM Estimates of the Individual Selection Effects (Level-1 Predictor) on the Proportion of Quality Ideas in Comparison to Nonselected Ideas for Each Condition (H5).
p < .05; **p < .01; ***p < .001 (two-tailed).
Only hybrid groups had a brief group discussion after the individual idea selection to decide the top three ideas at the group level (Table 6). So, we tested the discussion effect for hybrid groups based on the usefulness, originality, and overall quality of ideas they selected (H6). In hybrid groups, number of ideas selected through group discussion rated good on usefulness (M = 2.42) was greater than their individual selection (M = 2.20), γ01 = 0.21, df = 395, t = 3.14, p = .002, ICC = .34, and the effect was a 0.28 SD increase; H6-a was supported. In hybrid groups, group discussion also resulted in more original ideas (M = 2.21) than individual selection (M = 1.61), γ01 = 0.60, df = 395, t = 6.60, p < .001, ICC = .33, and the effect size was an increase of 0.80 SD; H6-b received support. In hybrid groups, more of the three selected ideas were of higher overall quality after group discussion (M = 1.89) than after individual selection (M = 1.31), γ01 = 0.57, df = 395, t = 7.01, p < .001, ICC = .26, and the effect was a 0.66 SD increase; H6-c was also supported.
HLM Estimates of the Group Discussion Effects (Level-1 Predictor) on Idea Selection in Comparison to the Preceding Individual Idea Selection (H6).
p < .05; **p < .01; ***p < .001 (two-tailed).
Discussion
A hybrid of noninteractive and interactive FtF brainstorming is widely practiced in the industry (e.g., Amabile et al., 2014; Osborn, 1953; Sutton & Hargadon, 1996). However, no research in FtF settings has compared noninteractive-interactive to all-noninteractive brainstorming on (1) motivational benefits, (2) idea generation, and (3) idea selection (cf. Girotra et al., 2010). The operationalization of hybrid brainstorming in this study was noninteractive-then-interactive without the explicit request for the interaction phase that would likely be found in an organization. There was a trained facilitator during interaction for hybrid groups.
Non-Idea Benefits
Hybrid groups, in comparison to all-noninteractive, resulted in enhanced goal clarity (H1); engagement, that is, vigor (H2-a) and absorption (H2-b); and task attractiveness (H3). Within the componential model of organizational creativity, synergistic extrinsic motivation is externally derived, but enhances the existing intrinsic motivation. Interaction within hybrid brainstorming can likewise be a synergistic extrinsic motivator. Thus, it does not seem to be a coincidence that experienced practitioners deliberately use interactive brainstorming as an idea generator and a motivator (e.g., Kelley, 2001; Sutton & Hargadon, 1996). More broadly, motivation and other non-idea effects from collective creative activities are beneficial to employees (for networking, well-being, Dennis & Reinicke, 2004; obtaining skill variety, Sutton & Hargadon, 1996; equitable decision making, Kramer et al., 1997; group/team cohesiveness, Henningsen & Henningsen, 2013; reinforcing organizational memory, Sutton & Hargadon, 1996). In the long run, these non-idea benefits could feed into group performance (for example, see Rodríguez-Sánchez et al., 2017). Further research could test if interactive-noninteractive brainstorming or alternating between the two modes of brainstorming would produce any non-idea benefits.
Idea Generation
In FtF settings, hybrid groups without a facilitator have been found to produce more ideas than all-interactive groups (Girotra et al., 2010; Rotter & Portugal, 1969) but fewer than all-noninteractive groups. When assisted by a trained facilitator, which is a normal industry practice (Kramer et al., 2001; Offner et al., 1996), all-interactive groups can be as productive as all-noninteractive groups (Kramer et al., 2001; Offner et al., 1996; Oxley et al., 1996). We compared the productivity of noninteractive-interactive hybrid groups in the presence of a trained facilitator with all-noninteractive groups. Even with a facilitator, our hybrid groups generated ideas of lower quantity (RQ1), usefulness (RQ2-a), originality (RQ2-b), and overall quality (RQ2-c, though marginally significant with p = .058) than all-noninteractive groups. Given quantity breeds quality (Stroebe et al., 2010), it is not surprising that our all-noninteractive group members, who produced ideas of greater quantity, also produced ideas of greater quality than the hybrid group members. There may be multiple possible explanations for this productivity loss in our hybrid groups. There likely have been production blocking, which arises because members must take turns to share their ideas in a group (Diehl & Stroebe, 1987). Moreover, the time required for each member to share with other members ideas that they individually generated prior to interaction would have prevented hybrid groups from using all their time for idea generation. Insufficient facilitator training is another possibility. However, we argue that, first, researchers need to look beyond the traditional productive-counterproductive discussion and address the situated nature of brainstorming (see Csikszentmihalyi & Sawyer, 2014; Nielsen & Miraglia, 2017). In real group or organizational contexts, some trade-off between idea-wise productivity and non-idea, socio-emotional benefits might be necessary. Second, interactive brainstorming would probably be most effective when a member of an organization, who has exhaustively generated ideas on their own and reached an impasse, seeks additional or breakthrough ideas (Sutton & Hargadon, 1996), which we have not operationalized.
Idea Selection
Individual selection by all-noninteractive versus by hybrid groups
Hybrid group members had more individually selected useful ideas (H4-a) than all-noninteractive groups, while there was no difference in originality (H4-b) nor in overall quality (H4-c). That noninteractive-interactive brainstorming seems to have an advantage in idea selection may carry a heavy implication for organizations that want to innovate, because selected ideas are the ones to be considered for development, elaboration, and implementation (Ahn & van Swol, 2021).
Better individual idea selection by hybrid groups may be due to several reasons including, first, greater cognitive ease after voluntary pre-evaluation that helps simplify the decision task (Iyengar & Lepper, 2000; Simon, 1955). This cognitive ease interpretation is based on our reasoning that sometime before interaction starts, participants would have quickly processed their own ideas. This would help participants present more appealing ideas first, so that they could appear as or more competent than others (see status auction, Sutton & Hargadon, 1996; see also social comparison theory, Festinger, 1954).
Another likely reason is that hybrid group members had a bigger pool of high-quality candidate ideas to choose from. For example, even though an average all-noninteractive member generated more ideas of good overall quality (M = 8.21; see Table 1) than an average hybrid member (M = 6.67), an average hybrid group of three members together generated far more good overall quality ideas (6.67 × 3 = 20.01) than a single average all-noninteractive group member.
Third, hybrid group members observed in real time during interaction the reactions of other group members to each idea being proposed. This experience could have increased hybrid group members’ acuity in idea evaluation. Despite the nonjudgmental brainstorming rules, spontaneous verbal (e.g., “Oh, I love that!”) and nonverbal (e.g., nodding) communication may provide more reference points by which to judge idea quality (Kerr & Murthy, 2004). Observing this feedback, the members in our study could have formed a preliminary impression of each shared idea, which would then let them make a more objectively informed, updated judgment (see Elqayam & Evans, 2013) at a later time to select ideas individually.
Fourth, path dependence may be a reason for the difference (Girotra et al., 2010). Participants in the interactive phase after the solo session were exposed to others’ ideas without prior information on the path by which those ideas were generated. Thus, participants in hybrid groups may have cognitively responded to ideas themselves, rather than to ideas and their history. In contrast, those in the all-noninteractive condition experienced the path by which the ideas were generated, which could have rendered them less able to judge from a more disinterested standpoint, as they evaluated only their own ideas, all of which they would already have formed some impression about.
Lastly, a social setting itself might have increased the sense of accountability (Lerner & Tetlock, 1999) and motivation to deliberate more. Relatedly, mere verbalizing of and hearing ideas during interaction could have increased attention on idea quality. We do not know from our data whether the ability of hybrid members to individually select better ideas than all-noninteractive members was due to either reduced cognitive burdens, having more alternatives, being less biased, or social feedback or motivation. Future research should tease out the sources of variation in the ability to detect more creative solutions.
Interestingly, findings in Putman and Paulus (2009) allow us to extend the advantage of going hybrid in idea selection to another combination of idea generation and selection. They showed that noninteractive individuals who later met and discussed the shared ideas made better selections than interactive individuals who likewise subsequently discussed the ideas as a group. Note that their noninteractive condition can also be referred to as hybrid in the sense that the selection process was done in groups. Comparing one’s own individual ideas (noninteractive condition) with the ideas of others likely involved less assimilation-contrast effects than discussing interactively generated ideas (Hovland et al., 1957). Participants who interactively generated ideas (interactive condition) would already have had some judgment of each idea’s worth (with a semblance of the said path dependence). This judgment, functioning as an anchor, could have undermined the potential benefit of group discussion in making better judgments. Communication from other members about an idea that is near one’s own pre-formed judgment of the idea would have been assimilated to it, and communication at variance with their own impression would have been contrasted away from it. In contrast, participants who generated ideas alone would have come to learn about others’ ideas for the first time during group discussion, thus unaffected by such pre-formed judgment. Another explanation is that, because open disagreement would have been more difficult among groups who had been brainstorming together and forming some bonds, any disagreement during group discussion in the noninteractive condition would have been more authentic. Authentic disagreement has been found to lead to deeper analysis and more unconventional solutions (Nemeth et al., 2001). Not only the hybrid of noninteractive-then-interactive brainstorming as in our data but also the hybrid of noninteractive brainstorming followed by group discussion as in Putman and Paulus (2009) benefits idea selection. This potentially indicates a high generalizability of the positive effect of a hybrid of noninteractive and interactive sessions on creative decision making.
Nonselected versus individually selected ideas
We tested, with a clear criterion to be critical, if there would be increased proportions of useful (H5-a)—and consequently high overall quality (H5-b)—ideas from individual selection in both conditions. For hybrid groups, individually selected ideas were more useful and of overall higher quality than nonselected ideas. For all-noninteractive groups, individually selected ideas showed a trend of being more useful (p = .098), but not of higher overall quality, than nonselected ideas. There was a common pattern in both conditions that usefulness of selected ideas was higher than nonselected ideas (H5-a), while the effect sizes differed by nearly 1 SD (1.30 SD for hybrid and 0.35 SD for all-noninteractive groups). This seems to suggest that the difference in the effect sizes of usefulness primarily accounted for whether overall quality of selected ideas (H5-b) was significantly higher (for hybrid groups) or not (for all-noninteractive groups) than nonselected ideas.
Several previous studies without communicating clear criteria did not find a difference between nonselected and selected ideas (Faure, 2004; Putman & Paulus, 2009; Rietzschel et al., 2006). In contrast, our study had a clear criticalness criterion, and it is interesting to see that this strong criticalness criterion likely induced participants to individually select more useful (H4-a) but no less original ideas. This is analogous to Rietzschel et al. (2010, 2014) who communicated to participants a specific criterion (i.e., creativity) and found results in that direction—selecting more original but no less feasible ideas. The present study extended their findings in the direction to criticalness and usefulness. Nonetheless, data do not suggest a causal link between our specific emphasis on critical selection and selecting more useful ideas, since we did not manipulate the selection criteria.
When evaluating creative ideas, members may not have many relevant past experiences to base judgment on, because by definition, creative ideas are new to some degree (Stein, 1953). Having few relevant reference points would make the evaluation difficult and its result inaccurate. Here, clear criteria would enable more concrete mental simulation of events based on the idea. Clear criteria could also prime us to restrict the search space in memory and facilitate finding a relevant anchor within it, with which evaluation and selection would become easier (see Schwartz, 2004).
Group discussion in hybrid groups
Among hybrid groups, after a brief, free group discussion following critical individual selection, participants as a group chose ideas that were more useful (H6-a), original (H6-b), and of higher overall quality (H6-c) than their individually selected ideas. Similarly to some of the possible explanations for H4, we interpret these results as indicating an effect of reduced cognitive burdens resulting from multiple individual prior assessments and/or having to compare a smaller number of ideas during group discussion than individual selection. Since there was no instruction to be critical for this group discussion, participants freely debated which ideas to decide upon. So, we do not think the presumed effect of being particularly critical during the preceding individual selection period carried over to this stage. More research is needed to disentangle these effects.
Applied questions
Finally, that hybrid structures can enhance idea generation (Korde & Paulus, 2017; Osborn, 1953; Sutton & Hargadon, 1996) as well as selection (Girotra et al., 2010; Putman & Paulus, 2009) prompts us to discuss, briefly, what an ideal creative procedure might be like. From a cognitive perspective, one might be electronic brainstorming in an adequately large group (e.g., four or more, Gallupe et al., 1992; eight or more, DeRosa et al., 2007), in which there is alternation between solitary and collaborative sessions. Since members can work simultaneously, electronic brainstorming notably reduces production blocking (Gallupe et al., 1991, 1992), which Diehl and Stroebe (1987) suggested as a foremost cause of productivity loss during FtF interactive brainstorming. Deciding upon an adequate group size would be necessary to maximize anonymity that increases with group size, so that evaluation apprehension may be decreased, and minimize the temptation to free ride (social loafing), which may also increase with group size (Gallupe et al., 1992). Unless members need basic topical knowledge to spark ideas (see Dugosh & Paulus, 2005), it might be helpful to begin with individual ideation to allow members to generate their own unique ideas without the biasing effects of exposure to others’ ideas (see Baron, 2005). One reason for alternation is that, although noninteractive brainstorming is highly productive, a subsequent interactive brainstorming can provide additional and breakthrough ideas as members with diverse cognitive styles and expertise participate (Stroebe & Diehl, 1994). The benefit will be even greater when individuals have already exhaustively generated ideasc and called for a brainstorming meeting (Sutton & Hargadon, 1996). Another reason is that social stimulation may occur due to exposure to a high performance standard (Paulus & Dzindolet, 1993). At some point another solitary session would be useful to allow individuals to reflect on the ideas shared by other members, without any possible distraction of the collaborative process (Pinsonneault et al., 1999).
With a high number of ideas generated in such a process, selecting the best idea(s) will be a challenge. However, either as individuals or as a group, during idea generation members will be provided access to the ideas already generated and continue generating new ideas or building on prior ideas (Gallupe et al., 1991). This facilitates focusing on more promising ideas and thereby spontaneously eliminating many other options (Harvey & Kou, 2013). Then, at some point the individual members as independent judges could be asked to highlight those ideas they considered the best (Amabile, 1982; Brown, 2009), and the high-scoring ideas could be selected, developed, and implemented (Cooper, 1990). Lastly, given the ubiquitous uncertainty inherent in creative projects, it will be necessary to update and revise beliefs about the idea or product based on iterative trials and errors throughout idea generation, selection, and implementation (Brown, 2009; Elqayam & Evans, 2013).
Besides the cognitive concerns, however, we should also consider social-affective and cultural-normative aspects. There is evidence FtF ideation excels electronic brainstorming in promoting group well-being (e.g., reinforcing the creative culture and help socializing) and member support (e.g., creating status auctions and knowledge networks) (Dennis & Reinicke, 2004; for review, see Maaravi et al., 2021). Also, during idea selection and implementation, a transition to FtF interaction might be necessary in order to visualize ideas particularly if the project involves physical products.
Limitations
This study has several limitations. First, most importantly, Sutton and Hargadon (1996) reported that interactive brainstorming provides the critical moments to get unstuck from an impasse to individuals who have been wrestling with a difficult problem for protracted periods. This may be one of the most important benefits of hybrid brainstorming that our study did not test. Second, we have not tested interactive-then-noninteractive hybrid brainstorming. Without more comprehensive comparisons, it is not possible to determine the unique effects of the hybrid experience versus others. Third, we had a facilitator for the interaction phase of the hybrid condition. This may have increased external validity, given the presence of a facilitator is a widely accepted practice in the industry (Kelley, 2001; Osborn, 1953), however, at the expense of internal validity. For example, the presence of a facilitator in the lab during brainstorming also could have had an influence on the self-report ratings in the hybrid condition. Fourth, it is possible that the brief, free discussion to select ideas among hybrid group members affected their reported motivational effects, although participants were asked to answer the questionnaire specifically regarding the idea generation phase. Fifth, the study design did not allow for a comparison between hybrid and all-noninteractive brainstorming on the effect of group discussion on selection. Lastly, we did not manipulate the evaluative criterion.
Conclusion
Hybrid brainstorming led to greater perceived motivation as it creates a clearer goal orientation, greater engagement (vigor and absorption), and stronger task attractiveness compared to all-noninteractive brainstorming. Thus, we argue that interactive or hybrid brainstorming can be an example of a synergistic extrinsic motivator (Amabile & Pratt, 2016). All-noninteractive groups generated more and better ideas than hybrid groups with a trained facilitator. Hybrid groups selected a greater number of useful ideas than all-noninteractive groups. The proportions of selected ideas in both conditions showed trends of being higher in usefulness than nonselected ideas. That the ideas were better particularly in usefulness may be due to our emphasis on being critical during individual idea selection. Additionally, hybrid groups had a free discussion and selected, as a group, ideas that were better in usefulness, originality, and overall quality than their individually selected ideas.
Our findings suggest that it may be a false dilemma to consider abandoning one mode of idea generation (e.g., all-interactive or real groups) in favor of the other (e.g., all-noninteractive or nominal groups) given the different relative advantages individuals and groups have. Certain hybrids of noninteractive and interactive modes appear beneficial for idea selection.
Supplemental Material
sj-docx-1-sgr-10.1177_10464964211043565 – Supplemental material for Enhanced Motivation and Decision Making from Going Hybrid
Supplemental material, sj-docx-1-sgr-10.1177_10464964211043565 for Enhanced Motivation and Decision Making from Going Hybrid by Paul Hangsan Ahn, Lyn M. van Swol, Sang Jung Kim and Hyelin Park in Small Group Research
Footnotes
Appendix
Survey items for hybrid (all-noninteractive) groups
Vision
Vigor
Absorption
Attraction to the task
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
Notes
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
