Abstract
This paper addresses the impact of dialogue strategies and functional features of spatial arrangements on communicative success. To examine the sharing of cognition between two minds in order to achieve a joint goal, we collected a corpus of 24 extended German-language dialogues in a referential communication task that involved furnishing a dolls’ house. Results show how successful communication, as evidenced by correct placement of furniture items, is affected by: (a) functionality of the furniture arrangement; (b) previous task experience; and (c) dialogue features such as description length and orientation information. To enhance research in this area, our ‘Dolldialogue’ corpus (www.dolldialogue.space) is now available as a free resource.
1 Introduction
Dialogic interaction is the prime use of natural language, and it is fundamentally influenced by our shared concepts about the world around us (common ground in the sense of Clark, 1996; Clark & Brennan, 1991). Since this world is constituted by spatial relationships, one prime target of cognition and communication research is a better understanding of how spatial concepts shape dialogue. Spatial cognition research has highlighted a range of systematic, but remarkably complex ways in which object relationships are conceptualized and represented in language (e.g., Miller & Johnson-Laird, 1976; Talmy, 1983). Beyond the purely spatial (geometric) aspects, objects are frequently functionally related (Coventry & Garrod, 2004), as when chairs are arranged around a table and tableware is set ready for dinner. Such functional relationships are related to the affordances of objects (Gibson, 1979), and are part of more general action scripts (Schank & Abelson, 1977) that are essential in human life, and systematically affect the way we talk about our world.
To date, very little data have been available which are suitable for showing relevant effects at work in natural communication, related to scenarios in which spatial object relationships and their functions are relevant. Moreover, although insights on the features of linguistic interaction are steadily increasing, we know surprisingly little about their relationship to task success with respect to actions in the real world. Here we introduce a novel dialogue corpus with carefully designed features that set it apart from previously available data, and provide the first evidence about the effects of spatial relationships and dialogue strategies on communicative task success.
Previous work on cognition and dialogue includes brief, rigidly controlled interaction settings involving a confederate (who controls the dialogue according to a script without their interaction partner’s knowledge; Branigan, Pickering, & Cleland, 2000), a wide range of controlled (yet more open) settings known as referential communication tasks (where reference needs to be established by verbal communication: Clark & Wilkes-Gibbs, 1986; Kemper, Vandeputte, Rice, Cheung, & Gubarchuk, 1995), and settings such as the 2D ‘map task’ with spatial information discrepancies needing to be resolved by dialogue (Anderson et al., 1991; Newlands, Anderson, & Mullin, 2003). Furthermore, there is an increasing range of natural language data available that does not incorporate experimental control or variation except for varying the dialogue topic and speaker background (e.g., Jurafsky, Shriberg, & Biasca, 1997). Confederate studies preclude a natural development of dialogue as a joint effort (Clark, 1996), as they do not allow for naturalistic and dynamic dialogue flow (Kuhlen & Brennan, 2013). Most referential communication tasks focus on the ways in which reference to pictures or objects is achieved between speakers, but do not involve a wider goal to be achieved through action. Further, while some of the more extended dialogue studies hitherto reported are concerned with spatial relationships, few incorporate spatial action such as object placement based on verbal interaction (see Tenbrink, Eberhard, Shi, Kübler, & Scheutz, 2013, for an overview). As a result, we still know very little about the features of a dialogue that is evidently successful with respect to its effects on real world actions. In spite of steadily accumulating insights about how communication works, in the absence of clearly evaluable communicative task goals we can only speculate what makes communication work successfully.
Moreover, there is no dialogue corpus available to show how central aspects of the real world, forming the topic and goal of communication, affect the success of the interaction. Yet human understanding of real world relationships is fundamentally based on experience about object functions, embedded into action scripts that guide everyday behaviour (Schank & Abelson, 1977) and affect how humans conceptualize objects in the world (Brewer & Treyens, 1981). Functional spatial relations (e.g., objects placed in an ‘expected’ arrangement affording object interaction in the sense of Gibson, 1979) are remembered better than non-functional ones (Radvansky, Copeland, & Zwaan, 2003), and arranging objects functionally is quicker than arranging the same objects non-functionally (Coventry, Venn, Smith, & Morley, 2003). Whether an object is conceived as over another depends not only on its precise spatial location but also on its nature: a coin is conceived of as over a piggy bank only if it is located above the piggy bank’s functional centre–the slot to fit the coin in (Carlson-Radvansky, Covey, & Lattanzi, 1999; Coventry, Prat-Sala, & Richards, 2001). Expectations regarding how objects typically interact or how people interact with two objects are also decisive for lexical choices across a range of spatial relations (see Coventry & Garrod, 2004 for review).
If object functions affect human understanding and verbalization of spatial relationships, this should shape the way humans communicate these relationships to one another–not only when talking about the relationship between two objects, but particularly in multi-object scenarios, which have only rarely been addressed. In dining rooms, chairs are typically arranged facing the table from all sides. This functionally based cultural convention predicts both the chairs’ location and their orientation relative to the table. A few studies are available that use monologue scenarios; these show that such functional relationships between objects indeed affect description strategies (Andonova, Tenbrink, & Coventry, 2010; Ehrich & Koster, 1983; Tenbrink, Coventry, & Andonova, 2011). However, to date no evidence has been brought forward to show how speakers incorporate these phenomena in action-related dialogue.
Functional relationships between objects are part of the interlocutors’ ‘communal common ground’ (shared experiential knowledge within a community: Clark, 1996). Speakers can therefore expect to share such knowledge, without needing to convey it explicitly. Moreover, they may not be consciously aware of possessing this knowledge, and may take it for granted. When faced with describing an arrangement that does not conform to this knowledge, one might expect participants to have difficulty in communication and ultimately with achieving the joint goal of successful arrangement. Specifically, if functional relationships between objects are to be expected, an array involving non-functional relationships (with objects not arranged where they usually are, and not oriented in any predictable way) should lead to communicative problems and/or to a change in dialogue strategies. However, so far no evidence is available to show whether, or in what ways, this is actually the case in a natural dialogue.
Here we introduce a dialogue corpus that allows these issues to be addressed, designed with the following core features that together set it apart from previous data:
Naturalistic action related dialogue. Two ‘naïve’ speakers (no confederate) are confronted with a task involving action, to be solved based on verbal interaction.
Asymmetric knowledge. Each dialogue involves a ‘director’ and a ‘matcher.’ The ‘director’ knows about the specific task goals (here: the target configuration), and informs the ‘matcher’ accordingly.
Everyday spatial concepts in a multi-object arrangement context. The task involves placing dolls’ house furniture in a target arrangement, similar to a range of common tasks in everyday life through evoking relevant action scripts and schemata that guide expectations about object location.
Experimental variation. Furniture arrangements differ with respect to the functional relationship between objects and therefore in their congruence with context frames (Bar, 2004), and hence common ground (prior to communication).
Operationalized success. Correct object placement informs about communicative success.
Task experience. After swapping roles, dialogue partners can build on previous task experience.
To enhance research in the area of cognition and communication, we introduce our German-language ‘Dolldialogue’ corpus (www.dolldialogue.space) towards further use. The transcripts (total: 126,846 words) and pictures showing the furnished dolls’ houses (allowing to evaluate task success) are downloadable by registered users as a free resource.
Here our main goal is to address the following research questions. Firstly, we were interested to what extent and in what ways functional features of object arrangements affect dialogue features such as description length and inclusion of object orientation information, as well as communicative success (measured through successful object arrangement). Speakers are known to refer to atypical visuospatial features more than to typical characteristics of a scenario (Lockridge & Brennan, 2002), and also to be more explicit when they cannot expect information to belong to their common ground (Clark & Brennan, 1991). Since communicating about a functional array allows for relying on shared common ground based on conventionalized schemata or context frames, we could therefore expect that the descriptions used would be shorter, and would contain less specific orientation information compared to descriptions for the non-functional array, where such common ground is not available (Andonova et al., 2010). Considering success, the assumed ease of communication and shared knowledge might lead to more accurate arrangements, or dyads might actually ‘overshoot’ and omit information required for precise placements, leading to errors of commission.
Secondly, we were interested in the ways in which task experience affects dialogue features (description length and orientation information), as well as success.
Speakers are known to adapt to each other over time (Garrod & Anderson, 1987; Giles & Coupland, 1991) and to change their referencing strategies as dialogue unfolds (Clark & Wilkes-Gibbs, 1986), including fewer words for reference; therefore, we expected less information to be explicitly provided in the second arrangement compared to the first. Concerning success, as with all learning, experience should lead to better performance. On the other hand, describing an unusual arrangement incompatible with a previously established discourse strategy may lead to potentially worse performance in the second task despite experience (Garrod & Anderson, 1987). If the functional arrangement is presented first, general experience with object arrangements will initially align with the current task, leading to a conflict in the second arrangement. If the non-functional arrangement is presented first, the experiential conflict coincides with the novelty of the task, before discourse strategies have been developed.
Thirdly, we asked how dialogue features (description length and orientation information) affect success. Since we expected non-functional arrangements to be more challenging than functional arrangements, they should elicit a higher number of words produced to explicate aspects that are not in common ground. We expected that speakers who invest this additional communicative cost should be more successful than those who do not. In a similar vein, we expected the inclusion of object orientation information to be crucial for task success especially in the non-functional situation where this cannot be taken for granted. In the functional situation, the inclusion of information may either be less relevant, or speakers may take too much information for granted.
2 Method
2.1 Stimuli
Two sets of dolls’ house furniture together with two open wooden dolls’ houses from the German toys manufacturer Selecta were purchased and used for this study (Figure 1). One of the houses was fully furnished so as to serve as a model, while the other was empty, with the furniture positioned randomly to the side of the house. Two arrangements were used: in the functional array (F), the rooms could be identified as kitchen, living-room, bedroom, and bathroom; and in the non-functional array (NF), the rooms could not be associated with a specific function, as the furniture was arranged randomly (though generally in a similar setup as in the functional version).
Top left: frontal view on the functional version of the dolls’ house; top right: schematic depiction of the experimental setting; below: model houses for the functional (left) and the non-functional condition (right).
2.2 Participants
Participants were native German speaking students (34 female, 14 male; mean age: 20.03 years; age range: 16–26). They received course credit, were paid for their participation, or participated as part of a university taster event. Parental consent was obtained for participants who were aged 16–17. The participants were assigned in dyads of matched gender and age, and assigned randomly to conditions in order to avoid unintended effects of verbal or cognitive ability, stylistic differences, verbosity, experience, and the like (as time constraints did not allow us to control for such factors explicitly). Moreover, participants were similar in age and sociocultural background. We reasoned that by the age of 16, speakers of a language have mastered the language skills necessary for this kind of verbal task, and familiarity with objects and their functions of the type used here should also be high. 13 dyads started with the functional condition, and 11 dyads started with the non-functional condition.
2.3. Design
The design was a 2 (array type: functional vs. non-functional) x 2 (order of presentation: functional first vs. non-functional first) mixed design, with array type as the repeated measures variable. Additionally, we examined the roles of director and matcher separately.
2.4 Procedure
Testing was in German. To begin with, both participants were led to the empty house without having seen the furnished one. They were allowed to familiarize themselves with the furniture by looking at and touching the objects, but without talking to each other. Afterwards they were positioned facing each other, but separated by a screen so that they could not see each other or the interior of their partner’s dolls’ house. One participant (henceforth called matcher) was placed in front of the empty dolls’ house; the other (henceforth called director) in front of the furnished one (Figure 1). Now they were given their task; the director was to describe the positions of the furniture in their (fully furnished) house in such a way that the matcher could furnish the empty one in exactly the same way (and as accurately as possible). They were encouraged to talk to each other and ask clarification questions, and they were told that the results would be photographed afterwards in order to check and measure accuracy. After the arrangement was completed, the experimenter photographed the resulting furnished dolls’ house and set up the director’s dolls’ house for the other condition (F vs. NF). The participants switched roles (director vs. matcher), and started their second arrangement. Descriptions were audio-recorded and later transcribed (taking several years due to the complexity of the material and several iterations to ensure correction). Each individual arrangement lasted approximately 20 minutes on average.
3 Analysis
We measured communicative task success by correctness of object location (see Figure 2) and orientation. Room errors were given an error score of 3, general location errors 2, specific location errors 1, and orientation errors 1. Objects were considered as oriented incorrectly when their orientation differed from the model by more than 45°. Error coding was undertaken by two independent raters who achieved an intercoder-agreement of 96.77% for all coding choices. A third coder resolved coding disagreements through re-measurement.
Left: schema used for error coding in the functional condition; right: non-functional condition. Circles were assigned according to the target objects’ centre of mass. Inner circles determine the boundaries of correct specific location, and outer circles those for correct general location. When an object was incorrectly placed in a room it was coded as a room error. If the room was correct, and the placed object’s centre of mass (plan view) was situated within its associated inner circle, its specific and general location was annotated as correct. If its centre of mass was between the inner and outer circle, general location was correct but specific location was incorrect. If the object’s centre of mass was outside of both circles, its general and specific location was incorrect.
For language analysis, we computed the number of words and turns (based on speaker change) for each speaker type (describer and matcher) separately. Next, based on iterative annotation we determined for each object placement description whether orientation information was provided. For operationalization, we identified topical units (which could contain several speaker turns) that dealt with the placement of a single object, and examined whether they contained information on object orientation. Transcriptions and annotations were undertaken by assistants blind to the study purposes.
4 Results
The data exhibit a broad range of inter-individual variability in speaking styles, verbosity, and the like. To account for this, our analysis is based on ratios rather than raw scores, leaving further scrutiny of variability to subsequent work based on this rich data resource.
Both the functionality of the arrangement and previous experience affected the dialogue partners’ communicative strategies, leading to a clear pattern of task success. Table 1 provides two example dialogue extracts by dyads who are discussing placement of the same object (the mirror) in each of the two conditions. This object had the same location in both conditions (see Figure 1; B7 in Figure 2). Both dyads negotiated the mirror’s location in some detail in both conditions, but only included reference to its orientation in the non-functional arrangement. The length of the relevant extracts varies; the functional version can be negotiated more efficiently when it comes second. The example extracts contain various further highly interesting negotiation and reference phenomena that lend themselves to further analysis, such as referring back to the previous discourse and experience, using diverse spatial and functional aspects to establish reference, negotiation processes between director and matcher, and so on. Here we focus on the distributional patterns of general quantity and orientation information throughout the corpus in relation to task success.
Example dialogues.
Notes: NF = non-functional array; F = functional array; D = director; M = matcher.
The functional features of the arrangements affected the dialogues quantitatively as predicted, but (throughout the corpus) only for speakers without previous task experience. We looked at description length using two (potentially interrelated) measures: number of words; and number of turns. While the first of these points to general wordiness, the number of turns highlights the interactive dynamics between the speakers. We ran separate 2 (condition: F, NF) x 2 (order: F first, NF first) x 2 (speaker: director, matcher) mixed analyses of variation (ANOVAs) for each measure. For the number of words we found a main effect of condition, F(1, 22) = 12.31, MSE = 125,388,p = 0.002, partial η2 = 0.359, with more words for non-functional (M = 1456 words) compared to functional (M = 1202 words) arrays. There was also a reliable interaction between order and condition, F(1, 22) = 7.65, p = 0.011, partial η2 = 0.258, showing that the effect of array was only present when the NF condition was presented first: 1197 versus 1652 words on average for the F and NF conditions respectively, p = 0.002, Tukey honest significant difference test (see Figure 3).
Number of words produced: interaction between condition and order. Bars represent 95% confidence intervals.
There was also a main effect of director/matcher, F(1, 22) = 191.49, MSE = 135,050, p < 0.00001, partial η2 = 0.897; as expected, directors produced more words (M = 1850) than matchers (M = 808). None of the other interactions were significant (all p > 0.1). The results for the number of turns mirrored those for number of words, with the same pattern of main effects and interactions.
Moreover, the functional features of the arrangements affected communicative success as predicted, particularly for speakers without previous task experience. Overall error scores (weighted as described above) ranged from 0 to 12 in a single arrangement. A 2 (condition) x 2 (order) ANOVA produced a main effect of condition, F(1, 22) = 6.51, MSE = 10.10, p = 0.018, partialη2 = 0.228, with more errors overall in the NF condition (M = 6.626) compared to the F condition (M = 4.276). There was also a reliable interaction between condition and order, F(1, 22) = 20.770, p = 0.0002, partial η2=0.486 (see Figure 4). For both the F and the NF conditions there were fewer errors when the arrangement was the second task compared to the first, but this was only reliable for the NF condition (p < 0.001).
Weighted errors: interaction between condition and order. Bars represent 95% confidence intervals.
The functional features of the arrangements also affected the extent to which speakers provided orientation information for each object arrangement. We analysed the percentage of topical units containing orientation information. There was a main effect of condition, F(1, 22) = 7.617, MSE = 101.43, p = 0.011, partial η2 =0.257, with a higher percentage of orientation information in the NF condition (M = 56.81) compared to the F condition (M = 48.76). The interaction between condition and order was not reliable (p = 0.853).
Finally, we found that description length affected task success, but only in the NF condition. We examined the relationship between success in arrangements and features of the descriptions, focussing on description length by director and matcher, and orientation information. For the success in F, there were no significant correlations with description length or orientation information (all p > 0.15). The success in NF (i.e., fewer errors) correlated with the number of words produced by the director (r = –0.430, p = 0.019), but not reliably with the number of words produced by the matcher (p = 0.145) nor orientation information (p = 0.415). Error scores for F and NF did not correlate with each other (p > 0.05).
5 Discussion
We collected a corpus of unconstrained dialogue data to address how speakers achieve task success in an object placement communication scenario. Speakers were confronted with an unexpected (non-functional) object arrangement either initially, or following task experience using a more natural (functional) scenario. One goal of this paper is to introduce this corpus for wider use by the research community. Its value is evidenced by the clear pattern in our initial analysis results, which highlight how current experience and expectations about the real world combine to shape the dialogue.
Functional features consistently affected task success and dialogue features, particularly in the absence of previous task experience. The functional tasks were generally solved most successfully, and involved fewer misconstruals of object relationships. This confirms our expectations about the vital importance of affordances (Gibson, 1979) and functional spatial relationships (Coventry & Garrod, 2004; Radvansky et al., 2003). Knowledge about the typical arrangement of furniture is part of a schema for object placement that is firmly rooted in human experience (Brewer & Treyens, 1981) and invokes specific context frames (Bar, 2004). In our context, this kind of knowledge serves as general expertise that enhances specific task performance (Weisberg, 2006). Also, the second task was generally more successful than the first, as could be expected based on task-specific experience (Bandura, 1977). If the non-functional arrangement was presented first, the task was solved least successfully, and speakers produced more words and more turns than in the functional condition and more than when the non-functional arrangement came second. Since atypical furniture arrangements did not comply with the shared cognitive schemata (in the cultural background of these speakers: Clark, 1996; Holland & Quinn, 1987), speakers had to resort to providing further explanation to achieve common conversational ground for the task at hand.
With respect to orientation information, the order of presentation did not matter; speakers consistently produced more orientation information with the non-functional arrangement. Clearly, directors tailored their descriptions to the information needs in the different situations, consistent with previous results in other areas (Fussell & Krauss, 1992; Klabunde & Porzel, 1998). Orientation information was provided when it was required, leading to enhanced orientation information in the non-functional condition. To achieve success, directors tended to need more words; therefore success was enhanced for those directors who invested additional effort. However, success in non-functional arrangements was related only to the number of words produced by the director, but not to the amount of orientation information given. It appears that the mere inclusion of orientation information in the negotiation of an object placement did not always lead to the desired task success; further elaboration of the spatial relationship would have been necessary. These issues call for scrutiny on the basis of in-depth qualitative analysis, so as to gain further insights into how communication succeeds and fails in relation to specific configurations and real-world aspects.
Since the error scores for functional and non-functional arrangements did not correlate, our results also suggest that the same dialogue strategy does not necessarily work as well with a different arrangement (Garrod & Anderson, 1987). This observation is relevant particularly for current endeavours to incorporate dialogue strategies in automatic systems (Van Kuppevelt, Dybkjær, & Bernsen, 2005; Walton, 2007), which by their nature tend to be unnaturally rigid. The availability of our extensive dialogue corpus now opens up various avenues for further research in this regard, as it allows for detecting principles and patterns of communication both on a quantitative scale and by closely examining a multitude of highly diverse individual examples. Crucially, the effects of the joint communicative efforts (both generally and specifically for a certain portion of a dialogue) can be directly identified by reference to the pictures taken of the fully furnished dolls’ houses. This will allow for in-depth scrutiny of individual instances of miscommunication as evidenced by erroneous object placement, as well as the identification of dialogue processes that overcome potential misunderstandings and facilitate task success.
Footnotes
Acknowledgements
We thank the participating schools in Delmenhorst and Bremen for their collaboration, and our student assistants for their invaluable help.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was partially supported by a HWK fellowship (Hanse Institute for Advanced Studies, Delmenhorst, Germany) awarded to Kenny Coventry, and by DFG funding for SFB/TR8 Spatial Cognition, I5-[DiaSpace]).