Abstract
Could a subtle shift in the language used by chatbots improve service interactions? This research suggests that a chatbot’s use of interjections (e.g., “wow” and “hmm”), can shape consumer attitudes and behaviors. Four experiments demonstrate that consumers are more satisfied, more willing to purchase, and more likely to remain loyal when chatbots use interjections. The studies find support for a sequential mediation model where interjections increase anthropomorphism, which in-turn increases perceived listening and then impacts consumer responses. Consumers respond positively to interjections because they feel the chatbot is more human-like, and thus capable of attending to them and understanding their needs. This interjection effect holds across a range of consumer contexts and at different stages of the purchasing process. The findings shed light on why anthropomorphism has a positive effect in human-chatbot interactions, while providing a straightforward means of enhancing customer satisfaction, purchase intent, and loyalty.
Introduction
Conversational technology (chatbots, voice assistants, social robots; henceforth: chatbots) is improving (Tiku et al., 2023). Advances in chatbot technology include the speed at which they learn and adapt based on previous conversations (OpenAI, 2022) and their ability to detect user sentiment (Genest et al., 2022; Singh & Sharma, 2022). With these increased capabilities, researchers predict that chatbots will be able to perform more complex tasks. For example, Huang and Rust (2018) suggest that chatbot tasks will evolve over time through four stages: mechanical (e.g., greeting customers) → analytical (e.g., tax filing preparation) → intuitive (e.g., reporting the news) → empathetic (e.g., providing social care). This predicted progress toward more complex, empathetic tasks is echoed by others. Clavel et al. (2022) describe the shift as being from “task” orientation toward “task + social orientation.”
However, if we want chatbots to assume human-like service roles, they need to interact in human-like ways (Go & Sundar, 2019). Hence the literature focuses on anthropomorphism via visual (Zogaj et al., 2023), identity (Jeon, 2022), and language (Kull et al., 2021) cues which enhance perceived humanness. Although anthropomorphism may have negative effects in specific situations (as indicated by Ciechanowski et al., 2019; Crolic et al., 2022; MacDorman & Ishiguro, 2006), meta-analysis indicates that overall, anthropomorphism in human-machine interactions is a net positive (Blut et al., 2021). Considering the positive impacts of anthropomorphism, it is worth investigating the particular human-like cues within the chatbot that serve as precursors to a user’s anthropomorphic perceptions.
This study introduces interjections as a means to increase anthropomorphic perceptions of chatbots. While many cues may increase anthropomorphic perceptions, we feel interjections are valuable in the context of text-based chatbots, where visual anthropomorphic cues (facial expressions, movement, vocal cues, etc.) are unavailable. Interjections are “words that constitute utterances by themselves and express a speaker’s current mental state or reaction” (Ameka & Wilkins, 2006, p. 2). They are very brief, discrete expressions of cognition or affect (Scherer, 1994). They exist in all languages, although the meaning of each interjection may differ between cultures. Examples include “oh,” “wow,” “hmm,” “uh huh,” or “aww.” Consider the following scenario: a chatbot may say “That’s cute.” Conversely, using an interjection, a chatbot may say “Aww. That’s cute.” This small shift in language may seem trivial. However, this research demonstrates that the use of interjections in human-chatbot interactions, has significant impacts on anthropomorphism, perceived listening, and subsequent consumer evaluations.
Furthermore, while anthropomorphic cues typically improve human-chatbot interactions, the mechanism to explain these positive effects remains unclear. The human-robot interaction literature offers some clues. Two socio-communicative accounts posit that robot anthropomorphism is beneficial due to increased rapport-with or trust-in a robot. Rapport in a human-robot context refers to feelings of a personal connection between the user and robot (Wirtz et al., 2018). Through anthropomorphism, people may feel a robot is more sociable, and thus a suitable target for emotional bonding and rapport (Qiu et al., 2020), and this increased rapport is what drives positive consumer outcomes. A second potential explanation is trust. Trust is the expectation that others will act reliably, keep their promises and tell the truth (Ooi & Tan, 2016). Some studies support trust as an explanation for anthropomorphism’s positive effects (Waytz et al., 2014), while others find no support (Hancock et al., 2011). Recently, a meta-analysis by Blut et al. (2021) found that rapport with—and trust in a robot failed to effectively explain the positive effect of anthropomorphism in human-robot interactions. In sum, explanations for the positive effects of anthropomorphism in human-chatbot interaction remain unexplored, while the explanations in human-robot interaction remain unconvincing. This study aims to test perceived listening as a potential explanation for why human-like chatbots typically outperform machine-like chatbots.
This research makes several theoretical contributions. First, the research adds to a body of works demonstrating that a small change in language, can have a significant impact on consumer attitudes and behavior (Packard & Berger, 2021; Packard et al., 2018; You et al., 2020). Second, this research is the first to examine interjection use by chatbots, demonstrating that they increase anthropomorphism, perceived listening and subsequent consumer evaluations. Third, while the existing literature suggests anthropomorphic cues are generally positive, this research suggests a novel explanation for why these positive effects occur. We demonstrate that increases in anthropomorphism drive increases in perceived listening. This advances research in human-machine communication, but is particularly useful to the study of mediated communication—where access to visual listening cues such as facial expressions and body language may not be available. The research also offers a clear practical contribution. To improve human-chatbot interactions, programmers should add interjections to their chatbots. This is likely to have a large impact, given chatbot use is predicted to grow rapidly (Tiku et al., 2023).
Literature Review
Given the majority of chatbots are commercial in nature, we contextualized our study into human-chatbot communication around customer service scenarios. Therefore, our dependent variables are customer satisfaction, that is, a consumer’s overall evaluation of the chatbot (Johnson & Fornell, 1991), purchase intent—the likelihood of making an initial purchase (Mittal & Kamakura, 2001) and loyalty—the likelihood of making a repeat purchase (Szymanski & Henard, 2001). These variables are important as they predict firm performance (Otto et al., 2020) and even consumer consumption at the country level (Yeung et al., 2013).
Interjections
Interjections are short, independent utterances that convey a speaker’s current thoughts and feelings (Ameka, 1992). Examples include wow (I’m surprised, amazed, or in admiration), uh huh (I understand or agree), hmm (I’m thinking or hesitant), and aww (I feel empathy, pity, or sentimentality). This definition has two parts. First, interjections are grammatically independent. Second, interjections convey information about the speaker’s thoughts and feelings. Several papers support the first proposition; interjections are independent. For example, Schröder (2003) found that presenting audio recordings of interjections in isolation (no words on either side of the interjection), without context or instruction, was sufficient to convey identifiable meaning. Other papers support the second proposition; interjections convey the speaker’s thoughts and feelings. For example, researchers have mapped specific interjections onto specific cognitive and affective states (Goddard, 2014). Hmm (consideration) and uh-huh (I understand) are demonstrative of cognition, while wow (surprise) or aww (empathy) signify affective states.
According to this definition, interjections may be words (e.g., damn) or non-words (e.g., aww). They often involve the production of sounds not found in other parts of the language (e.g., tut-tut) and may not include any vowels (e.g., psst, Ameka, 1992). As such, interjections are described as atypical with regards to phonology (referring to sound patterns) and morphology (referring to the formation and structure of words; Ameka & Wilkins, 2006). In online communication, interjections have been described as a form of textual para-language, conceptually related to emoji, excessive punctuation for emphasis, for example, “surprise!!!,” or the deliberate misspelling of words to suggest tempo, for example, “slooowww” (Luangrath et al., 2017). The use of interjections in face-to-face communication typically involves physical movements and body language (Ameka & Wilkins, 2006). Furthermore, the sound of interjections often mimics a physical action or reaction, such as gasping, gagging, sighing, which makes interjections somewhat like onomatopoeia (Libert, 2019). There are a variety of ways to categorize interjections (Ameka & Wilkins, 2006; Goddard, 2014) and they are thought to be very versatile. We believe any situation-relevant interjection, contributes to our hypothesized effect, thus we do not differentiate between types of interjection in these studies. A full explanation is provided in the hypothesis development section. Interjections can be used as an exclamation (ouch, wow), a greeting (hi), as part of a response to someone else (huh? oh!) or to indicate indecision (um) or deep thought (hmm). Norrick (2009) explains that interjections may function as a back-channel, discourse marker, attention getter, floor-holder, transition, to indicate conversational repair or to signal disapproval.
Anthropomorphism
Anthropomorphism is the attribution of human-like traits to non-human objects. The dominant three-factor model of anthropomorphism suggests it occurs due to effectance motivation (need to control one’s environment), sociality motivation (need for social connection) and elicited agent knowledge (EAK; object specific cues relative to preconceived ideas about what an object can or should do; Epley et al., 2007). Both effectance and sociality motivation reside in the user’s mind, while elicited agent knowledge is driven by human-like chatbot cues such as an avatar, message content or adherence to norms such as turn-taking and management of miscommunication (Go & Sundar, 2019). As such, anthropomorphism occurs as the union of the observer and the observed. The variable often features in human-chatbot interaction studies (Araujo, 2018; Go & Sundar, 2019; Sheehan et al., 2020), where researchers manipulate chatbot presentation or behavior to impact EAK. These studies typically report the positive effects of anthropomorphism. However, anthropomorphism is known to backfire in certain situations—such as where an AI threatens a user’s sense of personal agency (Alsaad, 2023) or triggers feelings of eeriness (Ciechanowski et al., 2019). In general, meta-analysis suggests anthropomorphism has a net positive effect (Blut et al., 2021).
Two recent advances in the field of anthropomorphism research are of note. The first suggests an inverse of anthropomorphism, that is, machine-like cues trigger machine-like attributions of personality. Hence laypeople often describe chatbots as accurate and objective, yet unyielding and transactional (TIME model cues; Sundar, 2020; Sundar et al., 2015). These perceptions may impede a chatbots ability to perform more complex social tasks as described in Huang and Rust’s (2018) or Clavel et al.’s (2022) chatbot development roadmaps. For example, according to Huang and Rust’s (2018) typology, present-day customer service chatbots have surpassed the mechanical intelligence stage (e.g., providing scripted responses) and the analytical intelligence stage (e.g., using customer data to personalize an interaction). They may even display intuitive intelligence–recognizing patterns, and connecting seemingly unrelated information. The latest chatbots are indeed capable of identifying user intent from limited input, with less need for clarification, in non-linear conversations. With intuitive intelligence a chatbot may be able to identify the hidden cause of a customer’s complaint. However, without emotional intelligence—that is, the ability to empathize—a chatbot may be unable to make a customer “feel heard,” and calm them down before the situation escalates. Empathetic chatbots might modify their tone based on understanding user sentiment or mirror emotions to make users feel understood and emotionally supported. Thus intuitive versus empathetic chatbots are likely to excel in different roles, based on differences in their abilities and user goals—intuitive intelligence for efficient task resolution, empathetic intelligence for satisfaction, and emotional wellbeing. In the second anthropomorphism research advance, Kühne and Peter (2023), argue for a more fine-grained study of anthropomorphism. They suggest that anthropomorphism is not just perceptions of human-likeness—but rather a user’s cognitive evaluation of an object as having mental faculties or abilities, that is, a mind. Using this logic, researchers should try to move beyond simply measuring and describing an object as human-like and attempt to identify how human-likeness triggers the attribution of specific mental faculties or abilities. In this context, that is the ability to listen and understand.
Perceived Listening
Listening comprises of a range of actions and processes including attentiveness, verbal and non-verbal cues, attitudes, memory, and behavioral responses (Lewis & Reinsch, 1988). Put more simply, listening is attending to and understanding what someone says (Steil et al., 1983). In this way, listening and empathy share some conceptual overlap (Drollinger et al., 2006; Steil et al., 1983). Despite conflicting definitions and operationalizations of listening, researchers agree about its effects. Listening is fundamental to interpersonal relationships and their success. It is studied in the context of marriages (Pasupathi et al., 1999), doctor-patient relationships (Fassaert et al., 2007), and student learning environments (Vandergrift, 2007).
A number of studies link perceived listening with customer satisfaction (de Ruyter & Wetzels, 2000; Drollinger & Comer, 2013; Packard & Berger, 2021). This includes data from sales and marketing (Aggarwal et al., 2005; Itani et al., 2019; Marshall et al., 2003), customer service call centers (de Ruyter & Wetzels, 2000), and healthcare (Wanzer et al., 2004). Most relevant are studies that manipulate a linguistic construct to demonstrate an indirect effect on satisfaction via perceived listening in a service context. Packard and Berger (2021, studies 3–5) demonstrate that using concrete (“Those blue jeans are great”) versus abstract (“Those pants are great”) language, increases perceived listening, which subsequently increases customer satisfaction. The authors argue that the use of concrete language “generates inferences that they (employees) are attending to and understanding the topics raised by a conversational partner” (Packard & Berger, 2021, p. 800).
Hypothesis Development
To reiterate, the goals of our study are to test perceived listening as a theoretical mechanism for the positive effects of anthropomorphism as reported in much of the human-chatbot interaction literature. While doing so, we aim to test a novel means of increasing anthropomorphic perceptions–interjections. As such, we propose that interjections increase anthropomorphism, which in turn increases perceived listening, which in turn has a positive impact on consumer outcomes (satisfaction, purchase intent, and loyalty). The logic for our model is best described in terms of the Computers’ as Social Actors Paradigm (CASA; Moon & Nass, 1996). Additional support comes from Social Presence Theory (SPT; Short et al., 1976) and Media Richness Theory (MRT; Daft & Lengel, 1986).
CASA states that humans may apply social rules to computers, when computers display social attributes. CASA encompasses the attribution of human-like traits (anthropomorphism), but goes beyond anthropomorphism in explaining that based on this cognitive bias, users may apply social rules (such as saying please and thank you; Nass et al., 1999) and norms (such as reciprocity; Fogg & Nass, 1997) to their interactions with computers. The degree to which users’ responses are conscious versus unconscious has been examined, with some describing it as mindless (E. J. Lee, 2010), and others as mindless anthropomorphism (Kim & Sundar, 2012). CASA supports the logic of our model as follows. First, to account for the predicted relationship between interjections and anthropomorphism, interjections mimic human conversational behavior. They are found in all languages (Ameka, 1992; Goddard, 2014) and are used by all age groups, developing early in childhood (Montes, 1999). Therefore, when a chatbot uses interjections, it appears more humanlike. Users evaluate the chatbots use of interjections against their own memories of interpersonal communication via elicited agent knowledge and anthropomorphic perceptions increase.
Once users perceive the chatbot as more anthropomorphic due to interjections, they are more likely to attribute other humanlike capabilities to the chatbot, that is, listening. People assume other social actors listen in a conversation. Thus, when a chatbot is more anthropomorphic, it is more likely to be perceived as—and capable of—listening, given human-like cues inform user expectations (Go & Sundar, 2019). Furthermore, using a situationally appropriate interjection at an appropriate time would signify listening in interpersonal communication. Imagine a colleague asked your opinion on a serious matter. Using the interjection “hmm,” before responding may indicate that you were listening, and that you understand that this serious matter deserves a considered response on your part. The same effect should occur in human-chatbot interaction.
Of course, anthropomorphism is not the only cause of listening. People may describe a smart speaker with no anthropomorphic cues as capable of listening. But listening is about event perception and shared meaning—beyond simply hearing sounds (Gaver, 1993). A smart speaker that plays a love-song when asked, may be listening in the conventional sense. But a smart speaker which responds with “aww,” when asked to play a love song, may be perceived as more anthropomorphic and thus more capable of listening in the academic sense, due to perceptions of shared meaning and context. As such, we believe that listening can serve as a socio-communicative explanation for the positive effects of anthropomorphism in human-chatbot interaction. We propose that anthropomorphism increases perceived listening.
We feel that listening is more appropriate explanation for the positive effects of chatbot anthropomorphism, than rapport-building and trust, which come from the human-robot interaction literature. Both rapport building and trust have provided mixed results. This is because, unlike robots, a chatbot’s primary function is communication—and listening is fundamental to communication. In the simplest communication models, with senders encoding a message and receivers decoding a message, listening (i.e., understanding meaning) comprises half of communication. Meanwhile, robots have many non-communicative functions. From our perspective, rapport and trust may occur after extended periods of interaction, but listening is a more proximal explanation for anthropomorphisms positive effects.
Finally, we propose that perceived listening improves consumer outcomes such as customer satisfaction, purchase intent and loyalty in human-chatbot interactions. The positive effects of listening are well established in human-human interaction studies. Thus, it is reasonable to assume that a sufficiently human-like chatbot which appeared to be listening (understanding and attending to a user’s needs) would also perform well.
Additional support for our model comes from Social Presence Theory (SPT; Short et al., 1976) and Media Richness Theory (MRT; Daft & Lengel, 1986). SPT considers the salience of the other party in a communication medium. Text-based chat is low in social presence. It has moderate immediacy—in that responses are typically faster than email, but slower than face to face communication, but low intimacy. Facial cues, eye-contact, feelings of physical proximity are not available. Conceptualizations of SPT include “being real” and “being there” (Cui et al., 2013). Our interjections → anthropomorphism → listening relationship impacts “being real” by increasing anthropomorphism. It also impacts “being there,” as interjections are immediate, spontaneous expressions of current thoughts and feelings, which may reduce psychological distance between text-chat communicators. As social presence improves through this process—so should consumer responses to a chatbot using interjections. According to SPT, effective communication is considered as interpersonal involvement—relative to the constraints of the medium used (Biocca et al., 2003). We believe interjections help to overcome the “being real” and “being there” constraints inherent to online text-chat.
Similarly, MRT is a framework to evaluate the richness of information that different communicative mediums can convey. For example, a video call is richer than a phone call (Daft & Lengel, 1986). MRT states that richer media are more effective. Text-chat would be considered a lean medium, richer than bulk email, but leaner than a two-way radio conversation. Given the risk of miscommunication increases in lean mediums, interjections—which signify listening, could be characterized as a means of increasing richness. The more recent media naturalness theory (Kock, 2004) asserts that face-to-face is the most natural form of communication, thus other mediums should attempt to replicate its features. Given interjections in face-to-face communication typically involve physical movements and body language (Ameka & Wilkins, 2006) and the sound of interjections often mimics a physical action or reaction, for example, gasping, gagging, sighing (Libert, 2019), interjections may function as textual representations of non-verbal cues—increasing the naturalness of text-chat. In other words, using an interjection may allow a message receiver to imagine a corresponding facial expression or physical reaction. This would increase anthropomorphism and because humanlike entities are assumed to be capable of thought and understanding, this would indicate listening. Given the logic described, we hypothesize relationships for the model as shown in Figure 1.
Model of relationships tested.
H1: Chatbots using (vs. not using) interjections will result in more positive consumer outcomes (satisfaction, purchase intent, and loyalty).
H2: Chatbots using (vs. not using) interjections are perceived as more anthropomorphic, which increases perceived listening and subsequent positive consumer outcomes.
Please note, we believe that interjections → anthropomorphism → listening → consumer outcomes is more appropriate than interjections → listening → anthropomorphism → consumer outcomes. This is because the listening → satisfaction relationship is robust and well established, while anthropomorphism is known to backfire and reduce satisfaction under certain conditions. Furthermore, recent conceptualizations of anthropomorphism (Kühne & Peter, 2023) make it clear that agent cues (interjections) are precursors of user cognition (anthropomorphism), which then informs attribution (agent morality, personality or the capacity to listen). Thus, interjection use is an antecedent of anthropomorphism, while perceived listening is a consequence. Finally, while interjections may differ in terms of morphology, semantics, or valence, we propose that any context-relevant interjection should increase perceived listening. If a customer was complaining about receiving expired milk, we predict that a response beginning with “eww” still signifies listening, despite its negative valence. As such, we do not differentiate between different types of interjections in this study. However, given our context of customer service, we try to focus on the interjections most likely to be used in customer service scenarios.
Overview of Studies
Four experiments are presented. Study 1 demonstrates that chatbots using (vs. not using) interjections are more satisfying for consumers. Study 2 extends upon these findings, replicating the positive effects of interjections in a different consumption scenario, where participants have a genuine chatbot interaction. Studies 3A and 3B test a serial mediation model, exploring whether interjections have a positive impact on consumer outcomes (purchase intent and loyalty), due to interjections increasing anthropomorphism, with a subsequent positive effect on perceived listening. An overview of each study, with results is presented in Table 1.
Overview of Experiments.
p < .05, **p < .01, ***p < .001.
Note. All variables except the DV in Studies 3A and 3B were measured on seven-point Likert scales. The DV in Studies 3A and 3B was binary choice. DV = dependent variable; M = mediating variable.
Study 1
Study 1 was a two-condition, between-group experiment designed to investigate whether consumers satisfaction with a chatbot increases when a chatbot uses interjections. Sixty-four Americans recruited via Prolific (57.1% female, Mage = 33.89, SD = 12.73) were asked to watch a short animation of a human-chatbot interaction. Data from 1 participant was removed as they failed an attention check question designed to test that they had paid attention to the animation.
Stimulus Materials
The stimulus consisted of an animation of a consumer-chatbot interaction. This method has been used before in chatbot experiments manipulating language (Sheehan et al., 2020). The animation showed a consumer attempting to make a restaurant reservation for a special occasion. Two versions of the animation were prepared; one in which the chatbot uses interjections and one in which the chatbot does not. For example, when the consumer says the reservation is for two people, the employee responds with “Let me see if I can fit you in” versus “Hmm. Let me see if I can fit you in.” A full transcript of the interaction for both conditions as well as screen-shots are provided as Supplemental Material.
The animations were high in ecological validity. They were designed to replicate an interaction occurring on Facebook Messenger. The restaurant was given a fictitious name with a custom logo. The animation was set inside a wireframe image of a mobile phone to add realism. Care was taken to ensure that all aspects of the animations were identical, except for the experimental manipulation of the presence or absence of interjections. For example, the pauses between the consumer-chatbot responses did not change between conditions. At the end of each animation was a 4-digit code, used as an attention check question to confirm participants had watched the entire animation. The animation was 1.5 min long.
Procedure
Participants were randomly assigned to one of the two conditions and began by reading the study instructions. They were told that “companies use chatbots to provide customer service” and that their task was to watch a customer-chatbot interaction via live-chat and then evaluate the chatbot’s performance. Participants then watched the animation. Finally, the participants responded to the survey items, measuring how satisfied they would be as the consumer and their demographic characteristics.
Measures
Satisfaction was measured using a single item, “I would be satisfied with the chatbot’s responses.” Scores were recorded on a seven-point Likert scale, anchored at strongly (dis)agree. A single item dependent measure was considered appropriate in this initial pilot study, given that research suggests there is no difference in the predictive validity of single versus multiple-item measures of concrete, singular constructs (Bergkvist & Rossiter, 2007). Demographic variables included age, gender, income, and education.
Results
First, the relationships between the demographic variables and satisfaction were examined. There was no relationship. Furthermore, demographic variables did not predict the dependent variables or mediating variables across any of the studies. As such, the demographic variables are excluded from further analysis.
An independent-samples t-test was conducted to compare satisfaction scores between the two conditions. There was a significant difference in mean scores between the interjections (n = 31, M = 5.48, SD = 1.20) and control conditions (n = 32, M = 3.66, SD = 1.65, t(61) = 4.98, p < .001, Cohen’s d = 1.45). These results suggest that interjections have a significant, large effect upon satisfaction. Specifically, the data suggests that chatbots which use interjections are more satisfying. Thus, Study 1 supports H1.
Study 1 provides preliminary evidence to suggest that interjections increase customer satisfaction with chatbots. However, there were several limitations with this study. As a pilot study, it used a small sample size, a single item dependent measure and did not test theoretical accounts for the effect of interjections. Furthermore, the stimuli involved a third party interacting with the chatbot, as opposed to the participants themselves. Therefore, a second experiment was prepared.
Study 2
Study 2 was a two-condition, between-group experiment, that aimed to replicate and extend upon the previous study. As such, several changes were made to the experimental protocol. First, participants in the previous study watched an animation of someone else interacting with a chatbot. They were asked to imagine they were the customer when providing a customer satisfaction score. A more realistic design involves participants having a conversation with a chatbot themselves and then rating their own perceptions of the chatbot. In this study, participants (124 Americans from Prolific; 62.4% female, Mage = 30.01, SD = 8.95; seven cases removed for failing an attention check) were asked to have a genuine instant messaging conversation with a chatbot. Several other, smaller changes were also made to enhance rigor and generalizability. These included (a) changing the consumer context, from a restaurant booking in the previous study, to ordering flowers for delivery in this study, and (b) changing from a single-item dependent measure, appropriate for preliminary studies, to multiple scale items for each construct in this study.
Stimulus
Participants were randomly assigned to the conditions and asked to hold a scripted conversation with one of two purpose-built chatbots. The chatbots used (vs. did not use) interjections. They were designed to assist consumers to order flowers for Mother’s Day via instant messaging. The chatbots represented a fictitious firm and were embedded in a separate website, independent of the data collection software. The chatbots were designed to greet the human user and respond to user input. They were built using an online platform, FlowXO, which does not require coding experience, making replication straightforward.
In order to eliminate extraneous variables, the participants were asked to use a script. Sheehan et al. (2020) developed this procedure to address methodological limitations within the chatbot literature. In prior research, participants have been asked to have open-ended conversations with chatbots and then evaluate them. However, this reduces internal validity and makes replication extremely difficult. This is because each participant has a different conversation with the chatbot. Where one participant may state, “What flowers do you recommend?,” another participant may ask, “I like roses. Can I get those?” Conversely, a chatbot may response with “Hmm. I recommend roses” or “Hmm. Yes, roses would make a great choice.” While both responses include the manipulated variable (hmm), they differ in terms of length and content. Thus, scripting of the conversation in this study is considered a strength of the design. Scripted interactions ensure that all participants have the same experience, which is appropriate for theory testing, where researchers try to simplify and isolate complex phenomena (Lin et al., 2021).
As per Study 1, the chatbots in each condition were identical, except for the presence (vs. absence) of interjections. For example, at one point in the conversation, the participants gave the chatbot a personal message to write inside a card, sent with the flowers. The non-interjection chatbot replied, “Your message will be included.” In contrast, the interjection chatbot replied, “Aww. Your message will be included.” Before running the study, we had 10 laypeople assess the conversation script for realism. Full details of the script are provided as Supplemental Material.
Procedure
The participants were randomly assigned to one of the two conditions. They were provided with an explanation of what a chatbot is and told that they were to help in the training of a new chatbot. The idea was to replicate the feel of a mystery shopping exercise. Participants were asked to stick to the script provided, to ensure the training session was successful. Next, the participants were given some task-related information, regarding their hypothetical need to purchase flowers. Participants then clicked a link to access the chatbot. Participants were asked to enter their scripted line, observe the chatbot’s response and then enter the next scripted line, until the chatbot advised them that the conversation was over. At the end of the conversation, the participants were presented with a 4-digit invoice number for the flower order. The invoice number was entered into the survey software to confirm that the participant had successfully completed the interaction. Following the chatbot conversation, the participants responded to survey items measuring satisfaction and demographic variables (age, gender, income, and education).
Measures
Satisfaction (α = .944): The participants completed a three-item measure of satisfaction. The items were, “I would be satisfied with the chatbot’s responses,” “I would be satisfied with the chatbot’s communication,” and “The chatbot was good at its job.” The items were modified versions from prior research (Packard & Berger, 2021). Responses to all items, were taken on a seven-point Likert scale, anchored at strongly (dis)agree.
Results
An independent-samples t-test was conducted to compare satisfaction between the interjections and control conditions. There was a significant difference in satisfaction scores (Minterjections = 5.80, SD = 0.55 vs. Mcontrol = 5.36, SD = 1.17; t(115) = 2.61, p < .01, Cohen’s d = 0.92). These results suggest that interjections have a significant effect upon satisfaction. Thus, Study 2 further supports H1. Using a second consumer scenario (restaurant booking vs. flower delivery), this study supports the claim that interjection use increases customer satisfaction with chatbots. Further studies were developed to examine our sequential mediation hypothesis.
Study 3A and 3B
Study 3A and 3B were both two-condition, between-group experiments. Several changes were made to the experimental protocol. The previous studies featured service delivery in which the customer got what they wanted. They were able to book the restaurant table for their anniversary. They were able to order the flowers for delivery on time and under budget. However, perfect outcomes are not always possible. These studies were designed to examine the role of interjections in service delivery when the customer can only achieve a sub-optimal outcome. This was done for several reasons. First, it increases the generalizability of the findings. Second, it is theoretically interesting, because a sub-optimal outcome may impact perceived listening scores. Consumers may feel less heard when they cannot achieve their desired outcome. Third, from a methodological perspective, using a sub-optimal context, reduces the likelihood of a ceiling effect in the data. Ceiling effects, also known as scale attenuation effects occur when a high proportion of participants have the maximum possible scores for a particular variable (Salkind, 2010).
A major aim of these studies was to test our serial mediation account for the effect of interjections in human-chatbot interactions. Studies 3A and 3B test whether interjections increase anthropomorphism, which then increases perceived listening, which in turn has a positive impact on consequential dependent variables such as purchase intent and customer loyalty. Finally, we wanted to test whether interjections were sufficient to have an impact on consequential consumer choice. Therefore, our dependent variables were binary measures of purchase intent (Study 3A) and customer loyalty (Study 3B).
Study 3A
Stimuli and Procedure
Participants from Prolific (168 Americans, 72.2% female, Mage = 36.47, SD = 14.70; three cases removed for failing attention check) were randomly assigned to one of two conditions and asked to hold a scripted conversation with a purpose-built chatbot. The chatbots used (vs. did not use) interjections. The chatbots were designed to answer users’ questions on behalf of the fictitious Waterford Hotel. The participants were told that they were looking for a romantic hotel trip to celebrate their wedding anniversary. They asked the chatbot if they could book one of the rooms with the best view (the Lakeview rooms). The chatbot informed them that those rooms were sold-out. However, there were still other rooms with a partial view of the lake available. The dependent measure reflects one’s willingness to book the room despite the sub-optimal view, that is, purchase intent. The procedure was identical to that used in Study 2. A full transcript for the stimulus is provided as Supplemental Material.
Measures
Dependent measure: This study uses a consequential, binary choice dependent measure. The dependent variable was measured as follows; “Given the rooms with the best lake view are sold out; would you reserve a room with a partial view of the lake?” This binary choice reflects consumers’ willingness to accept a sub-optimal outcome. There were two response options: Yes and No.
Anthropomorphism (α = .958): This variable measured the degree to which participants felt the chatbot was humanlike. Five semantic differential items were taken from the Godspeed Questionnaire (Bartneck et al., 2009). They were, (a) fake–natural, (b) machinelike–humanlike, (c) artificial–lifelike, (d) unconscious–conscious, and (e) communicates inelegantly–communicates elegantly. Responses were recorded on a seven-point scale.
Perceived listening (α = .861): This variable measured the degree to which participants felt the chatbot was attending to and understanding them. Three items to measure perceived listening were taken from Packard and Berger (2021). They were, “The chatbot gave me personal attention,” “The chatbot understood my specific needs,” and “The chatbot was listening to me.” Responses to all items were taken on a seven-point Likert scale, anchored at strongly (dis)agree.
Results
The dependent variable was coded such that a participant’s acceptance of the sub-optimal outcome was coded as 1, while not accepting the sub-optimal outcome was coded as 0. A logistic regression was conducted, with the use of interjections as the independent variable and acceptance of the sub-optimal outcome as the dependent variable. Results revealed a significant increase in the probability of accepting the sub-optimal outcome, when interjections were used (No interjections = 71.60% vs. Interjections = 84.52%; B = 0.77, SE = .390; χ2(1) = 4.07, p = .047). Next, an independent-samples t-test was conducted to compare the perceived listening and anthropomorphism scores between the interjections and control conditions. There was a significant difference in mean scores for both variables. The presence of interjections appears to have increased perceived listening scores (Minterjections = 5.57, SD = 1.15 vs. Mcontrol = 4.74, SD = 1.19; t(163) = 4.49, p < .001, Cohen’s d = 1.17) and anthropomorphism scores (Minterjections = 4.69, SD = 1.64 vs. Mcontrol = 3.54, SD = 1.55; t(163) = 4.62, p < .001, Cohen’s d = 1.59). These results suggest that interjections have a significant effect upon consumer choice, further supporting H1.
Mediation Analysis
Next, serial mediation was run using PROCESS Model 6 (Hayes, 2017) to test our interjections → anthropomorphism → listening → purchase intent predictions. The results are shown in Figure 2. Interjections predict anthropomorphism (b = 1.15, BootSE = 0.24, 95% CI [0.66, 1.64]), anthropomorphism predicts listening (b = 0.45, BootSE = 0.04, 95% CI [0.36, 0.54]), and listening predicts purchase intent (b = 0.45, BootSE = 0.20, 95% CI [0.04, 0.85]). The indirect effect of interjections → anthropomorphism → listening → purchase intent was significant (b = 0.23, BootSE = 0.14, 95% CI [0.02, 0.56]). The data supports our serial mediation hypothesis (H2), given the 95% confidence intervals for the indirect effect does not span zero. Please note, the direct effects were non-significant (b = 0.14, SE = 0.44, 95% CI [−0.71, 1.01]), as were the indirect effects for each mediator on their own (interjections → anthropomorphism → purchase intent; b = 0.36, BootSE = 0.21, 95% CI [−0.01, 0.85]) and (interjections → listening → purchase intent; b = 0.13, BootSE = 0.11, 95% CI [−0.01, 0.43]).
Results of serial mediation analysis (Study 3A and 3B).
As additional analysis, we ran serial mediation with the position of anthropomorphism and listening reversed (i.e., interjections → listening → anthropomorphism → purchase intent). The total indirect effects of this model were non-significant (b = 0.22, BootSE = 0.14, 95% CI [−0.005, 0.55]). The mediation path between anthropomorphism and purchase intent was non-significant (b = 0.32, BootSE = 0.16, 95% CI [−0.01, 0.65]). Some may reasonably suggest that people ascribe the capacity for listening to devices without anthropomorphic cues (e.g., a voice recorder) and thus, anthropomorphism could be treated as a moderating versus mediating variable—increasing the strength of interjections on listening. To investigate this alternative model, Process Model 7 was used. The results of this analysis show that interjections do not predict listening (b = 0.26, SE = 0.40, p = .50), anthropomorphism does predict listening (b = 0.44, SE = 0.14, p < .01), however the interaction effect is non-significant (b = 0.00, SE = 0.09, p = .93). Perceived listening continues to predict purchase intent (b = 0.67, SE = 0.16, p < .01). Because the interaction between interjections and anthropomorphism is non-significant, the confidence intervals for the index of moderated mediation is also non-significant, Index = 0.005, BootSE = 0.076, BootLLCI = −0.164, BootULCI = 0.132.
Study 3B
Stimuli, Procedure, and Measures
Study 3B (n = 170 Americans—9 removed, 51.2% male, Mage = 36.53, SD = 11.55) used the same, two-group experimental setup as Study 3A. However, the context for the experimental stimuli was changed (from a sub-optimal hotel booking to a sub-optimal warranty claim) as was the dependent measure (from purchase intent to customer loyalty) to increase generalizability. Participants were told that they had purchased a laptop. It was still under warranty, but had been damaged in an accident. Participants chatted with a customer support chatbot, who asks questions to determine how the accident happened. Concluding the conversation, the chatbot explains that the damage is not covered under warranty. The dependent variable was measured as follows; “Given you cannot have the laptop fixed or replaced free of charge, would you still shop with this company?” There were two response options: Yes and No. The dependent measure reflects one’s willingness to remain a customer, despite having a warranty claim rejected, that is, customer loyalty. Measurement of perceived listening and anthropomorphism was as described in the previous study. A full transcript for the stimulus is provided as Supplemental Material.
Results
The dependent variable was coded such that a participant’s decision to remain a customer, despite the sub-optimal outcome, was coded as 1, while deciding to discontinue patronage was coded 0. A logistic regression was conducted, interjection use as the independent variable and choice to remain a customer as the dependent variable. Results revealed a significant increase in the probability of retaining a customer, when interjections were used (No interjections = 39.76% vs. Interjections = 61.54%; B = 0.89, SE = .323; χ2(1) = 7.69, p = .006). An independent samples t-tests was conducted to compare perceived listening and anthropomorphism scores between the two conditions. There was a significant difference in mean scores for both variables. Perceived listening scores were higher in the interjection condition (Minterjections = 4.99, SD = 1.46 vs. Mcontrol = 4.48, SD = 1.33; t(159) = 2.30, p = .01, Cohen’s d = 1.40) as were anthropomorphism scores (Minterjections = 4.67, SD = 1.47 vs. Mcontrol = 4.00, SD = 1.75; t(159) = 2.61, p < .01, Cohen’s d = 1.62).
Mediation Analysis
Next, serial mediation analysis was performed. The same pattern was observed as in Study 3A. The results are shown in Figure 2. Interjections predict anthropomorphism (b = 0.66, BootSE = 0.25, 95% CI [0.16, 1.17]), anthropomorphism predicts listening (b = 0.50, BootSE = 0.05, 95% CI [0.39, 0.61]), and listening predicts purchase intent (b = 0.67, BootSE = 0.18, 95% CI [0.32, 1.03]). The indirect effect of interjections → anthropomorphism → listening → purchase intent was significant (b = 0.23, BootSE = 0.11, 95% CI [0.05, 0.48]). The data supports our serial mediation hypothesis (H2), given the 95% confidence intervals for the indirect effects do not span zero. Please note, the direct effects were non-significant (b = 0.59, SE = 0.37, 95% CI [−0.13, 1.32]), as were the indirect effects for each mediator on their own (interjections → anthropomorphism → loyalty; b = 0.16, BootSE = 0.11, 95% CI [−0.01, 0.42]) and (interjections → listening → loyalty; b = 0.11, BootSE = 0.13, 95% CI [−0.15, 0.40]).
As additional analysis, we ran serial mediation with the position of anthropomorphism and listening reversed (i.e., interjections → listening → anthropomorphism → purchase intent). The total indirect effects of this model were non-significant (b = 0.24, BootSE = 0.14, 95% CI [−0.03, 0.53]). Again, it was the mediation path between anthropomorphism and loyalty which was non-significant (b = 0.08, BootSE = 0.07, 95% CI [−0.010, 0.263]). To again examine the potential for anthropomorphism as a moderating (vs. mediating) variable in the relationship between interjections and listening, Process Model 7 was used. As per Study 3B, the results show that interjections do not predict listening (b = 0.45, SE = 0.53, p = .39), anthropomorphism does predict listening (b = 0.59, SE = 0.16, p < .01), however, the interaction effect is non-significant (b = −0.06, SE = 0.11, p = .56). Perceived listening continues to predict purchase intent (b = 0.84, SE = 0.16, p < .01). Because the interaction between interjections and anthropomorphism is non-significant, the confidence intervals for the index of moderated mediation is also non-significant, Index = −0.054, BootSE = 0.108, BootLLCI = −0.280, BootULCI = 0.154.
Discussion
This research proposes and then demonstrates that the use of interjections by a chatbot increases customer satisfaction scores, purchase intent and loyalty. Study 1 shows the effect of interjections upon satisfaction. Study 2 replicates the effect, with genuine human-chatbot interaction. Studies 3A and 3B suggest the effect is strong enough to impact consequential, binary consumer choices such as purchase intent and switching intent in a sub-optimal scenario. Most importantly, Studies 3A and 3B shed light on the processing mechanism, that is, anthropomorphism → perceived listening. The effect of interjections appears to occur across a range of consumer scenarios. Those tested include purchasing flowers, restaurant bookings, hotel inquiries, and warranty claims on technology products. Furthermore, the effect appears to occur at different stages of the purchasing process, including information search and post-purchase evaluation. Finally, the effect appears to occur consistently, irrespective of a consumer’s age, gender, education, or income.
Theoretical Contribution
This research demonstrates the importance of perceived listening in human-machine communication. In the context of Huang and Rust’s (2018) chatbot task typology, we believe our study provides preliminary evidence to suggest that perceived listening can bridge the gap between chatbots performing third stage (intuitive) work and fourth stage (empathetic) work. Intuitive work requires a chatbot to adjust its approach to novel situations, where empathetic work requires a chatbot to recognize and respond appropriately to user emotions (Huang & Rust, 2018). Matching chatbot performance to user goals remains fundamental. For example, if customers want efficient task resolution, empathy may not be required. However, increasing the ability to replicate empathy increases the scope of work that chatbots may perform. While a chatbot may never understand emotions in the human sense, situationally appropriate interjections and perceived listening may give the impression that it does. Just as humans stereotype each other, machinelike cues trigger stereotyping of machinelike personalities (Sundar, 2020; Sundar et al., 2015). Thus, consumers often perceive chatbots as logical, systematic, and somewhat impersonal. This makes chatbots well-suited for analytical tasks. However, chatbots that present as humanlike, thus capable of active listening, may excel in handling more intricate and socially-oriented tasks. In the language of Clavel et al. (2022), this research suggests listening can shift chatbots along the “task”–“task + social” continuum. Furthermore, by examining why anthropomorphism has (net) positive effects in human-chatbot interaction, our paper attempts to shift the chatbot discussion from how to maximize anthropomorphism, toward a more fine-grained examination of why to maximize anthropomorphism. Clearly, anthropomorphism will remain fundamental to the study of human-machine communication. However, our studies suggest the importance of understanding the psychological consequences of anthropomorphism (attribution of capabilities, Go & Sundar, 2019; Kühne & Peter, 2023)—that precede the commercial consequences of anthropomorphism (improved consumer attitudes and behavior).
Our studies are a step toward answering an important, yet open question in the literature—why does anthropomorphism have a positive impact in human-chatbot interactions. In line with CASA and SPT we provide evidence to support perceived listening as a socio-communicative explanation for the positive effects of anthropomorphism. Of course, there are still other plausible explanations from other academic disciplines. For example, anthropomorphism may increase familiarity via elicited agent knowledge, which may then increase perceived ease of use. However, given the defining feature of chatbots is their ability to communicate, we believe this communication-based explanation is valuable. Furthermore, we believe it captures the unique differences of human-chatbot versus human-robot interaction where rapport-building and trust have been examined as processing mechanisms. Again, chatbots are communicative—that is their defining feature. Thus, a communication-based explanation for the positive effects of chatbot anthropomorphism is valuable. Meanwhile, social robots—which may communicate, also perform a range of non-communicative tasks. Finally, we believe our explanation of listening (vs. rapport building and trust) is a more proximal explanation. In other words, listening is required before rapport building or trust with a non-human agent can occur.
Drawing on MRT, our demonstration of interjections as sufficient to influence perceived listening is also useful. This is because many traditional listening cues are unavailable in lean communication mediums. For example, in face-to-face interactions, message receivers can provide many non-verbal listening cues—facing the speaker, maintaining eye contact, head nodding, appropriate facial expressions, etc. However, in a live chat, with a human or a chatbot, these visual cues are not present, making it difficult for message senders to know if their message was received and understood. Interjections may help to overcome limitations inherent to the text-chat medium—increasing naturalness à la media naturalness theory. Comparing interjections to other mediated listening cues such as paraphrasing (restating someone’s ideas back to them), highlights the strength of interjections in live-text chat. Paraphrasing is used to make people feel heard (Jones, 2011; Min et al., 2015; Van Hasselt et al., 2006). While paraphrasing and interjections can both be used in mediated communication (telephone, email, live-chat with a human or chatbot), interjections are more efficient and immediate because typing a 3-letter interjection is faster than paraphrasing whole sentences. This is important, given response delays in instant messaging may have negative consequences (Hwang et al., 2019).
We did not test interjection use in human-human text-chat. However, we may be able to draw some tentative, preliminary findings from our results, given modern chatbots may be indistinguishable from human interlocutors (Biever, 2023). For example, in customer service interactions, it may be that a human employee’s use of interjections decreases the likelihood of a customer abusing an employee. It is well known that customer service staff often face abuse and dehumanization (Korczynski & Evans, 2013; Terskova & Agadullina, 2019). Dehumanization is thought to be the opposite of anthropomorphism (Schroeder & Epley, 2016; Waytz et al., 2014). Given that interjections increase anthropomorphism, using interjections may decrease dehumanization and thus, employee abuse. This may be particularly useful with the rise in off-shore, outsourced customer service (Lu et al., 2020; Thelen et al., 2011), given dehumanization typically occurs along racial and ethnic lines (Haslam & Stratemeyer, 2016). Further examination is warranted.
Practical Contribution
Feeling heard appears to have consequences in both human-human and human-machine interactions. Consumers’ desire to feel heard will likely increase over time, as remote-working and online shopping may increase social isolation and decrease face-to-face interaction. Interjections can help in this regard, making a cold, impersonal, computer-mediated interaction feel humanlike. Programmers are advised to add interjections to text-based chatbot programming. This should be possible with both simple rule-based chatbots and AI chatbots using natural language processing. Implementing this change is likely to have a large impact given 48% of consumers world-wide have used a live-chat for customer service and 65% of consumers feel comfortable handling issues without a human involved (Bazilian, 2017). Furthermore, due to labor shortages, chatbots are being used to fill the live-chat supply gap (Microsoft, 2020). Meanwhile firms are driven to deploy chatbots because they dramatically reduce phone call and email volumes (Moore, 2018), offering a return with little effort (Srinivasan et al., 2018). The data suggests that interjections improve human-chatbot service interactions. However, firms looking to adopt a more fine-grained approach may build systems which use interjections more liberally for customers who use interjections themselves or display other personality traits. This is possible. A number of papers explain how chatbots can detect user personality and sentiment and then reflect or mirror those traits back to consumers to improve satisfaction (D. Lee et al., 2017; Shumanov & Johnson, 2021). Finally, the positive benefits of interjections, anthropomorphism, and perceived listening appear to enhance a consumer’s experience, whether they receive their desired outcome or not.
Future Research and Limitations
The most compelling avenue for future research is a test of interjections in face-to-face service delivery. However, in face-to-face interactions, employee authenticity will be important. For example, saying “Wow,” with congruent facial expressions is likely to be satisfying, however saying “Wow,” with a sullen expression and apathetic posture, is likely to have a backfire effect, being perceived as rude or sarcastic. Testing interjections in face-to-face interactions would open new lines of theoretical enquiry, while providing significant practical benefit. Second, interested parties are encouraged to examine interjection use in populations outside of the United States. Interjections are universal (Ameka, 1992; Libert, 2019). However different cultures have different communication styles and linguistic preferences (Rau et al., 2009; Sanchez-Burks et al., 2003). Testing the model with additional samples could increase generalizability. Third, this research examined the effect of interjections in text-based interactions. Testing the effect of interjections with audible chatbots (Siri, Alexa, etc.) could prove fruitful. Fourth, this research presents the first attempt to link interjections with perceived listening and consumer attitudes. We propose that situationally-relevant interjections increase perceived listening—regardless of interjection type. As such, we used multiple types of interjections within our stimuli. Future research may wish to test the effect of different types of interjection, test whether a single interjection is sufficient to have an impact, or if there are differential effects of interjections when used at the start versus end of a sentence. Fifth, others may wish to replicate the effects presented, while controlling participants preexisting ideas about chatbot capabilities. Replication in the future would also rule out novelty effects.
Beyond these four avenues for future research, new studies could examine moderators or boundary conditions for the relationships presented here. Three moderators of interest are; (a) task complexity and risk, (b) the frequency of interjection use, and (c) interjection valence. First, it may be that task complexity and risk interact with interjection use. For example, the present research examined interjection use in low-risk consumption scenarios. However, it would be interesting to study how interjections perform in high-risk scenarios such as health diagnoses or scenarios high in formality, such as financial or legal services. Using interjections in high-risk, complex scenarios may reduce psychological distance between the two interlocutors (Trope & Liberman, 2010). However, interjection use may backfire, decreasing the perceived professionalism of the interjection user. Furthermore, this research tested interjections in optimal and sub-optimal conditions. However, interjections may also backfire when the consumer receives a very poor outcome—in other words, a worse than sub-optimal outcome. The second potential moderator is interjection frequency. Researchers may also which investigate if too many interjections have a negative effect on consumer attitudes. Finally, interjection valence. Interjections can have negative emotional valence for example, “yuk.” It is unclear how an interjection such as yuk would be perceived by consumers. Used at the correct time, it might increase anthropomorphism and perceived listening as yuk may be a contextually appropriate response.
Supplemental Material
sj-docx-1-crx-10.1177_00936502241273259 – Supplemental material for Wow! Interjections Improve Chatbot Performance: The Mediating Role of Anthropomorphism and Perceived Listening
Supplemental material, sj-docx-1-crx-10.1177_00936502241273259 for Wow! Interjections Improve Chatbot Performance: The Mediating Role of Anthropomorphism and Perceived Listening by Ben Sheehan, Hyun Seung Jin, Brett Martin and Hyoje Jay Kim in Communication Research
Footnotes
Author Contributions
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
