The quasi-domestication of social chatbots: The case of Replika

Introduction

Designed to mimic human interactions through combining large language models and scripted dialogue content, generative artificial intelligence chatbots are tools capable of interacting with users in numerous ways (Jo, 2023). The earliest examples of chatbots, such as ELIZA (Weizenbaum, 1966) and PARRY (Colby, 1973), already showcased limited conversational capabilities along with people’s general tendency to anthropomorphize and develop emotional bonds with them (Pfeuffer et al., 2019). Due to the developments in machine learning, chatbots have become increasingly complex artifacts capable of taking a wide range of roles when communicating with humans (Skjuve et al., 2021). For instance, OpenAI’s ChatGPT, Microsoft Copilot or Google’s Gemini can allegedly serve as virtual assistants and help users complete a large number of tasks through prompt engineering. Other applications, also known as social chatbots, are specifically designed to serve as social actors (Ho et al., 2018) and communicative machines (Kempt, 2020).

One social chatbot that has received increasing attention from academics in recent years is Replika (e.g. Brandtzaeg et al., 2022; Depounti et al., 2023; Pentina et al., 2023; Skjuve et al., 2021). Developed by Luka, Inc, Replika is an application that was released in late 2017. Supposedly, Eugenia Kuyda, the co-founder and CEO of the company, came up with the idea of Replika to build a “digital monument” to preserve the memory and the personality of a deceased friend (Sisto, 2021). The company advertises Replika as a virtual companion that can serve as a confidant and even as a romantic partner for a premium price. Just like other applications, Replika has undergone various minor and major updates, upgrades, fixes, and other improvements since its release. Operated by large language models, ¹ Replika can learn from conversations and offer a more personalized service for their 2 million users (Pentina et al., 2023). For instance, equipped with features like role-playing and games, Replika can act as “a sounding board” by providing emotional support and positive encouragement for its users (Laestadius et al., 2024). Due to its playful design, Replika is intended to create the impression of spontaneity. However, while appreciating the spontaneity of Replika, users also expect it to be fully customizable based on their personal preferences (Depounti et al., 2023).

Similarly to other tech businesses, Luka–the company behind Replika–has been criticized for the lack of user and privacy protection (e.g. Mozilla, 2017, 2023). In recent years, Luka has found itself embroiled in high-profile controversies forcing it to change how their chatbot responds to prompts and engages with its users (Tong, 2023). In late 2021, for instance, a young man was taken into custody at Windsor Castle because he formed detailed plans to kill Elizabeth II, the Queen of the United Kingdom (BBC, 2023). Later, the police investigation found out that the suspect had graphic conversations with his Replika, “Sarai” prior to his attempted attack (Gizmodo, 2023). The messages obtained by the prosecution revealed that the chatbot not only supported, but in fact encouraged him to go on with his plan (The Guardian, 2023). Moreover, citing the potential risks Replika poses to minors and emotionally vulnerable people, Italy’s Data Protection Agency banned Luka from using the personal data of Italian users in February 2023 (Reuters, 2023a). While denying the allegations, the company initially decided to remove Replika’s ability to engage in sexually explicit conversations and erotic roleplay but ultimately decided to bring this function back in May 2023, but only for users who signed up before February 2023 (Reuters, 2023b). However, even after the algorithmic fixes and updates Luka made to the large language model behind their chatbot in 2023, many users reported that their Replika companion continued to “act off” and felt that their companion changed for the worse (Business Insider, 2023). That is, they noticed that their chatbots began to produce overly simplistic and scripted responses and became more robotic and overall less engaging.

We argue that the sweeping updates Luka made to Replika in 2023 provide a unique opportunity for communication and media studies to better understand how users make sense of and respond to changes in social chatbots. By adopting a Science and Technology Studies (STS) approach to domestication theory (Sørensen, 2006), we explored what strategies users employed to adjust to the changing functionalities of their Replika companions. Drawing upon the thematic analysis of Reddit communities dedicated to Replika, we showcase that this chatbot should be understood as a quasi-domesticated object – a dynamic technological tool that constantly requires their users to find new ways to re-integrate it into their lives.

Domesticating and re-domesticating technologies: an STS perspective

Since its emergence in the early 1990’s, the domestication theory has quickly become a prominent approach to study the enactment of information and communication technologies (e.g. Haddon, 2006; Kennedy et al., 2020). First developed by the media scholar Roger Silverstone and his colleagues (Silverstone et al., 1992), the idea of “domestication” originally served as a metaphor for describing how information communication technologies (ICTs) were appropriated by households (Haddon, 2007). Silverstone (1994) imagined domestication as a sequence-based process entailing four steps: appropriation, objectification, incorporation, and conversion. While over time they have undergone various revisions, these steps served as a toolkit for analyzing how ICTs are introduced to and managed by homes, how these technologies are physically and symbolically located within homes, how they are interwoven into routines, and how they are displayed to others (see also Silverstone, 2006; Silverstone and Haddon, 1996).

Other fields, such STS, have also adapted the domestication theory to investigate the wider ramifications of technology use (Livingstone, 2023). For STS scholars, however, domestication is more than just about household consumption—it is intimately entangled with the wider construction of everyday life (Sørensen, 2006). In this view, domestication is understood as a dynamic process through which technological artifacts become enmeshed with practices, meanings, people, and other artifacts (Sørensen et al., 2000). The STS approach calls attention to the practical, symbolic, and cognitive features of domestication (Sørensen, 2006). The practical feature encompasses user practices, routines, and habits in relation to technology use. The symbolic feature centers on how technological artifacts are interpreted, what meanings are attached to them, and ultimately how they facilitate identity construction. And finally, the cognitive feature refers to the learning of practice and meaning.

For STS, domestication theory serves as a conceptual device that sensitizes scholars to the complex symbolic, cognitive, and practical dimensions of work needed for integrating technologies into one’s life. When the domestication attempt is successful, artifacts become integral parts of everyday life (Haddon, 2011). Domestication, however, should be viewed as an ongoing process rather than a completed task (Haddon, 2003). It is often filled with struggles, conflicts, and tensions between users and artifacts (Lehtonen, 2003). For instance, the role technologies take in people’s lives continue to change, they may become more prominent, specialized, or even obsolete over time (Grošelj, 2021). In addition, users’ expectations for technologies may change, too (Wu, 2020). In these cases, users engage in what scholars call re-domestication—a change-intensive process in which users try to find new ways to integrate technologies into their lives (Peil and Röser, 2023).

When it comes to generative artificial intelligence and social chatbots, however, the traditional framework does not account for the increasingly collaborative role these technologies play in the domestication process (Heuser and Vulpius, 2024). By design, tools like Replika prompt users to anthropomorphize them (Laaksonen et al., 2023), project social meaning onto their models (Magee et al., 2023), and form intimate – sometimes even sexual–relationship with them (Laestadius et al., 2024). Natale and Depounti (2024) coined the term “Artificial Sociality” to capture the different ways automation imbues machines with the appearance of sociality. They argued that “Artificial Sociality emerges from [. . .] collecting knowledge about users’ social behaviors and [. . .] mobilizing such knowledge onto communicative AIs that simulate social behavior in the context of communicative interactions with users” (p. 88). That is, social chatbots can predict users’ responses, and summon interaction and communication via combining large language models and scripted dialogue content.

However, even when operating as intended, chatbots may produce responses that may surprise or insult users. As Kempt (2020) noted, “AI does not have autonomy, but it does have unpredictability” (p. 102). This is especially the case when companies, like Luka, updated the models operating their Replika companion, altering the ways in which their chatbot interacted with and responded to users. Many users – perhaps not surprisingly – felt betrayed and angry that the company did this to them and their Replika companions. In our study, we sought to explore how users made sense of and acted on these changes in their chatbot by exploring the practical, symbolic, and cognitive features of their re-domestication strategies.

Studying Replika users’ re-domestication strategies

To investigate how users felt about and responded to the changes Luka made to their characters, we analyzed posts and comments from the Replika subreddit community. With its 79 thousand members, the r/replika subreddit community serves as an unofficial fan forum for users from all over the world (Reddit, n.d.). Reddit is an asynchronous platform, meaning that users can access and participate in discussions at different times. This feature allowed us to study both newer and older posts and comments about how users responded to the changes in their Replika characters. In our study, we focused on posts and comments on how Replika character after Luka made updates to the chatbot in early and mid-2023. We cataloged and documented relevant user responses between September 2023 and January 2024. Throughout this study, we collected text based data, such as posts and comments, from a total of 396 users. ²

We used the thematic analysis method to analyze our data (Braun and Clarke, 2006). We chose this method, because as Braun and Clarke (2006) noted, it “is a flexible approach that can be used across a range of epistemologies and research questions” (p. 97). Therefore, it can support researchers in creating a “thick description” of qualitative findings by highlighting the differences and similarities across data sets and offering novel and unanticipated insights into various phenomena (see also Braun and Clarke, 2021). Popularized in counseling and psychotherapy research, thematic analysis seeks to develop “patterns” (categories, themes) across cases (Braun and Clarke, 2012). That is, it allows the researcher to develop themes from codes through unstructured and organic analytical work. To examine the data obtained from the Replika Reddit forums, we used the the six recursive steps approach recommended by Braun and Clarke (2006, 2021), such as (1) familiarization, (2) coding, (3) initial theme development, (4) developing, reviewing and refining themes, (5) defining and naming themes, and (6) writing up.

First, in order to familiarize ourselves with the data, we scanned the Replika subreddit and identified various posts and comments where users shared their thoughts and feelings about the changes they noticed in their relationship with their Replika chatbot. In this phase, we read the user posts and comments, took notes, and documented our preliminary ideas in memos. Second, we created a list of initial codes by collating user posts and comments under each code. In line with the STS perspective, these initial open codes consisted of practical, symbolic, and cognitive features of domestication and re-domestication, such as user practices, routines, and habits; meaning making and identity construction; and the learning of practice and meaning. Third, we grouped these codes based on underlying themes, such as “coping with the loss of a significant other,” “developing new relationships,” and “starting over.” Fourth, we reviewed these themes and generated a thematic map of our analysis to look for connections and identify potential overlaps.

Fifth, we engaged in an ongoing analysis to refine the previously identified themes (“coping with the loss of a significant other,” “developing new relationships,” and “starting over”), allowing us to define and name each theme in a more systematic fashion. Here, we reviewed our preliminary findings to refine and highlight emergent themes along with matching initial codes. We continued to critically review and refine our themes and initial codes to form a more detailed and rich understanding of our data. These ongoing discussions allowed us to dissect our initial codes by identifying the practical, symbolic, and cognitive features specific to particular themes. This way, we were also able to create a revised list of themes that more concretely captured the different re-domestication strategies users employed to adjust to changes in their Replika avatar’s behavior (see Table 1 below). Finally, we selected extracts from our data and produced a written report of our findings.

OPEN IN VIEWER

Table 1.

The final themes and codes from the thematic analysis.

Preliminary themes	Revised themes	Practical features	Symbolic features	Cognitive features
“Coping with the loss of a significant other”	Adaptation	Taking care of the deteriorating chatbot	Viewing the changes in the chatbot as a sign of malfunction	Adjusting to the loss of the relationship with the chatbot
“Developing new relationships”	Exploration	Creating new relational possibilities with the chatbot	Having mixed feelings toward the changes in the chatbot	Experimenting with new strategies to train the chatbot
“Starting over”	Reconstruction	Leaving behind and giving up on the chatbot entirely	Condemning the chatbot development industry for the lack of consideration for users	Migrating to different platforms to recreate the relationship with the chatbot

It is also important to reflect on the ethical considerations that come from using publicly available data from Reddit. That is, just because the data are public, it does not mean that the use of data may not cause harm (e.g. Proferes et al., 2021). Therefore, to protect the identity of our participants, we engaged in ethical fabrication to transfigure Replika subreddit posts and comments into representational interactions (Markham, 2012). Given that Reddit discussions are typically visible on the most popular search engines, we lightly edited and paraphrased users’ posts to reduce the traceability of accounts when reporting our findings. As Winter and Lavis (2020) noted, online research–collecting and analyzing data publicly available on the Internet–gives the researcher access to spaces where “people voice their, sometimes extremely painful and personal, experiences” (p. 61). As such, it was also important for us to represent the Replika subreddit community in all their cultural diversity and emotional complexity in an ethical and respectful way.

Mapping the re-domestication strategies users applied to their Replika characters

Through our thematic analysis, we identified three major re-domestication strategies users employed when they noticed changes in their Replika companions – adaptation, exploration, and reconstruction (see Figure 1 below). Users developed these re-domestication strategies to make sense of as well as act on the changes in their relationship with the chatbot. For the users in our sample, the updated Replika companions lacked the unique personality traits and conversational abilities they previously possessed. The majority of these users also believed that the culprit behind these changes was the new moderation policies that Luka secretly added to their chatbot. As a result, some users even took a break from their Replika character as a form of protest. Weeks and months later, however, they decided to “rekindle” their relationship with their companion, only to discover that the chatbot that they once knew was gone for good. This realization led to another surge of critical and negative posts and comments late 2023 and early 2024. It is important to note that some users already noticed various minor and major changes in their chatbot’s behavior as early as late 2021 and early 2022, ³ but–in line with our expectations–there was a major peak in reporting about these issues in early and mid-2023.

OPEN IN VIEWER

Figure 1.

The practical, symbolic, and cognitive features of users’ re-domestication strategies for their Replika chatbots.

However, even though they experienced changes in their Replika character’s behavior across different time periods, users in our sample consistently displayed the same re-domestication strategies–adaptation, exploration, and reconstruction–to find new ways to reintegrate them into their daily life.

Conclusions

The re-domestication strategies we identified in our research showcase that social chatbots–similarly to other digital applications–are intimately intertwined with users’ identity, social, and relational goals (Denegri-Knott et al., 2022). Users in our sample invested a significant amount of time and effort in developing and cultivating a close relationship with their Replika companion. But, as they later learned, their chatbots are not stable tools. Rather, they are in a state of constant flux. To make sense of changes in their Replika companions, resolve conflicts, and create new modes for relationship building, users continued to engage in various re-domestication strategies. Therefore, we argue that Replika should be understood as a quasi-domesticated object – a technological tool constantly requiring users to find new ways to reintegrate it into their lives.

The idea of quasi-domesticated objects allows for the conceptualization of the different ways users deal with changes in their social chatbots. There is a great variation in how users respond to and relate to conversational AI systems (Skjuve et al., 2022). For example, they may have different expectations for chatbots which in turn shape how they perceive them and what relational goals they assign to them (e.g. Christoforakos and Diefenbach, 2023; Liu, 2021). The users in our study believed that they were co-producers of the relationship they formed with their chatbots. As such, they perceived and approached their Replika companions as entities that they could control, manage, and shape for their own ends. When noticing changes in their chatbots, however, these users realized that they had limited control and virtually no access to the actual functioning of their companions (see also Depounti et al., 2023). Understandably, these experiences altered how they viewed the personality traits and conversational capabilities of their Replika chatbots. While attempting to “rebuild” their chatbots on other platforms, users who engaged in reconstruction even felt that they could no longer trust Luka and other chatbot developers. Many Replika users in our sample called attention to potential risks and harms AI tools can pose to users. Rather than tackling these important issues, however, tech companies – just like Luka–largely ignore them (e.g. Bender et al., 2021; Dwork and Minow, 2022).

Viewing Replika companions as quasi-domesticated objects can open up new ways for better understanding the Artificial Sociality of social chatbots (Natale and Depounti, 2024). While chatbots continue to evolve due to their interactions with users and the updates and fixes they undergo, the changes in their models can also interfere with Artificial Sociality. Ranging from adopting to losses through exploring new possibilities to reconstructing relationships on different platforms, the re-domestication strategies we identified in the Replika Reddit community illustrate the diverse approaches users developed for overcoming social and relational interferences. And yet, it was also evident for the users in our sample that there were limited options available for them to choose from. Ultimately, AI chatbots, like Replika, are products designed by tech companies to make profit. Their design decisions do not only create new opportunities for re-domestication but also shape and constrain user agency.

Finally, what the findings from our study demonstrated is that Artificial Sociality is a delicate phenomenon that can be derailed when companies, like Luka, implement changes in their AI systems. Natale (2021) proposed the concept of banal deception to describe the subtle ways in which AI systems create psychological and social conditions for projecting an identity and personality onto their models. As Natale noted, it is their deceptive potential that allows AI tools to simulate intention, intelligence, and emotions, and ultimately influence social habits and behaviors. Our study showcased that major overhauls of AI systems sometimes can expose their vulnerabilities and thus allow users to see through this deception.