From Poacher to Protector of Attention: The Therapeutic Turn of Persuasive Technology and Ethics of a Smartphone Habit-breaking Application

Introduction

“[They] call me both someone who runs a game reserve and is a poacher,” says Tim. ¹ I am sitting in a café in Venice Beach, Los Angeles, speaking to Tim, who is a behavioral scientist and software developer (what I refer to as a “persuasive technologist”) and senior executive of behavioral software company Boundless Mind. ² Tim is responding to a question I put to him about a possible conflict between Boundless Mind’s popular behavioral software products. One of the products is an Application Programming Interface (API) for third-party businesses to optimize the capture of end user attention. The API accesses behavioral data from third-party commercial software (e.g., a medication adherence application) and applies machine learning and behavioral design techniques to enhance the persuasiveness of interactive features. The other is an application called Space for end users to make third-party applications less persuasive. In effect, these products, mean Boundless Mind both “poach” and “protect” ³ end user attention, which has attracted notable mainstream media attention (Parkin 2019; Shieber 2017). However, Tim does not believe that Boundless Mind’s products are in conflict with each other. From a legal perspective, he has a point: the responsibilities of persuasive technologists like Boundless Mind have yet to be formalized into professional codes of conduct ⁴ and largely exist as guidelines in design books (Eyal 2014; Fogg 2001) or blog posts (Harris 2016), or arguments by philosophers and legal scholars (Dorrestijn and Verbeek 2013; Frischmann and Selinger 2018; Verbeek 2009). Yet Tim does not even believe that his company is acting unethically or in bad faith, instead justifying their software products as a response to the variety of end user interest in behavioral change and evidence that Boundless Mind is “defending [the] cognitive liberty [of end users].” On the surface Tim’s claim that his company defends cognitive liberty appears to be the kind of empty self-posturing that is typical of a spokesperson. However, as I explore in this paper, Tim’s comment reflects a perception of Boundless Mind, and persuasive technologists more broadly, that they are enablers of an ethical or good life. Boundless Mind’s credentials were affirmed in 2019 when the company was acquired by corporate and consumer well-being conglomerate Thrive Global. Thrive Global has raised over 50 million USD in funding (Crunchbase n.d.) and has the goal to be a leader of behavior change. In a statement about acquiring Boundless Mind, Thrive Global founder Ariana Huffington (2019) stated: “We’ll never be able to change behavior at scale if we don’t use all of the tools in our toolkit, including the most cutting-edge technology.”

In this paper I critically assess the ethical perspective of Boundless Mind. I depart from prevailing ethical inquires of persuasive technologies and technologists that offer valuable insights but tend to focus on hypothetical thought experiments, how the persuasive technology is put to use, or the robustness of the design process. Humanist critiques of persuasive technologies focus on the potency of the manipulation, claiming a line when the design becomes too persuasive or coercive, and the subsequent threat to human agency and autonomy (Frischmann and Selinger 2018; Gray et al. 2018; Williams 2018) ⁵ . Instrumentalist critiques focus on how the persuasive technology is put to use (Berdichevsky and Neuenschwander 1999) and tend to come from persuasive technologists themselves. For example, popular persuasive technologists B.J. Fogg (2001) and Nir Eyal (2014) claim activities such as gambling or any product/service that harms the environment or animals are unethical applications of their persuasive technologies. Post-phenomenologists claim persuasive technologists act ethically if they demonstrably anticipate and factor in a range of usages and effects of their inventions into the design process (Verbeek 2006, 2011). From a humanist, instrumentalist or post-phenomenological perspective, Boundless Mind appears to act ethically. In a self-published design manual titled Digital Behavioral Design (2018), Boundless Mind explicitly detail their methods of persuasion and their process of undertaking background checks of every commercial client before providing the API.

The ethical question that Boundless Mind raises relates more to their oxymoronic mix of software products and Tim’s confidence that they act in the interests of end users. In other words, what is unclear is how Boundless Mind come to determine their practices as ethical. An ethical approach that is situated in the perspectives and actions of practitioners comes from anthropology and science and technology studies (STS). Empirical investigations into how ethics develop in practice in relation to technological innovations and wider political and economic contexts is known as an “ethigraphy” (Lynch 2001, 3) and the emphasis on how people evaluate the ethical dimensions of their own actions as “ordinary ethics” (Lambek 2010). The ordinary ethics approach is particularly useful to trace the development and implications of ethical reasoning in sectors or disciplines that lack conceptual or institutional unity around ethical practices (Metcalf, Moss, and boyd 2019). For example, Ziewitz (2019) observes that the ethics of undertaking search engine optimization for consultants is also a struggle for authority given the line between “good” optimization and “gaming the system” is unstable and unclear. Other studies have sought to understand how practitioners reconcile their personal ethics in relation to negative perceptions of the industries that they work in (Martin 2006; Sismondo 2015). Tønnesen (2009) offers the term “ethicalizing” to denote when ethical solutions and problems are constructed in practice and into business solutions. Persuasive technology is a relatively nascent practice with contentious ethics that has attracted mainstream media attention and scrutiny, particular in relation to smartphone “addiction” (Leslie 2016). The belief from Boundless Mind that their products enable an ethical life presents an opportunity to learn how practitioners determine the ethics of their practice. Therefore, I ask: how do Boundless Mind come to view the poaching and protecting of end user attention as ethical? What are the implications of Boundless Mind’s ethics for persuasive technology more broadly?

My approach to answering this question draws upon the work of Michel Foucault. ⁶ Toward the end of his scholarship, Foucault offered an approach to ethics that departed from the Kantian tradition of moral philosophy or normative conceptions of right or wrong. Foucault (1990) criticized Immanuel Kant’s construction of moral philosophy and ethics, contending that any conception of morality—such as Enlightenment humanism—was already shaped by power and therefore inherently problematic. Foucault refused to acknowledge an a priori or ahistorical conception of the subject and considered ethics to be a self-directed exercise of resistance to power or an attempt to be less governed. As the subject was already determined by power, Foucault considered ethics to be an “aesthetic of existence” exercise in developing a relation of self to itself in terms of its moral agency. To demonstrate how individuals could conduct themselves ethically or resist power, Foucault (1988) identified Ancient Hellenic practices that he called “technologies of the self,” such as self-writing, to enlarge the possibilities of self-determination within the existing matrix of power relations.

Nikolas Rose (1998, 1999) observed the influence of scientific practice on Foucauldian ethics and analyzed the rise of 20th century “psy” (psychology, psychiatry, and psychoanalysis) disciplines into “therapeutic authorities.” By offering unprecedented insights into human behavior and interior states or subjectivities, Rose contended that the psy disciplines came to view the human subject as “fixable” and their expertise as giving them the authority to administer therapeutic solutions to correct behaviors that they had constructed as abnormal (Miller and Rose 2008). By selling software products that claim to shape the behavior of end users across conflicting vectors, Boundless Mind is a therapeutic authority of persuasive technology. The therapeutic tool that I analyze in this paper is Boundless Mind’s technology-habit-breaking smartphone application Space. My analysis of Space uses the walkthrough method (Light, Burgess, and Duguay 2016), accompanied by an interview with Boundless Mind Senior Executive Tim conducted in 2018. The walkthrough method derives from science and technology studies (STS) and cultural studies, and seeks to understand how a software product is intended to be used. I walkthrough Space to identify what software protocols and persuasive techniques are embedded into the app to guide user behavior.

The rest of this paper has three sections. In the second section I outline the expertise of persuasive technologists. I trace the development of persuasive technology by key figures such as B.J. Fogg and Nir Eyal, as well as Boundless Mind. By identifying subjectivities in relation to user behavior and encoding such knowledge into models and software, I argue that Fogg, Eyal, and Boundless Mind are experts in “mobilizing” (Miller and Rose 2008) the end user without overtly manipulating them. In the third section, I walk through the app Space to demonstrate how Boundless Mind put their expertise about user subjectivities to therapeutic ends. In particular, I highlight a behavioral technique embedded in Space that enables the user to develop a transformative relationship with their technology habits and persuasive applications. In a neoliberal society where individuals are encouraged to pursue well-being for personal fulfillment, the expertise of Boundless Mind places them in an ideal position to offer the therapeutic cure to persuasion. I conclude the paper by arguing that an ethical perspective based on fixing the user obfuscates the power of persuasive technologists by limiting the scope of ethical inquiry to the activities of the user.

The Mobilization of the End User

Persuasive technologies utilize behavioral science to non-coercively shape the actions of end users to a predetermined goal (Verbeek 2009) and have become commonplace in digital media technologies and formalized in various disciplines. Perhaps the most established field using persuasive technologies is “nudges” or choice architecture (Thaler and Sunstein 2009), which is a subfield of behavioral economics that uses persuasion to enhance public policy outcomes in Western democracies. Another popular application of persuasion is “gamification,” the practice of embedding game elements in non-game contexts (McGonigal 2011). In the 2010s, a popular technology criticism was that social media platforms were persuasive by design, enabling internet addiction or compulsive use of digital media technologies (Alter 2017; Lanier 2018; Newport 2019) and preyed upon psychological weaknesses of end users that were originally identified and exploited by the gambling industry (see Schüll 2012). Scholars contend persuasive technologies further blur the boundary between the technological and human (Dorrestijn and Verbeek 2013; Verbeek 2009, 2011), reinforce the power of platform technologies (Stark 2018; Yeung 2017), are emblematic of wider institutional “traps” in society (Seaver 2019) and are complicit in the construction of mundane user digital practices (Jovicic 2020).

Persuasive technology offers the potential to shape the conduct of end users. This is done by applying the psy sciences on human conduct beyond the laboratory. Rose (1998, 1999) argues the psy sciences extended governmentality, or governing the human subject, by not only normalizing the idea of a human psychic interiority and exteriority, but also by offering techniques to calculate the human psyche and therefore make it more knowable and manageable. The psy disciplines have been similarly influential in the formation of persuasive technology. The first published work concerning persuasive technology came from social psychologist B. J. Fogg (2001). Fogg is the founder of the Persuasive Technology Lab at Stanford University where he formalized behavioral and social psychology research into a design model. Fogg was influenced by Albert Bandura’s (1977) theory of social learning, a theory in psychology that considers behavior a combination of observing stimuli and internal psychological processes. Fogg’s (2001) key insight was that computers could operationalize basic principles of social psychology, which led him to coin the term “captology” (a neologism for computers as persuasive technologies). Fogg (2001) believed that computers can persuade as either direct tools or passive intermediaries between actors by performing simulations, calculations, or measurements to understand and elicit human motivations and behaviors. Captology “enabled human powers to be transformed into material that could provide the basis for calculation” (Rose 1999, 7). Notable graduates of the Persuasive Technology Lab include Mike Kreiger, the co-founder of Instagram, Evan Spiegel, the co-founder of Snapchat, and Tristan Harris, a former Google design ethicist and founder of the Center for Human Technology (Leslie 2016).

Crucially, persuasive technology design does not necessarily equate to manipulation. When analyzing the use of the psy sciences to engineer demand for consumer products, Miller and Rose (2008) do not imply that consumers have diminished agency. In their view, psychological techniques do not necessarily dominate or manipulate consumers, but mobilize them “by forming connections between human passions, hopes and anxieties, and very specific features of goods enmeshed in particular consumption practices” (Miller and Rose 2008, 115). Similarly, popular persuasive designer Nir Eyal (2014) claims to influence end user behaviors by creating strong relations to consumer products. In his bestselling design manual, Eyal offers the “The Hook Model,” which includes four phases of designing user habits: identifying external and internal triggers of behavior; encouraging behaviors via the lure of reward; offering rewards at variable internals to generate a craving; and incentivizing the user to invest in the product/service to improve their experience (Eyal 2014, 4-7). Other design models utilize affective aesthetics ⁷ to enhance the emotive appeal of the end product (Norman 2004), while others lean more on behavioral data to measure the impact of the design and draw insights to refine the user experience (Wendel 2014). These persuasive models mobilize the end user to act by identifying and positioning certain relations to the user in subtly different ways and have become formalized in User Experience methods such as A/B testing (Stark 2018). Employing such design models, many apps or digital platforms use nudges, gamification, or other persuasive techniques to maximize user retention or generate behavioral outcomes for the benefit of the commercial client. In environments and economies where user attention is sought after (Crogan and Kinsley 2012; Wu 2016), persuasive technologies provide an edge for businesses over their competitors. Taken together, these models of behavioral change establish persuasive technology as best practice in software development and design.

Boundless Mind claims to offer best-practice methods in mobilizing the end user to commercial ends. Boundless Mind has published its own persuasive models and techniques to program user habits (Combs and Brown 2018). Their Cue-Action-Reward (CAR) Model follows previous behavior change models by systematically instructing readers how to mobilize the end user by identifying cues, actions, and rewards in relation to behavior. Combs and Brown (2018, 21-25) claim to recognize the interior subjectivities of end users at an individual level, combining neuroscience with behavioral science to create persuasive techniques such as “optimal information flow” and “cognitive load balancing” in the Boundless Mind “Behavioral Design Toolbox.” Moreover, on the Boundless Mind (n.d.) website, prospective commercial clients learn how Boundless combines their behavioral expertise with software development. Readers are told that the brain is “programmable,” and to do so, you just the need the “code.” The code is Boundless Mind’s commercial persuasive technology: the Boundless API. APIs are software that standardizes access to particular data sets or software protocols (Helmond 2015) and the Boundless API combines machine learning protocols and neuroscience, and applies them to a customer’s app to increase user retention. Past clients include education and nutrition businesses who used the Boundless API to increase adherence to learning or dietary digital programs (Shieber 2017). Tim believes the Boundless API has enhanced outcomes for health and education businesses and their consumers: “Over the past few years we’ve been able to help people better adhere to their medication, pay down their debt earlier, study harder [and] mediate more.” Given that users of these health, productivity, or education apps are likely to be unaware the app has been optimized by the Boundless API, Boundless Mind mobilize the end user from a distance. ⁸

Despite a popular perception that the development of persuasive technology is correlated to the immobilization of the end user by tethering them to their digital devices (Leslie 2016; Lewis 2017) popular persuasive technologists have acquired a “therapeutic vocation” (Rose 1998) by repackaging their expertise as therapeutic solutions consumers can apply on themselves. Both Fogg and Eyal have turned their expertise on persuasive technology into self-help books, offering tips directly to consumers on how to develop everyday micro-habits (Fogg 2019) or become “indistractable” (Eyal 2019) to the lures of social media. The therapeutic vocation is a reminder of the double-sided disciplinary nature of governmentality where a “social problem” of bad habits is viewed as fixable via habit-modification methods (Bennett et al. 2013). In the next section I demonstrate how Boundless Mind act as a therapeutic authority by walking through their application Space.

The Therapeutic Authority of Boundless Mind and Space

Boundless Mind have also applied their expertise to a therapeutic solution for end users. This therapeutic solution is an iOS and Android app and Chrome web browser extension called Space, which aims to help users disconnect from any app that the user wishes to use less. Moreover, Space reveals how persuasive technologists derive their therapeutic authority. My walkthrough of Space and analysis of Boundless Mind’s therapeutic authority are organized around the two discrete steps of using the app: obtaining a replacement app icon and undertaking the therapy of a design technique called stimulus devaluation.

Obtaining a Replacement App Icon

When opening Space for the first time, users are taken through an onboarding process that provides information about the app in the wider context of persuasive technologies. This onboarding information reinforces Boundless Mind’s expertise on persuasion and their view of the fixable subject. Users are informed that “it’s not [their] fault [that they are] hooked on apps” and that “Space puts you back in charge.” The message of self-help evokes a popular genre of literature in the 1970s that packaged seminal behaviorist B.F. Skinner’s ideas in the format of self-control (Martin and Pear 2010; Watson and Tharp 2013). ⁹ Yet the promise of Space is that the app (and not “you,” the user) is the reason “you take back your brain.” As behavioral and software experts, Boundless Mind claim an “internal truth” (Rose 1998) about the level of persuasion embedded in apps and the effects on users. Unless the user is well versed in behavioral science, neurotransmitters such as dopamine, or software development they are likely to interpret the onboarding information as “not an external truth—be it divine right or collective good—but an internal truth, one essential to each individual person over whom it is exercised” (Rose 1998, 91).

The onboarding process suggests to the user that Boundless Mind is a therapeutic authority. Therapeutic authorities are organizations with psychological capability and expertise to remedy recognized mental health deficits and deliver therapies (Miller and Rose 2008). However, if therapeutic authorities require official recognition of mental health deficits, then there are doubts as to what exactly Boundless Mind is remedying. “Internet addiction” or “screen addiction” have not been officially recognized as a mental health deficit in the sixth revision of the Diagnostic and Statistical Manual of Mental Disorders (DSM-6), due to significant doubt concerning the causality between screen time and mental ill health (Orben, Dienlin, and Przybylski 2019). But the purview of therapeutic authorities goes beyond resolving mental health disorders that are grounded in empirical evidence. Rose (1999, 231) argues therapeutic authorities have additional social power as they are also “sought when individuals feel unable to bear the obligations of selfhood,” meaning the pressure to make strategic and justifiable decisions in a neoliberal society where day-to-day activities are subject to market forces and demand choices. One of the major concerns about persuasive technologies is the impacts on end user productivity and attention spans (Carr 2011; Newport 2019; Soojung-Kim Pang 2013). The popular perception of a “digital distraction epidemic” (Twenge 2017) warrants the demand for a therapeutic authority like Boundless Mind.

For Space to work, users must first set up the intervention, which is achieved by obtaining a replacement icon. The homepage of Space is a directory of third-party apps from which users commonly seek to disconnect. Apps are listed under categories of “social,” “entertainment,” and “productivity,” with users asked which app they would like to spend less time on. If the user wishes to use less of an app that is not listed, they can request that app to be added to the list. Users choose which app they wish to use less by selecting the button “Create Space.” Doing so automatically opens the user’s Safari mobile browser, taking the user to a website with the URL: https://canaveral.usedopamine.com (Figure 1). The user is instructed on how to obtain a replacement icon for the app they previously selected. Here, Boundless Mind utilize an existing iOS functionality in Safari browsers called “Add to Home Screen.” The “Add to Home Screen” functionality is an iOS feature that enables users to create convenient shortcuts to favorite websites on their home screen. In this instance, Boundless Mind use the functionality of “Add to Home Screen” to offer a replacement icon of the app the user has selected they want to use less. After adding the website to their home screen, a replacement icon appears that is visually distinct from the original app. The iconography and colors of the original app are redrawn into white wireframes and a space themed background.

OPEN IN VIEWER

Figure 1.

Process to get a replacement icon.

The replacement icon is not a direct replacement of the original app because Boundless Mind do not want the user to remove the flagged app. If the user removes the original app, then Space stops working. Users are therefore instructed in the onboarding process not to uninstall the flagged app, but instead to hide the original app icon in another screen or folder. If the user uninstalls the app, then Space will not work. However, relying upon the presence of the original app dilutes the impact of Space, comparable to giving a smoker a nicotine patch on the premise that the smoker must keep their cigarettes. Users are given the opportunity to customize the name of the replacement icon. Once they have completed these steps, the replacement icon appears on the desktop of the user’s iPhone. The user can repeat these steps to obtain as many replacement icons as they would like.

The replacement icon is intended to be used instead of the original app. The replacement icon therefore signifies the level of intimacy that Boundless Mind seeks to have users; operating as an intermediary or invisible layer of software to intervene in instances where the user has highlighted a behavior they wish to change. Tim explained his vision of Boundless Mind to me:

I want Boundless [Mind] to operate as the invisible layer of the net that glues all of that together. In the same way that Google gets to act as the invisible layer of the net around routing and search. I’d like us…to operate as a technology that we look back on as a turning point in a lot of the things that hurt really bad around behavior, that we can walk back from some of these problems that totally suck.

In addition, Tim’s vision of Boundless Mind as operating at an invisible layer of the internet reinforces that Space is not an app that is intended for either active or passive use. Rather, Space offers replacement icons that function as enhanced settings for smartphone users—a functionality that, by itself, is useless but, once set up in combination with other applications the user wants to limit the use of, provides an optimal process to avoid those apps.

The Obfuscation of Persuasive Technology

Boundless Mind and the Space app perform ethics by helping users to disconnect from their smartphone. Individuals seeking productivity and focus to better navigate neoliberal market realities may see the appeal of Space and become sympathetic to the ethics of Boundless Mind. Yet, the allure of Boundless Mind’s ethics has the potential to obfuscate the expansion of persuasive technology. One of the subtle implications of Space is that the habit-breaking app positions Boundless Mind as indispensable to self-directed ethical conduct. Foucault’s imagination of a mode for subjects to be less governed “through reinventing subjectivity and autonomy within the existing social order” (Zamora 2019, 263) reinforces a vulnerable neoliberal subject (Chandler and Reid 2016) that must continuously reinvent themselves and adapt to their surroundings. But Foucault could not have anticipated that experts (and not individuals) would become integral to the reinvention of the self, nor that self-optimization culture would itself be normalized and institutionalized into everyday life (Binkley 2014). The appeal of Space to users is the opportunity to apply behavioral expertise to curb their undesirable technology habits; a habit that by itself may be difficult to treat. When critically assessing nudges, Pedwell (2017, 82) argues “responsibility is…delegated to “choice architects” with the requisite knowledge and foresight to steer behavior in appropriate directions.” Space is a delegated authority to act on behalf of the user that intermediates “enterprise and submission, responsibility and discipline” (Schüll 2016, 12) both reinforcing and curtailing cultural ideals of personal responsibility. The underlying basis of nudges or persuasive technologies is that users do not always make the best decisions for themselves and reductions on individual autonomy are therefore justifiable. Yet the delegation of authority to Boundless Mind to fix the users’ habits obfuscates the organization’s role in creating compelling and compulsive digital products in the first place. It is worth reemphasizing that persuasive technologies drove models of user behavioral change and engineered consumer demand for products. The original purpose of persuasive design was not to remedy perceived anti-social or undesirable behavior like the psy disciplines (Rose 1998), but to mobilize the end user to economically profitable ends. The par excellence of persuasive technology has arguably compounded the perceived undesirable behavior of mindless scrolling. Yet the ethical perspective of Boundless Mind positions the organization as providers of both the poison and the antidote to the issue of habitual user scrolling.

A persuasive technology ethics based upon expertise in user subjectivities also flattens the power of the persuasive technology industry. There is no account of the materiality (and subsequent asymmetrical relations) of the persuasive industries (Stark 2018; Yeung 2017; Zuboff 2019) in relation to the user. Yeung (2017) suggests that persuasive technologies become highly persuasive when combined with the engine of big data analytics. Big data has the potential to incorporate the user into feedback loops and personalize digital experiences, which in turn can amplify the persuasiveness of technologies. In addition, the behavioral insights that Boundless Mind generate with Space renders more user subjectivity into calculable data: which apps users choose to disengage from, what times of the day users disconnect, or how successful their attempts to disconnect are. Data about user’s technology habits could be valuable to a range of third-party organizations and industries, such as employers, educators, marketers, or health insurers, raising significant privacy concerns. Numerous scholars have raised concerns about the fusing of big data and technologies of behavior to shape consumer behavior (Tufekci 2014; Yeung 2017). Not only could data about technology habits be used to govern workers habits (Gregg 2018; Guyard and Kaun 2018) but also limit access to credit and other schemes that determine certain behaviors to insinuate anti-social behavior or financial precarity. The power and potential profit in operating Space lie not only in the immediate nudge, but the broader behavioral insights that only a persuasive technology like Space can offer. For example, the German credit lender Kreditech claims to be able to assess borrowers’ reliability by analyzing over 6000 data points including social media platforms (Vasagar 2016). What could data on breaking technology habits reveal about a users’ trustworthiness or credit worthiness? Not having a social media presence—a type of passive disconnection—has been interpreted by recruiters in the past as socially abnormal behavior (White 2012).

Conclusion

This paper has critically assessed the perspective and activities of a persuasive technology organization that claim to be at the forefront of “doing ethics” in their industry. By walking through their habit-breaking app Space, I have argued that Boundless Mind’s ethics derives from their expertise in user subjectivity and capability in governing the behavior of users. I have also criticized this ethical perspective for obfuscating the role of Boundless Mind in creating compelling environments in the first place. The practitioner-based persuasive technology ethical perspective may scale up to broader ethical guidelines across the technology industry. Boundless Mind anticipate a “Corporate Behavioral Responsibility” scheme by which the technology industry could self-regulate, suggesting such a scheme would: “hold firms accountable for honesty, transparency, and choice in how their products—intentionally or unintentionally—reprogram human behavior” (Brown 2018). While corporate responsibility schemes appear to demonstrate good faith, they have historically enabled businesses to exercise soft power and function to forestall independent regulation (Rajak 2011). Perhaps we should be wary of the ethical authority that organizations such as Boundless Mind claim to exercise. Especially if they claim to understand our behaviors better than we do.