AI and the future of digital public squares

Origins of the digital public square

A “public square” traditionally refers to an open, communal space where people gather for various social, political, and cultural activities. Public squares are a focal point globally for community interaction and expression, facilitating everything from casual socializing to organized events where people can exchange both ideas and goods. The term “public square” has expanded beyond physical gathering points to include the whole set of spaces—physical and online—that fulfill these roles in modern society.

Jürgen Habermas wrote about the “public sphere” as a two-track system comprising both a formal and an informal track (Habermas, 1992). The metaphorical “public square” is really the informal track of the public sphere, the place where “deliberation in the wild”—unconstrained communication between groups and individuals—takes place in a way that ends up shaping, constraining, and ideally setting the agenda for the more formal track of courts, parliaments, senates, and so on, that govern functioning democracies (Landemore, 2022b).

The concept of the internet as a public square traces its origins to the early visions of digital communication theorists who compared the burgeoning network to traditional town squares. The 1994 book The Virtual Community by Howard Rheingold envisioned the internet as a new frontier for public dialogue and community formation (Rheingold, 1994). Legal scholars like David R. Johnson and David Post, in their seminal article Law and Borders—The Rise of Law in Cyberspace, explored the potential of the internet to transcend traditional governmental boundaries, highlighting its capacity to foster a more participatory democracy by enabling global conversations and decentralized governance (D. Johnson and Post, 1996). While the metaphor continues to evolve, the throughline is consistent; the internet serves an unprecedented role facilitating public debate, civic engagement, and social movements (Bohman, 2000).

By the 2010s, internet studies scholars like Ethan Zuckerman illuminated how the internet was democratizing both the production and sharing of information, enabling global participation in public discourse and expanded access to more diverse voices (Zuckerman, 2013). Danielle Allen and Archon Fung expanded on this, noting that the internet’s unique communication forms could “enhance inclusion, political equality, public deliberation, and civic engagement” due to its capacity for many-to-many interactions, structured and open-ended communication, and lower barriers to the production and circulation of information (Fung and Shkabatur, 2015; Kahne et al., 2015). The design of the internet of the 2010s laid foundations for the kinds of democratic processes that function primarily outside formal political mechanisms.

Defining an ideal digital public square

By bolstering these informal mechanisms for democratic participation, the internet brought about some of the conditions of what Hélène Landemore (Landemore, 2022b) defines as the ideal of an “Open Democracy.” It created open-ended channels for debate, such as Twitter, that improved access to agenda-setting power for ordinary citizens. In a systematic review of the evidence, greater digital media use was correlated with greater political knowledge, expression, and participation internationally—but also greater polarization (Lorenz-Spreen et al., 2023).

By this light, digital public squares have developed many of the ideal qualities of a democratic space that DuBois envisioned in the 1920s, where each individual can contribute uniquely to human knowledge and wisdom (Du Bois, 2017). These online spaces also move us closer to Rawls’s ideal of public reason, where citizens can justify their political decisions with reasons that fellow citizens, as free and equal individuals, can reasonably accept or reject within a democratic framework (Rawls, 2005). The ideal of the public sphere as described by Jurgen Habermas, however, one of free deliberation and political influence through communicative action aimed at mutual understanding and consensus, remains to be fulfilled at scale in online spaces (Habermas, 1991).

A wide range of scholars have studied the role of technology in supporting a more ideal digital public square. In Plurality: The Future of Collaborative Technology and Democracy, Glen Weyl and Audrey Tang lay out the opportunity for digital platforms to enable real time, widespread participation in governance, allowing citizens to engage directly in decision-making processes (Tang and Weyl, 2024). By integrating advanced digital tools powered by AI into the democratic framework, Tang and Weyl propose a model whereby technology acts as a catalyst for creating more responsive, inclusive, and accountable democratic institutions. Digital platforms designed for the health of the public square can and have enabled large groups of people to listen to and better understand both each other and experts. By tapping into the collective “wisdom of the crowd,” these technologies serve in Tang and Weyl’s model as enabling factors for empowering the digital public square as a space for meaningful and impactful participation.

Building on this pluralistic vision, Tang and Green propose treating civic care as a structural safety feature through the emerging framework of Civic AI (Tang and Green, 2026). This approach operationalizes ethics into concrete metrics—such as voice equity and trust-under-loss—to measure an AI system’s accountability and solidarity. Crucially, to avoid the mental health risks and extractive dynamics associated with individualized, dyadic human-AI attachments, the framework ensures its relational focus is purely civic: “the AI tends the community’s relationships; it doesn’t replace them or mine them.” This “6-Pack of Care” model shifts the focus from top-down alignment to cultivating AI as bounded, local infrastructure that facilitates relational health.

Researchers at New_Public developed a set of “Civic Signals” that characterize a flourishing digital public space (“Civic Signals”, 2023). These four components include:

• Welcoming requires a space to be accessible, to feel safe, attractive, and navigable.

The unrealized potential of the digital public square

While the digital public square is an invaluable expansion of offline fora for discourse, there are many ways in which our online sphere has fallen short of the ideals above. At least three categories of the potential of today’s digital public squares, undermining democratic participation in the process:

• Access and Inclusion: The promise of a universally accessible public square remains more an ideal than a reality. Rural communities, the poor, minority language speakers and other marginalized groups do not have the same access to digital technologies. Additionally, online spaces often become venues for harassment, disproportionately affecting women and members of historically marginalized communities. This has led members of these groups to refrain at times from participating in public discourse (Matias et al., 2020).

These distortions tend to paint public opinion as more polarized than it really is, amplifying extremes (Lees and Cikara, 2021), and degrading trust between groups and between the public and civic institutions. These perceptions further contribute to making public spaces less welcoming and inclusive, and limit our ability to participate meaningfully in civic life (Settle, 2018). Together with the rest of the access and inclusion issues, and the threats to privacy from surveillance, the digital public square can be seen as falling short of early techno-enthusiasts’ optimistic ambitions.

Current applications and challenges of collective dialogue systems

Collective Dialogue Systems facilitate collective feedback gathering at a scale previously unimaginable. However, they still currently require a significant amount of human labor at every stage of the process. Just understanding enough about these systems to ethically and effectively set up, facilitate and moderate CDS conversations requires a significant amount of training. Gleaning insights at the end of the process is similarly laborious, given how mapping high-dimensional opinion spaces often requires data science skills to distill outcomes for decision makers and participants themselves (C. Small et al., 2021).

The following obstacles presently limit the adoption of CDS and deliberative technologies:

• Deciphering meaning from data: While the sensemaking features built into these platforms can often surface valuable insights somewhat quickly, more in depth understanding of the full dimensionality of the emergent opinion landscape typically requires dozens of hours of labor per dialogue, a technical understanding and comfort with statistics, and the ability to write up the results clearly.

Opportunities for AI-enhanced collective dialogue systems

The following opportunities explore how LLMs can address many of the obstacles described above for adopting and scaling CDS. These instructive visions describe how AI can augment peer-to-peer and collective level dialogues, which are gaps in the field of digital participation platforms as identified by Rossello et al. (2024) (Figure 2).OPEN IN VIEWER

Risks and mitigations for AI-enhanced collective dialogue systems

While CDS and deliberative technology stands to benefit greatly from application of LLMs and related technical innovations, this adoption does not come without risks:

• Aggregation of opinions is one part of deliberation: It is important to distinguish the use of LLMs in CDS to solicit high-definition snapshots of public opinion from mass deliberation, the latter of which involves an iterative exchange of reasons, arguments, and justifications (Landemore, 2022b). LLMs can help scale such deliberations through mechanisms discussed above, but there are risks with any aggregation of opinions. For example, LLMs used to synthesize opinions in a deliberation can, without careful finetuning, elevate the majority opinion at the expense of minority opinions or lose important nuance in the distillation. Mitigations for this include careful prompt tuning to represent distinct perspectives. LLMs should be understood as augmenting deliberative processes rather than substituting wholesale.

Future research on AI-enhanced collective dialogue systems

• Design for “deliberation in the wild” on existing platforms: The existing digital public squares can be designed to promote healthier discourse and mutual understanding. A core area of future research could focus on prosocial designs on these more engagement-based platforms to incentivize collective dialogue.

Current applications and challenges of bridging systems

Aggregating signals from diverse users has been a part of tech products for decades. Internet sites like Metacritic and Ranker have built popular consumer platforms by aggregating diverse opinions into crowdsourced information. Google’s pagerank algorithm uses links from diverse webpages as a signal of trust and Meta has used a similar signal within its systems to fight misinformation (Rodriguez, 2019). All of these systems require some understanding of the diversity of the signal, in order to reduce the correlated error that comes from spam, manipulation, or a particularly active group of users.

In the last few years there has been a more specific focus on bridging signals, defined in this context as where there is agreement or overlap across diverse users. Meta incorporated “diverse engagement” into comment ranking in 2021 after internal experiments found that prioritizing comments that receive positive engagement from diverse other users led to quality improvements such as “less polarizing content,” decreased views of “bullying comments,” and fewer violations of community standards. Comments on civic or political topics with “diverse positive engagement” were perceived by users as “healthier” and “good for the community.” There were also gains from upranking content from users with a history of writing posts that garnered diverse approval. These changes also led to a 0.69% increase in Facebook comment views, a significant number in the context of platform optimization (“Fbarchive.org”, n.d).

In Meta’s work, diversity was calculated using the World2Vec vectors of individual users. World2Vec is a trained model in wide use at Facebook that produces an “embedding” for each user, an opaque sequence of numbers that summarizes the character of their social network and interaction history. These vectors can be used to calculate a similarity score between any two people, even though they do not directly encode sensitive information like demographics or political views (Lerer et al., 2019).

X’s Community Notes allows users to propose labels or “notes” to posts that might be misleading. Only notes that are endorsed by diverse users are displayed (Wojcik et al., 2022). Inspired by the algorithms used by Collective Dialogue Systems like Polis to identify consensus, Community Notes works by separating a user’s rating of a note into two contributing factors: how much that user agrees with the politics of the note, and how broadly helpful that note is independent of politics. Those notes which score highly along the helpfulness dimension are selected for display on the corresponding posts (Figure 3).OPEN IN VIEWER

This is a matrix factorization approach, of the kind commonly used in recommender systems to predict which items a user will like (Koren et al., 2009). In this case, exactly one factor is calculated for each note and each user, representing ideology or perspective along one axis. This axis isn’t necessarily left-right, nor is the algorithm trained on any sort of political data set. Rather, this method finds the primary axis of division between users, whatever that axis may be in the context of those specific users and posts.

In practice this approach selects misinformation labels that are both more persuasive and more widely perceived as helpful. It beats other algorithms such as supermajority voting or simple average rating (Wojcik et al., 2022).

YouTube also released a feature allowing the crowd to add notes to videos leveraging bridging. Although they have not published the rating algorithm, it is “a bridging-based algorithm” that “helps identify notes that are helpful to a broad audience across perspectives” (“Testing New Ways to Offer Viewers More Context and Information on Videos”, 2024).

Bridging can also be based on content analysis, which requires no information about the user and requires fewer inputs, meaning it may simplify explainability and implementation. Jigsaw’s Perspective API has leaned into this content-based approach to bridging by adding signals for attributes users would likely value in conversation such as curiosity, nuance, personal stories, etc. using a suite of trained models (“Perspective API Bridging Attributes”, n.d). This would enable comment hosting platforms to be consistent and explicit about their guidelines for participant interaction as well as how they score and rank posts. Another benefit here is that platforms and moderators wouldn’t need to collect additional user information or have prior comment platform activity history; a win for privacy and moderators of small communities who might not have platform activity access. Additionally, identifying explicit qualities of bridging as opposed to leaning on a black box of agreement across parties could help with fairness and algorithmic explainability. It is likely that some combination of both methods (user diversity and content quality) will be necessary for optimal bridging outcomes.

The above examples are just a sample of what could be possible by adding bridging signals to systems. While current bridging implementations have focused on finding consensus content, addressing information quality, or reducing bad experiences, there may be demand for systems that connect diverse people or increase empathy.

Bridging signals

As the examples above show, there are different ways to think about what “bridging” means, and correspondingly many metrics or scores or indicators that might indicate which content has high “bridgingness.”

Many bridging systems are based on some numeric measure of similarity or difference between users. Once there is a way to know that two users differ along some axis, many kinds of bridging signals are possible. For example, the idea of “diverse engagement” is to select content that diverse users would “like” or engage with in some positive way.

In social media contexts, it is possible to identify users as similar or different to one another based on the content they have posted or engaged with. There are a variety of techniques for turning collections of content into abstract feature vectors or embeddings that can be compared to measure the degree of difference. One example is topic modeling, which can be used to determine if two texts are on similar subjects. However, such general content analysis methods might differentiate users because they post on different topics (e.g., beekeeping vs Polish cinema) rather than having opposing positions on the same topic. Leveraging these historical content-based bridging signals can be useful in social media contexts and when users move between interoperable decentralized social networks, but may not be appropriate in all deliberative dialogue contexts where participants’ may prefer the ability to share opinions with a blank slate and change their mind across iterative dialogues.

Alternatively, there is increasing use of models trained to capture the aspects of individual posts that are relevant to bridging. This line of work began with evaluating sentiment, emotion or incivility, but has progressed to more complex constructs such as “conversational receptiveness,” a score that predicts positive discussions (Yeomans et al., 2020). The experimental “bridging” scores recently added to Jigsaw’s Perspective API evaluate comments for affinity, compassion, curiosity, personal story, or reasoned argument (“Perspective API Bridging Attributes”, n.d).

Behavior-based signals use clusters of behavior to define distance. Users who follow different influencers, buy different things, or use different site functionality will be scored as more distant than users who have more in common behaviorally. Behavioral methods can be truly content agnostic, in which case they are likely to pick up on geographic and language differences. Or they can be narrowed to more socially relevant domains, such as whether users follow differing political news sources (Cunningham et al., 2024).

Another approach is social network analysis, where patterns of friending, following, or other interaction reveal clusters of like-minded people (Conover et al., 2011). Generally, people associate more with like-minded individuals, and in highly divisive or polarized settings they tend to avoid or even cut off contact with people from “the other side.” This means that clusters in the social network often correspond to particular sets of values and attitudes. Alternatively, users might simply be asked to self-identify along relevant axes of difference. This is likely to produce more accurate data, but comes with many concerns about appropriateness, privacy, transparency, and misuse (Andrus et al., 2021).

Each of these methods for identifying dissimilar users captures a different type of diversity: what they say, who they interact with, how they identify. Behavioral and network approaches are likely to be more palatable for companies that do not want to collect or infer data with political, demographic, or religious ramifications, especially given legislation that requires the disclosure of such inferences when asked by the user (such as the GDPR).

For many methods there is likewise no need to try to describe racial, political, or ideological divisions, such as providing training data describing what “left” and “right” mean. Instead these methods try to find the primary axes of division, the main ways that users differ and disagree. However, this broader approach may mean that bridging systems fail to specifically address the most important divisive issues of our time. For such cases, it may be more beneficial for platforms to create dedicated spaces where self-identified groups can engage in meaningful dialogue across differences, or in collective dialogues.

Opportunities with bridging systems

Metrics used to select content are conceptually different from metrics used to evaluate the success and future opportunities with bridging. There are different reasons for using bridging systems, leading to different ways of measuring their success.

In the case of Facebook’s “diverse engagement,” success was measured in terms of existing quality and integrity metrics. Including diversity signals led to fewer views of content rated low quality by existing integrity metrics, and fewer user reports of content violating platform rules.

Bridging might also be done with the goal of improving relations between groups, such as increasing understanding or reducing polarization. For these broader outcomes, social scientists and conflict professionals typically use surveys asking users about how they perceive the outgroup, that is, people on the “other” side of some division. Examples include affective polarization, meaning how warm or cold someone feels toward the outgroup, and various scales of distrust and dehumanization (Finkel et al., 2020). Surveys provide a type of ground truth that may not exist in platform behavioral data.

It is relatively difficult to algorithmically influence broad user attitudes (Cunningham et al., 2024)—for one thing, much else is happening in someone’s life—so the challenge is to find a survey question that is narrow enough to move yet broad enough to be relevant. For example, general well-being measures seldom change during platform experiments, but questions like “have you had a negative experience on this platform in the last week?” can be much more informative.

Sometimes it is possible to train a model to predict the survey outcomes of displaying certain content. When this becomes reasonably accurate such a predictive model can be used to help identify appropriately bridging content, and many platforms optimize for survey outcomes in this way (Cunningham et al., 2024). Survey data is still quite scarce and expensive. Ideally, the outcome of bridging could be assessed from the purely behavioral user data that comes from regular platform use. There are several types of plausible behavioral signals, both positive and negative:

• How often people engage positively and negatively with outgroup content

Any of these could be used for evaluating the effectiveness of a bridging system, especially if their relationship to traditional survey measures can be better understood.

Online public squares could be designed to bridge political divisions. One problem with homogenous online discourse is that it allows for more division and misinformation, as there is less room for diverse audiences to correct correlated error (Shi et al., 2019). However, simply increasing the heterogeneity of online content is not always effective. Increased exposure to outgroup content had no effect on political attitudes in a large controlled experiment on Facebook (Nyhan et al., 2023), while exposure to uncivil opposing content, as distinguished from civil but disagreeing content, can lead to greater levels of attitude polarization (Stray, 2022).

To help facilitate bridging across these divisions, we can design spaces for civility in public discourse, and leverage LLMs to help identify and amplify the rhetorical strategies that promote trust and mutual understanding. Examples of these include constructiveness, sharing personal stories, and proposing solutions (Kolhatkar et al., 2020; Stray, 2022). Research has also shown that upranking specific qualities of conversation such as constructiveness and curiosity can increase understanding and reduce partisan animosity (Saltz et al., 2024). Such changes to ranking systems could even be beneficial for business goals given that consumers say that toxic political discourse leads them to use products less and some platforms have found that improving the quality of content can lead to long term usage gains (Cunningham et al., 2024). Indeed, a recent large-scale test of five bridging algorithms on three different platforms found that some algorithms succeeded in both reducing affective polarization and increasing time on site (Stray et al., 2026).

Risks and mitigations of AI-enabled bridging systems

Rewarding bridging content may sometimes be at odds with constructive conflict and disagreement that are essential to democracy. Increasing mutual understanding might not always be the main objective of a community, especially where it is important to express dissent or protest. Bridging could lead to a decrease in trust in the platform or community, if these systems are perceived to be illegitimate or opaque. Given these risks, it is important to architect and implement bridging systems with transparency and a diversity of signals for ranking (Ovadya and Thorburn, 2023).

One risk of bridging systems that leverage LLMs is the elevation of generic, broadly appealing but insubstantial content at the expense of meaningful discussion. Bridging changes ranking incentives and can thus introduce the risk of users gaming ranking algorithms, posting content explicitly for the purpose of being upranked. For example, if a platform explicitly upranks curiosity and compassion, users could game the system by posting sarcastic comments with ironic curiosity or pairs of empathetic responses to cruel remarks. Beyond ranking abuses, there are important privacy considerations related to classifying a users’ behavior or content as more or less bridging. This is akin to the existing privacy issues that arise with predicting which content is engaging.

Future research on bridging systems

Bridging systems have great potential to improve mass market discussions of important topics. We have powerful examples that illustrate the benefits as well as the way we would implement bridging and measure those benefits. However, several open questions remain:

• Can bridging be gamified? Companies have gone to great lengths to gamify engagement and could find ways to reward users for writing bridging content. Some example implementations could be amplifying their content, giving them badges or points (e.g., Reddit karma), and encouraging comedy. Could we trigger interventions by people who opt-in to be bridged, via survey questions?

Current applications and challenges with community-driven moderation

Community moderators develop, refine, and uphold criteria for acceptable engagement in community spaces by determining what are relevant practices. For example, a group for sharing memes and humor may choose to disallow posting knock knock jokes, or a disability group may disallow unsolicited advice. Members of these groups do not necessarily expect these rules to be upheld across a platform, but find value in upholding them within their group to maintain trust and engagement. This is because norms and values are contextual. A work meeting and a bachelor party may take place in the same city, with the same laws, but each has different standards for acceptable social behavior. In some cases a community’s rules may even seem to be in tension with the platform’s broader policies; for example, the Reddit community, r/BlackPeopleTwitter, includes racial verification as a moderation practice—a policy that could be considered exclusionary at a platform level (Smith et al., 2024).

Opportunities for AI-enhanced community moderation

Some of the challenges that community moderators and members face could potentially be addressed with AI-powered tools, including tools that moderators can use to proactively maintain a healthy or prosocial environment, and tools that can support moderators to reactively deal with community issues.

Risks and mitigations for AI-enhanced community moderation

While AI has significant potential to help with community-driven moderation, it also has risks and limitations that may hinder the legitimacy and effectiveness for the communities it is meant to serve. For example, opportunities to augment moderation with AI through user-facing Authorship Feedback and Community Nudges may work well in certain contexts, but can also be biased towards certain styles of language that are better represented in LLMs, subsequently disadvantaging underrepresented groups or distorting non-majority voices. In addition, as described above under Quality control and problematic moderators and processes, even aligning on community norms in the first place may be a fraught process that can lead to disagreements and conflicts, especially in communities with diverse values. It is crucial to ensure fairness and inclusivity in the norm-creation process, so that these diverse voices are heard without violating the rights of others. More generally, AI-supported community moderation technologies should be designed with sociotechnical drivers of user trust and uptake in mind, if they are to be accepted and taken up by communities that might benefit.

There are also things that AI cannot, or should not, be expected to address without human input, such as relational conflicts, crisis management, and economic incentives for moderation. To mitigate some of these challenges, we recommend the following:

• Pay the Moderators or Invest in Community Leaders: Provide financial or other incentives for engaged moderators to fairly and equitably compensate them for their labor, and ensure systems are addressing real problems faced by communities. Even as tooling improves, the work of relationship building and dialogue is at its core a human enterprise. Even if rewards aren’t financial, more support by platforms can help reduce burnout and empower them to do the community work that matters to them. Fundamentally, this requires platforms to invest in co-creation processes and support community-driven moderation efforts.

Future research on community-driven moderation

Future research depends on the development of better metrics for evaluating the success of community-driven moderation. Given that standard benchmark datasets for AI are not representative of the language of specific spaces, typical evaluation metrics like AUC - ROC say little about the actual quality of LLMs as applied for a given community (Narkhede, 2018). Further, those users and moderators judge effectiveness in ways that may be less readily quantifiable, such as the mental toll of moderation or social cohesion within a group. More foundational research is needed to define which outcomes and metrics can be used to understand how different moderation solutions may be helping or hurting communities. Researchers should also investigate how these outcomes and metrics may vary depending on the type of platform and community.

With these outcomes and metrics in mind, both platform and independent researchers can design and study effects across the range of proactive and reactive opportunities described above, and more. For example, in proactive opportunities, researchers could study the efficacy of nudges that reinforce or build on existing community norms. Research could also investigate the potential role of the source of the nudge (e.g., from a creator or bot) and contexts (1:1 or in a group space) for different niche communities. For reactive moderation, there is abundant research to be done on the efficacy of moderation based on content identified by steerable LLMs as compared to existing systems, as well as on the efficacy of AI-powered systems for designing and enforcing alternative governance structures, like juries, and in streamlining appeals.

This work could also be advanced by further research into present and historical patterns around trust and legitimacy (how it is made and unmade) within online community spaces, especially as AutoMod tools have proliferated and automatic or algorithmic judgments have been playing a greater role in moderation. Understanding these dynamics will be crucial for designing more advanced AI interventions that can run with the grain of user trust criteria in these communities, not against it.

Finally, an important area of study is to understand the potential of layering techniques. For example, community-driven moderation may also be enriched by approaches described in previous sections on bridging systems and deliberation and collective dialogue processes. More research is needed to understand the uses of deliberative conversations by and for online communities, specifically, and how these may aid in the development of norms, rules, and other practices typically associated with community moderation. While community-driven moderation may increase perceived legitimacy of moderation within communities, there remain distinct risks associated with both identifying and anonymizing community members involved in those processes.

Challenges with existing proof of humanity systems

There are numerous challenges to developing proof of humanity systems in a manner that preserves human rights and privacy. For example, virtually any indicator that one might choose is unlikely to be universal—for example, some biometric systems rely on scans of human irises, but not everyone has an iris. Others relate to adoption: in the case of validating a specific identity, someone has to devise, implement, and validate the methodology for attestation. That is likely to be a government, a large organization, or a standards body, all of which introduce trade-offs in terms of data protection.

Risks to proof of humanity posed by LLMs

The weaknesses described above may be amplified and augmented by the introduction of LLMs. Several types of malign activity will be made easier through LLMs, including both attacks on model systems and exploitation of models (Blazina, 2020; Marchal et al., 2024).

LLMs transform this malicious activity by reducing the costs to operate accounts, increasing the volume of output, raising the quality, persuasiveness, and/or precision of output, and reducing detectability. Diffusion models (that generate images) are also impactful here: it is now possible to create images of the same figure at different points in time, and in different locations, clothing, etc. Plausible persona creation is advancing rapidly. Mechanisms for identifying whether an account is what they claim to be, such as examining past behavior, posts, or broader/prior online presence for signs of manipulation, are going to be rendered increasingly ineffective. For years, reCAPTCHA has been one the main defenses used online against automated bots by certifying proof-of-humanity through interactive puzzles. As LLMs become more sophisticated, particularly at solving image-based puzzles, a study by Plesner et al. concludes that “we are now officially in the age beyond captchas” finding no observable difference between humans and LLMs on captcha-solving challenges (Plesner et al., 2024).

Principled deployment of proof of humanity systems

Proof of humanity systems must consider the risks above to ensure equitable and secure access to the services behind the systems, as well as cultivate user trust in the technology. The following is a list of considerations for principled deployment of PoH systems:

• Accountability: A robust legal and regulatory framework is imperative to establish accountability and provide avenues for remedy and redress in case of misuse, error, or data breach.

There is no one-size-fits-all solution to PoH that mitigates all risks while maintaining a high level of effectiveness. Therefore, proper scoping is essential to balancing risks and benefits for each particular use case, balancing efficacy against the severity of potential harm based on what lies behind the humanity attestation. Relevant questions to ask when considering which method to adopt include: What is the tradeoff between false positives (mistakenly blocking real humans) and false negatives (letting bots in)? What is the threat model? What are the characteristics of the population in question? How interoperable is this PoH system beyond one platform or ecosystem?

Protection from fraud and manipulation is not something that can be achieved by merely verifying humanness. This is in part because no PoH method is 100% foolproof.
 Additionally, malicious activity can be conducted by humans as well as bots, or by both in concert. This underscores the need to complement PoH systems with other approaches to fraud prevention, such as inspecting patterns of user activity over time, observing networked user behavior, and checking posts for known malicious URLs.

Future research on proof of humanity

Advances in AI necessitate simultaneous advances in digital security techniques such as PoH systems, as well as parallel legal and social challenges associated with adopting these systems. These PoH systems must balance accessibility considerations to avoid being overly stringent such that they exclude people or violate privacy rights. Future research should invest in alternative methods of human verification or machine verification that adapt with the trajectory of LLMs and consider the principles of ethical PoH deployment described above.

One active area of research is developing a robust method for allowing users to reveal information about themselves online without revealing details about who they are, such as verification with anonymity. For example, ongoing work on zero-knowledge proofs such as zk-SNARKs (“What Are Zk-SNARKs?”, 2021).

Personhood credentials are one promising application of zero-knowledge proofs for PoH. PHCs aim to strike a balance between maintaining the Internet’s commitment to privacy and unrestricted access while providing a means to combat the potential overwhelming increase in AI-powered deceptive activities. Further research can help ensure that PHCs carefully navigate complex trade-offs in ecosystem design, including minimizing data storage and processing, preventing credential transfer or theft, and avoiding the concentration of power in a single issuer, all of which can potentially compromise the system’s integrity or user rights if not managed properly. To offer one example from Adler et al., research can explore digital accessibility and literacy to facilitate PHC adoption, including studying effects of pre-installing digital credential management applications on smartphones, exploring the use of password managers for securing PHCs, and investigating ways to normalize PHC use through integration with digital civic activities (Adler et al., 2024).

In order to move toward a global model where identity assurance is required for trusted communication, investment is needed in universal identity attestation standards, identity portability and adaptive PoH systems that can evolve with rapidly-advancing AI.