Making sense of negotiation and AI: The blossoming of a new collaboration

Introduction

The idea of merging negotiation and artificial intelligence (AI) has been a dream for many decades. Wouldn’t it be wonderful to outsource to technology the sophisticated, laborious, and emotionally intense process of mixing collaborative and competitive moves to overcome complexity, information asymmetry, and suspicion toward an acceptable outcome that satisfies the interests of all parties involved? Until recently, most people could not believe that machines would ever do such a human-like task as negotiation.

Meanwhile, research on the intersection between negotiation and different AI technologies has included machine learning (ML), ¹ deep learning, ² computer vision (e.g., facial recognition), ³ natural language processing (NLP), ⁴ and even robotics. ⁵ In attempts to emulate negotiation behavior or improve performance, researchers and entrepreneurs have explored different AI systems, including rule-based systems, predictive analytics, generative models, and autonomous systems, ⁶ to create support systems or virtual agents that can help us perform as well or better than human negotiators alone.

After the awe or jump scare most people felt following the ChatGPT launch on November 30, 2022, large language models (LLMs) popularized AI and opened people's imagination to how machines can become a part of negotiations. LLMs revolutionized our perception and adoption of AI with their user-friendliness and ability to engage in undistinguishable human-like, fluent, and multi-topic conversations. ⁷ While many are exploring LLMs in all sorts of different applications, ⁸ relatively few LLM applications exist for competitive or mixed-motive settings, namely negotiations. ⁹ Still, given the opaqueness of their reasoning processes and outputs ¹⁰ and their rapidly rising adoption, even in negotiations, it is helpful to anticipate the potential development, risks, and possibilities that this new wave of AI can generate. ¹¹

The recent interplay between AI and negotiation represents a paradigm shift in the latter. Negotiation has been seen by most as an art. Only in the past few decades has it evolved into a science ¹² focused on minimizing the human cognitive and socio-emotional obstacles and codifying a systematic analysis of problem-solving toward successful negotiations. ¹³ With the rise of AI, the human–machine collaboration raises the ceiling of what we can achieve or how negotiations may evolve. AI in negotiation is drawing from economics or game-theory methods ¹⁴ to identify Nash equilibrium and Pareto optimality and facilitate convergence. ¹⁵ On the other end, LLM-powered agents emulate human behavior in negotiations based on social science techniques for managing the human aspect of negotiation. ¹⁶ Even if much of it is being done without a rigorous theoretical foundation, ¹⁷ AI is opening unprecedented possibilities for the future of negotiations.

In this article, I present thoughts and questions on the current state of negotiation and AI that may help us see, understand, or influence their unfolding future. For that purpose, I conducted a review of the existing literature on negotiation and AI, complemented by an examination of recent developments in negotiation and AI through news sources. Additionally, I relied on insights from my dual roles as an academic and entrepreneur operating at the intersection of AI and negotiation to provide my perspective.

How are negotiation and AI interacting?

AI and negotiation have been interacting in three main ways: AI assistance, semi-automation, or automation support to negotiation. These three categories do not represent differing quality levels; instead, they represent modulations or use cases of AI to provide more tailored support to different negotiation needs.

Regarding assistance, negotiators use LLMs as a more sophisticated search engine to prepare before and during negotiations. At a corporate level, given the increasing volume and complexity of data, the existence of legacy, siloed, and non-standardized systems or data, and the need for technical and analytical skills, companies such as Eularis ¹⁸ or Tellius ¹⁹ are selling AI data analytics services to help their clients prepare for their negotiations. While the amount of data and the quality of insights may be superior due to AI tools, their role is still limited to assistance.

Regarding semi-automation or automation, the intersection of negotiation and AI happens mainly at the corporate level, given their higher sophistication and need for investment. A few companies, such as VN Tech, ²⁰ Pactum ²¹ , and Luminance ²² , fit into these two categories as they develop their AI-enabled negotiation tools to help their corporate clients conclude more efficient and higher-quality agreements. While the automation category involves a virtual agent who operates almost independently from the human negotiator, the virtual agent in the semi-automation category automates several functions beyond the assistant category but still needs more touchpoints and inputs from the human negotiator.

AI and negotiation preparation and training (assistance)

Before LLMs, negotiations had leveraged technology through negotiation support systems (NSSs), which facilitate information analysis and communication protocols to successfully assist negotiators in reaching higher joint outcomes and satisfaction. ²³ NSSs deliver value by presenting mathematically optimal options for negotiators to choose from or deliberate over. NSSs work by having negotiators input the negotiation issues, their interests and preferences (weights), and the option ranges to build utility curves and compute solutions that maximize value for all parties. ²⁴

Despite being around for decades and providing validated value, NSSs were never adopted by the average negotiator, thus limiting their real-life impact. With the recent popularity explosion of LLMs, some more NSS benefits became available to negotiators, with added accessibility, user-friendliness, flexibility, low cost, and additional functionalities. Besides, LLMs have been novel and exciting, as almost anyone, including many negotiators, is curious and eager to learn or adopt them.

Indeed, generic LLMs (ChatGPT, BERT, BART, LaMDA, Gopher, PanGu-α, etc.) ²⁵ can be used as preparation or negotiation assistants. They can gather relevant market information, advise on negotiation and preparation best practices, generate reasonable assumptions about the counterparties’ interests and priorities, expand your set of interests, suggest different or ideal proposal packages based on revealed or assumed preferences, recommend arguments to support desirable options or counter undesirable ones, accept prompts to role-play more straightforward negotiation cases (as you or as the counterparty) with impressive realism to stress-test strategies, and give feedback to refine the negotiation preparation and strategy further.

While the advice is generic, the information is sometimes wrong, and the role-plays are still glitchy; future LLMs are expected to fix these weaknesses. For now, generic LLMs have dramatically lowered the preparation costs of researching and gathering relevant information, learning and adopting essential negotiation advice, and preparing a robust negotiation strategy. As a negotiation preparation assistant, generic LLMs can increase the average negotiation preparation quality at a fraction of the time or cost, which may eventually increase the number of negotiators that enter negotiations well prepared and improve their performance and outcomes. ²⁶ LLMs make for an excellent preparation tool before and during negotiations, even if they have not yet had a direct impact on the negotiation process itself.

Besides being a negotiation preparation tool, LLMs have great negotiation training potential. Companies such as Qinect, ²⁷ Simulation Labs ²⁸ and iDecisionGames ²⁹ offer LLM-supported negotiation training with simple (for now) negotiation scenarios, the possibility for text-based role-playing in many different languages, and individual feedback and data on performance. Students, individually or in groups, can download the text-based dialogues for in-class or homework analysis. Such an inside view of the negotiation dynamics took time and effort. Still, they can now quickly help students identify the effectiveness of different moves, enhance their self- and other-awareness toward specific areas for improvement, and thus provide a superior, customized learning experience. The AI-powered negotiation avatar also provides a scalable and consistent counterparty for negotiation role-plays, allowing for a more balanced benchmarking among students, given that no one negotiated against a more or less challenging counterpart. It also allows students to role-play the same negotiation multiple times with the AI negotiation avatar but with different strategies to compare their performance.

Despite these many advances, these companies’ use of LLMs for negotiation training is yet to have a rigorous evaluation of their performance in improving negotiation skills. ³⁰ In their search for survival and constant improvement, these companies will seek to accumulate an immense amount of negotiation text and data as fast as possible to develop and train proprietary and negotiation-specific LLMs. In addition, their users will demand the most realistic and human-like training experience possible to ensure that the skills learned are applicable in negotiations. Hence, these companies may soon offer interfaces where students can talk while looking at a realistic avatar face to approximate the learning experience to that of an actual F2F negotiation.

For example, Qinect leverages currently available LLMs (brand agnostic) to power its virtual negotiation agent. It then adds its layers of programming to ensure that the virtual negotiation agent's responses are as human-like as possible, including emotional parameters, such as trust and anger, and personality levers to the virtual negotiation agent. This addition mimics the counterparty serendipity and emotional volatility from real-life negotiations. In time, these educational companies can use such levers to customize different agents and increase human likeness driven by the incentive to innovate and create a virtual negotiation agent indistinguishable from a human one. Educational companies will likely take a different AI development path than companies focused on using AI to produce superior negotiation results for their clients rather than human likeness.

AI and negotiation automation or semi-automation

AI is already a game changer for business negotiations. However, most negotiators lack the expertise in computing, negotiation, economics, game theory, or psychology to individually leverage AI beyond the assistance level. ³¹ That said, AI can drastically reduce decision costs (limited cognitive abilities, time, or information) and help negotiators consider and compute the necessary variables and negotiation rules to reach rational outcomes ³² as opposed to having to settle for a satisficing solution. ³³

Over a decade before LLMs arrived, AI technology based on game-theoretical insights could already power automated negotiations. ³⁴ Adoption of AI-supported negotiation systems (semi-automated or automated) by large corporations can translate into competitive advantages, short-term gains, and an early learning curve, ³⁵ and yet such adoption requires time and cost investments to change existing processes and systems. ³⁶ Hence, it may take a while before we see a significant shift in the daily negotiation routine of most corporations toward automated or semi-automated negotiation agents.

Regarding semi-automated negotiations, the VN Tech example resembles the NSSs mentioned before with several AI improvements. VN Tech uses AI to help negotiators define their negotiation issues and interests, facilitate the calculations of their utility functions and value curves, prepare different value packages (options), and research legitimate targets to create reusable negotiation preparation templates. Once ready, the negotiator can initiate a gamified, visual negotiation process by inviting and providing its counterparties with ample information and AI assistance to play with different preference combinations and craft high-quality mutual-gains agreements. After the counterparties respond, the platform suggests Pareto improvements to increase value for all and invites them to decide, thus requiring human oversight to close the deal. Given how the loss of perceived control over the negotiation process may lead to a rise in negotiator anxiety, ³⁷ some individuals or companies may prefer semi-automated systems over fully automated ones.

Luminance and Pactum are current examples of automated negotiations. Pactum has been offering automated bots to negotiate long-tail procurement agreements with few issues and small values. Their text-based chatbots are programmed under a rule-based AI system and do not rely on LLMs. Pactum agents draw from large amounts of real-time market data to negotiate with suppliers via their platform, searching for superior deals for both sides. The bots can agree to acceptable pre-approved deals without human supervision.

Luminance claims to have developed the first AI in the world to completely automate contract negotiation between two opposing parties without human intervention. So far, it has focused on non-disclosure agreements. It leverages a database of 150k contracts to build its legal LLM, which powers its program Autopilot to analyze contracts, suggest changes, and send the redlined version to the counterparty. In an experiment where both sides had the Autopilot application, the virtual agents only needed a few rounds over email to settle on a deal.

It is curious to find the most visible AI and negotiation initiatives involve legal, procurement, or government contracts. This pattern may result from these areas being cost centers, negotiation heavy, and eager to find savings, making them hungry for new solutions that can alleviate process backlogs or inefficiencies. Besides, these negotiations have lots of available data (e.g., former contracts), formal and structured processes (e.g., RFPs), large power differences, and higher transactional than relational concerns, which facilitate introducing new AI-supported processes.

Human or machine negotiators?

As seen above, the negotiation and AI landscape does not seem to fit the categorization of human–human, human–machine or agent, computer, robot, or model, ³⁸ or machine–machine negotiations anymore. The different levels of AI support or human involvement during negotiations can be quite diverse. For example, we cannot consider a negotiation in the assistance category as exclusively human–human anymore, even when they are the ones negotiating, as they are highly guided by AI input. Similarly, labeling a negotiation as machine–human or machine–machine in the automated category seems to ignore how much it still relies on significant inputs from or leaves the final signing responsibility to a human negotiator.

As we advance, it may seem more representative to label negotiations as if describing a human and machine negotiation team or collaboration, such as human/machine (HM) or machine/human (MH), placing the lead negotiator first. We could also describe the relevance of their role in the negotiation by capitalizing the negotiator who worked most. We would then describe assistance negotiations as human/machine (HM) to demonstrate how humans still carry out most of the talks. For semi-automated negotiations, we use human/machine (HM) or machine/human (MH) to represent how both are still heavily or equally involved, though one still leads. Finally, we describe automated negotiations as machine/human (Mh), and once AI helps negotiations become truly autonomous from humans, we call these agents Machine (M) only.

Potential AI and negotiation opportunities

So, what does AI bring to the negotiation team? The answer is the ability to compensate for human shortcomings, machine-efficiency advantages, and the potential to reinvent how we negotiate.

Human negotiators are limited by bounded rationality, cognitive biases, ignorance over negotiation best practices, and emotions that hinder our communication, rationality, negotiation performance, and thus our ability to craft and agree to optimal solutions. ³⁹ That said, given that AI systems are trained on historical data, they also develop biases ⁴⁰ but reducing, even if not eradicating, them in machines seems to be easier than with humans. ⁴¹ While AI systems do not know everything, they can be trained much faster than humans as more data becomes available. And while LLMs can mimic emotions, as far as we know, they do not feel them. Thus, they have an easier time remaining rational and sticking to negotiation best practices than a human negotiator.

AI is advertised as able to leverage automation and machine-like benefits for repetitive tasks, such as increased task speed, consistency, scalability, replicability, or compliance. Currently, negotiations are rarely recorded, leaving us ignorant over what was said or done in the room, including unethical or illegal practices (e.g., bribes, abusive behavior, conflicts of interest). Automating or semi-automating negotiations can increase process traceability and transparency and open the door to negotiation audits. This is another way AI can help organizations increase negotiation fairness and accountability, as they can now ensure that their negotiators and negotiation processes uphold legal and ethical standards. Other advantages include cost and error reduction and faster adoption of new instructions, best practices, upgrades, or learning. Compared to older machines, limited to repeating a narrow set of programmed instructions, AI provides significantly more flexibility and automated learning to adapt to new tasks.

One of the more exciting opportunities regarding AI and negotiation is the possibility of using AI to rethink or redesign negotiation processes. Right now, it is natural that those using AI in negotiations are anchored on historical human–human processes and mostly use AI to help us negotiate better, but still replicating much of how we negotiate. For example, Pactum has its chatbots engage in normal-looking text conversations with suppliers (e.g., “we can offer you better payment terms if you give us a discount.”). Meanwhile, Luminance exchanges AI-redlined contracts over emails, much like lawyers have done for decades.

In human–human negotiations, the parties either use their power to extract advantages from their counterparties (win-lose) or attempt to communicate clearly and build trust to craft a deal to their liking (win-win). Whether win-lose or win-win, current processes usually entail a careful dance of information exchange as the parties gauge each other's intentions and try to share information to help them create and claim value while not oversharing to avoid exploitation during the value-claiming phase. However, given the AI advantages listed above and other yet-to-be-learned opportunities, those exploring AI and negotiation will soon learn to leverage AI and AI–human collaborations to reinvent and thus improve negotiation processes.

AI can handle so much data at once that either side can share all their interests and preferences in an AI “black box” or mediator, where neither negotiator learns the limits or secrets from the other, which the AI can use to produce optimal solutions that humans are unlikely to craft on their own or through normal negotiation processes. With such new possibilities, do we need to continue negotiating the same way?

For example, value creation in negotiation has often been associated with reframing the process from a single- to multi-issue agenda. However, recent research has found that the complexity of negotiating too many issues at once can be cognitively overwhelming for humans to handle, which reduces or caps value creation. ⁴² Indeed, human–human multi-issue negotiations often engage in lengthy, non-linear, tentative, partial, or overwhelming exchanges within or across a subset of issues, round after round, and under the constant threat of misunderstandings, emotional escalation, or opportunistic moves. Conversely, with AI's computational ability, negotiation can juggle an enormous number of issues simultaneously to find optimal trade-offs and solutions quickly and with fewer communication or relationship risks.

In addition, AI may also have an easier time sticking with negotiation best practices, such as following a tit-for-tat approach, starting positively (as it does not feel fear), reciprocating negative moves (not to punish or escalate, but to teach the counterparty), and forgiving and returning to collaboration in response to a positive move (as it does not feel the need for revenge or retribution). ⁴³ Moreover, AI may also resist bias exploitation (as it can be resilient against anchoring or other biases), power moves (as it does not feel pressure), or manipulations (as it feels no emotion) and thus become a great negotiator against win-lose tactics.

Transparent sharing of optimal solutions may disincentivize win-lose or power-based moves (e.g., pressure, lies, bluffs, or manipulations) and thus increase the ethicality of negotiation processes. For one, the more powerful negotiator now faces the trade-off of a greater risk of generating resistance to the deal or damaging the relationship versus the reduced opportunity to capture additional value given that the negotiation is already at an optimal point. This rebalanced trade-off may tilt the more powerful party's decision toward exercising restraint and accepting the optimal solution. Similarly, the weaker party, knowledgeable of optimal solutions, has the power of information and legitimacy to stand its ground and stay close to the optimum.

Current AI and negotiation challenges

Despite the abovementioned opportunities, there are still several kinks to the emerging AI and negotiation partnership, many of which may get sorted in due time, but others that may evolve and plague the field for years to come.

Many of the current AI and negotiation challenges impede a broader AI scalability and applicability within negotiations at this stage. For example, AI automated negotiations, for the time being, are limited to small-value, few-issue, repetitive, long-tail negotiations to contain the losses and risks from AI glitches or weaknesses and AI's inability to automate some essential parts of more complex processes, such as trust-building. Similarly, LLMs are still limited to an assistance or training role. Given the higher human involvement, these limitations are less prominent in semi-automated negotiations. Of course, companies and academia are working to overcome such risks and constraints to increase the scope of AI automation.

As automated negotiations become commonplace, there will be an incentive for some companies or individuals to discover, hack, and exploit the virtual agents’ rules, decision trees, patterns, or weaknesses through multiple strategies. ⁴⁴ For example, an automated agent that offers options sequentially in a text-based conversation is vulnerable to the strategy where the counterparty rejects all options, and only after exhausting them all, selects the best one available. Counterparties motivated by self-interest may devise ways to fool an agent into giving away too much. In an experiment by Schneider, Haag, and Kruse (2023), the researchers found that many students hacked the prompts of their LLM-powered virtual negotiation agent to secure superior deals, some beyond the agent's reservation price, compared to those students who just played along with the exercise. ⁴⁵

Given the reduced human oversight over automated negotiations, such hacking attempts may yield significant returns in the short- or medium-term. Semi-automated processes, or those that still keep the final decision in the hands of human negotiators, may screen and thus be protected against such attempts. Meanwhile, automated negotiations may write off or hope these low-value hacks will amount to acceptable losses. But as we see with cyber hacking, AI-automated negotiations may become a new source of value for hackers and generate an arms race within the field.

As more parties engage in automated negotiations, each agent's automated process may be different or incompatible (e.g., email vs. chat), creating a standardization problem. The companies must pre-negotiate which automated process to follow or offer agent versions that operate in different channels. Such channel negotiation may offer no middle ground in a dispute between my system or yours. In such cases, the weaker party may have to yield and reprogram their agent or engage a human to negotiate with the counterparty's agent.

Another challenge can be automated agents deliberately created to negotiate win-lose or exploit collaborative agents and humans. Right now, most designers of AI-automated or semi-automated agents claim to promote value creation and optimization to increase gains for all parties. Unfortunately, such environments invite free riders or other opportunistic players to exploit collaborative agents. ⁴⁶ Some companies may disguise their agents as win-win, but given the black box of how they operate, they may employ subtle win-lose tactics to squeeze more value from counterparties. Companies claiming, truly or falsely, the superiority of their win-lose agents can become a tempting proposition for powerful clients who can impose their choice of agents on their smaller counterparts.

Even if not deliberate, AI-powered negotiation agents are likely to develop biases and create unfair deals or unethical interactions, especially when trained or given rules that make them purely utilitarian. Will legitimacy matter if agents are trained toward a reductionist optimization exercise? Even though LLMs can make and respond to subjective arguments well, will it matter? Do they understand, relate, or value the legitimate arguments behind options? Or will negotiations quantify everything but value nothing? If agents only pursue the maximum economic utility in a deal, how will they behave when faced with moral or ethical concerns? Should an agent aware of the counterparty's gender or race employ tactics to exploit well-researched gender or race-based negotiation differences ⁴⁷ to secure a superior deal? In sum, it is necessary to instill ethical, legal, and optimization principles in upcoming AI algorithms to avoid the negative consequences of AI biases. ⁴⁸

AI-powered agents can also hallucinate ⁴⁹ or be too sensitive. Generic or even negotiation-prompted LLMs may behave wildly unexpectedly, end a negotiation abruptly, or force suboptimal outcomes. For example, an agent may stop the conversation at the slightest (mis)perception of an ethical violation, leading to type I errors. The same agent may overreact or walk away after receiving a threat, an insult, ⁵⁰ or, from personal experience, even just a persistent request that it had denied once before. Ending negotiations at the slightest infraction or disagreement may be necessary for compliance purposes, given the ability to record the interactions within automated negotiations. It may also raise the ethical bar for future negotiations. However, in the short term, it may significantly reduce the number of closed deals, an unaffordable luxury for some organizations.

Another challenge is our ignorance whether human–human negotiation best practices will translate well to human–agent or agent–agent ones given that research has found that digitalization changes negotiation processes, and yet have remained understudied. ⁵¹ Will our research validating integrative strategies matter in semi-automated or automated negotiations, including such tactics as information or priority exchanges, ⁵² multi-issue offers (MIOs), ⁵³ perspective taking, ⁵⁴ sequential discussions, ⁵⁵ positive mood, ⁵⁶ or multiple equivalent simultaneous offers (MISOs)? ⁵⁷ Value creation strategies that support our bounded rationality or that positively nudge individuals toward cooperation could be less impactful when used in conjunction with AI. For example, AI may dispense with the need for issue bracketing, ⁵⁸ as it can compute different permutations of all issues at once and thus does not need the cognitive simplification strategy of categorizing issues into distinct subsets to increase value creation or outcome quality.

Given that negotiation best practices face boundary conditions, we face the challenge that winning strategies in some negotiations may fail in others. For example, Rapoport, Seale, and Colman (2015) found that tit-for-tat is a resilient collaborative strategy under certain conditions, but inappropriate for some negotiations. ⁵⁹ As such, AI algorithms must develop the sophistication to diagnose the contextual and structural elements of a negotiation so they can identify the right strategy for each particular negotiation in order to perform to its potential.

Automated or semi-automated negotiations may be inappropriate at the start of relationship-dependent negotiations. ⁶⁰ Part of negotiating entails building trust and rapport as the parties get to know who they are dealing or partnering with under tense or competitive dynamics. ⁶¹ Trust seems to develop to comparable levels between computer-mediated or face-to-face teams over time. ⁶² However, another study found that humans trusted a virtual agent teammate slightly higher than other human teammates. ⁶³ As we move from teammates to counterparties given a negotiation's rise in competitive dynamics compared to teamwork, an agents’ transparency and predictability, as well as a strategically consistent and informational display of emotion, may be essential to build trust with humans. ⁶⁴ However, transparency and predictability are also elements that can be exploited in negotiations, and displays of happiness or anger at inappropriate times can negatively impact one's negotiation performance, be it among humans or between human and agent. ⁶⁵ Humans trusted LLMs even less after interacting with one than they anticipated, ⁶⁶ but as humans increased their AI literacy they harbored fewer negative emotions toward LLMs. ⁶⁷ Conversely, an LLM's natural language capability increases human trust to a point that they notice fewer of its errors. ⁶⁸ As such, agents need to apply a patient balancing act if they want to build or promote trust in negotiations, ⁶⁹ or consider leveraging AI capabilities in ways that help craft deals even in no- to low-trust contexts. ⁷⁰

While only some human negotiators have enough of a reputation that precedes them, the few companies offering the first automated or semi-automated AI-powered negotiation services will need to shape their reputations strategically. Win-lose agents, or those known as unskilled in protecting data privacy or crafting superior deals, will be shunned by most clients and counterparties. For example, hiring an agent with a win-lose reputation will signal the negotiator's choice for win-lose tactics even before the actual negotiation starts. Win-lose agents may be limited to powerful companies that can afford such negative signaling and still extract negotiation premiums from their weaker counterparts.

Culture impacts digital negotiations, ⁷¹ and AI-powered agents can be culturally insensitive and unable to adjust to how relationships are built or negotiations are conducted across countries. While LLMs can change and converse in several major languages, it does not mean that the underlying content or process is culturally adapted.

Finally, besides the ethical challenges mentioned above, the expansion of the role of AI in negotiations also brings legal concerns, even though legislation on AI is still in its infancy at the time of writing this article, and even more so as it relates to negotiations. Those legal concerns may include data privacy, confidentiality, and compliance with regulations such as data protection laws, including who owns the data on the dialogues or the negotiation moves made within the semi-automated or automated negotiation environment. Disclosing secrets or confidential information informally in a negotiation may be a common practice to build trust or untangle impasses. However, in semi-automated or automated negotiation, this information will likely be registered and captured, and thus, it risks being either divulged, leveraged, or exploited at another time without consent. What if the AI agent uses information or ideas from previous negotiations stored in its database to generate options for new, unrelated parties without obtaining consent from the original negotiators, thereby risking intellectual property infringement?

Another legal concern revolves around liability for AI-generated decisions or AI misbehavior during negotiations. In the same way that LLMs can hallucinate, ⁷² they could make wild mistakes that result in unacceptable or illegal behaviors or extremely unprofitable outcomes to the side represented by the AI agent. In such cases, can an individual sue a company for being discriminated against or receiving worse offers in ways that indicate a gender bias, for example? Who becomes responsible for AI negotiation mistakes, such as poor information or misrepresentation? Moreover, if an unprofitable deal is closed by an organization's AI, can it blame the AI's mistake to excuse itself from performing its obligation?

In one of the first such cases, Air Canada was court-ordered to compensate a customer because its chatbot gave inaccurate information. The decision argued that a company is responsible for all the information on its website, whether from a static page or a chatbot. While not a negotiation case, it hints at where some jurisdictions may draw the AI negotiation agent liability line.

In short, AI agents still have several shortcomings and face significant challenges in negotiations. Hopefully, none of them seem unsurmountable. Technology-based solutions, such as AI agents, tend to increase their reliability rapidly with time, as problems are continuously spotted and addressed to improve the system. In time, the balance will likely tilt toward the success of automated and semi-automated processes, even if they may or may not fully substitute human-to-human negotiations.

Conclusion

We are witnessing a significant evolution where AI, made famous and more accessible through the recent advancements in LLMs, has begun to reshape the negotiation landscape. The integration of AI in negotiations spans from assistance in preparation to the potential of full automation, where we can challenge traditional negotiation practices and experiment with new dynamics. While AI offers the promise of overcoming human cognitive, emotional, and computational limitations to enhance negotiation outcomes, it also brings forth challenges related to biases, scope, strategy, compatibility, trust, ethics, reputation, culture, legal, and the adaptation of human negotiation strategies in the digital realm.

As AI continues to evolve, it will be crucial for researchers, practitioners, and developers to think about how to navigate these challenges carefully. AI's integration into negotiation processes requires a balanced approach to harness its advantages while mitigating its risks to ensure that it is beneficial, ethical, and effective. The AI and negotiation collaboration will continue to blossom, offering exciting opportunities for innovation and improvement in negotiating and reaching agreements.