Five tensions of artificial intelligence adoption for organ allocation: Applying the technology–organization–environment framework

Relative advantages

The primary relative advantages OPOs expect to receive from AI systems are (1) usability and (2) bias elimination. OPOs expect to benefit from AI systems that are easy to use, simple, and accessible for users without a strong technical background:

If it is the right AI system, it should be pretty easy, Chat GPT is easy. There's other things that are pretty easy. The build out is complex and hard, but the user interface shouldn't be, and I think they [OPOs with limited technical expertise] would adapt to that pretty quickly. (ID 10)

While usability reduces the learning curve and encourages AI adoption, concerns about bias in the allocation process may prevent AI adoption. As mentioned here, there are misconceptions that AI may resolve bias issues:

[The transplant system is] loaded with bias. As soon as you start talking about age or the type of donor like a DCD donor [when circulatory and respiratory function are irreversibly ceased] versus a brain-dead donor [organs meeting neurological criteria for death], immediately the bells start going off and you have surgeons who remember the one bad case. [….]. So, my hope is in AI and it's so true for anything. Ideally, the algorithms will not be loaded up with so much of that bias. (ID 6)

While AI and algorithms in general may be able to help with decision-making heuristics like availability bias, ³⁹ they are not able to fully remove bias. Bias is often embedded in algorithms due to its presence in training data and assumptions made during data cleaning. Unfortunately, the optimism that AI systems can mitigate bias in general is at odds with the existing technological capabilities, the regulatory issues, and practical limitations of bias mitigation strategies to fully eliminate bias from the allocation process.^40,41

Compatibility with the business case

OPOs mentioned three key use cases of AI systems that are expected to increase the likelihood of AI adoption: (1) improving organizational outcomes, (2) managing out-of-sequence allocation, and (3) identifying behavioral patterns of surgeons.

Compatibility with business process

AI adoption is likely to be limited by the need for human involvement, but may be encouraged by the observation of an AI system's outcomes in a test environment. A few OPOs were reluctant to completely rely on AI systems and remove human involvement in the process:

AI can add a lot of value in making sure that we remove human error. But, I don't think that AI should be used to remove human instinct. I think that's one of the most important things we want to try to maintain the human touch, the human understanding, the human compassion of the gift. (ID 9)

An AI system that is designed to function as a collaborative tool rather than a replacement is more likely to be adopted.

The novelty of AI systems makes OPOs hesitant to incorporate these systems in their business process without clear evidence of their effectiveness. Running AI systems in a test environment allows OPOs to observe the effects of AI systems on their business process and outcomes:

What would entice me to do it [adopt AI] is if we could have a simultaneously running test environment. We tend to adopt things and run it in real life and then see if it works. It would be really great if you have the live system in the test system and it runs both; and it can tell you, hey if AI was live, this is what would happen versus seeing the results of your current practice. (ID 8)

This suggests that OPOs expect tangible evidence of AI efficacy before fully integrating it into their existing process. Therefore, it is valuable to run the AI system in a test environment and adopt phased implementation strategies that allow stakeholders to observe the benefits of AI while the integrity of existing transplant operations is not compromised.

Culture

Based on the TOE framework, culture includes (1) top manager support, (2) change management strategy, and (3) innovative culture, all of which increase the possibility of AI adoption. There was strong agreement that top management support and innovative culture are critical to motivate AI adoption and appropriately allocate resources:

I think the biggest push would be coming from senior leadership, and I think they would help support it in every way possible. (ID 10)

While most OPOs emphasized the use of a change management strategy during AI adoption decisions, there was disagreement regarding how to approach AI training. First, there is an awareness problem both at the staff-level and for the public more broadly. This creates external pressures associated with AI adoption:

I think you know not everybody is aware of what AI actually is. […] we have to be conscious of how we communicate about what we're doing, so that people have a comfort level that we're using it appropriately. (ID 5)

Unclear understanding of how AI works, its benefits, and its risks require transparency and effective communications. There are a wide range of stakeholders, ranging from patients to government regulators to transplant surgeons to OPO staff, who need to be comfortable with the adoption of AI.

Second, OPOs disagreed about how much training should be required for staff to interact with AI systems. Some OPOs argued they only want an AI system that is easy and intuitive to use without training (see Table 1). This approach prioritizes ease of use for employees to effectively utilize AI systems. This suggests simplifying technology could reduce the barriers to adoption and foster immediate integration into workflows. However, there is a risk that easy-to-use AI systems may be perceived as better or be more convincing even if there is no evidence.45 Others believed that training about accuracy, fairness, and reliability was critical to support decision-making with an AI assistant. This training could range from an onboarding program to a public campaign to the inclusion of explainable AI interfaces. The quotes in Table 1 highlight the range of perceptions.

OPEN IN VIEWER

Table 1.

Example quotes for artificial intelligence (AI) training tension.

AI training is unnecessary	AI training is necessary
“I think it [use of AI] is intuitive. I mean, if you have a good AI system with an intuitive interface that is easy to use, I don't think you need any training. I think it is like opening a word document and typing once you know how to type. Then you can open a word document and use all of its features because it is all intuitive.” (ID 9)	“The other part [challenge] would just be educating people about the role of AI. What it is. What it isn't. Educating them about the controls that are in place so that there is some peace of mind that the decisions are appropriate and those types of things.” (ID 8)

For innovative culture, three OPOs mentioned that possessing a higher risk tolerance is a vital driver for AI adoption. However, financial allocation was a concern for all interviewed OPOs due to different underlying reasons. Some mentioned AI adoption as an intelligent risk:

I think you [OPO] have to have an organization that's willing to take intelligent risk and you have to have an organization that has the financial wherewithal to take that intelligent risk which is going to cost some amount of money. So, you've got to have the financial resources and the business acumen to take some risk because we don't know going in what we're going to get out of AI. But we can make some assumptions. (ID 1)

This shows a calculated approach where organizations should consider uncertainties before committing to AI investments, which have the potential to be high risk, high reward.

Organizational resources, size, and structure

Resource availability (e.g. budget, employees, and data) and organizational structure are influenced by organizational size. To explain the overlapping effect of organizational resources, size, and structure, all of these factors are discussed together in this section. For employees, there was disagreement about the need for AI expertise. This is summarized in Table 2.

OPEN IN VIEWER

Table 2.

Example quotes for artificial intelligence (AI) expertise tension.

AI expertise is unnecessary	AI expertise is necessary
“It [availability of knowledgeable employee] shouldn’t matter, if it's executed well. Good AI is going to look a little bit like having somebody sitting next to me doing it for me. So hopefully you're not relying on too much technical knowledge from the end user at that point.” (ID 8)	“Just expertise in the field. I mean, like I spoke to earlier, we don't have the knowledge inside the organization, and so to be able to partner with somebody who has the expertise is critical. Otherwise, we don't have the ability to implement.” (ID 1)

Smaller OPOs had both benefits and drawbacks compared to larger OPOs. Smaller OPOs tended to have fewer resources (e.g. budget and employees). However, even medium and large OPOs have budget constraints. OPOs are non-profit organizations that generate revenue based on gifts (i.e. donated organs):

Any money we make, any revenue we make, is from the gifts of others. So, we want to be careful that we practice good financial stewardship of that gift. (ID 9)

Therefore, there is an emphasis on good financial stewardship and may be an aversion to taking financial risks. For organizational structure, smaller OPOs tend to have reduced bureaucracy compared to larger OPOs:

I think in many ways it [being a small organization] is a blessing, because we don't have as many people to involve as if you were like in a big university hospital that has 7000 employees and layers of management. (ID 5)

For the tension around the need for AI expertise, OPOs disagreed about the importance of staffing AI experts (see Table 2). Some emphasized the importance of having AI-knowledgeable employees or a health informatics team to provide a foundation to adopt AI:

So within the organ procurement organizations having a health informatics infrastructure to know how to do this, whether it's 1 person or 2. Most OPO's don't have any. They'll have kind of like an IT person that is playing with the boxes and the wires and things, but then also have a responsibility to do the math, and that's not very effective. (ID 6)

However, the cost for these employees may be prohibitive in small organizations:

If it [an AI system] was a labor-intensive thing as well, that'd be something we'd have to consider if it took 5 or 6 or 7 people to run it. We'd have to adjust our staffing costs internally, this would be a pretty big adjustment. (ID 2)

Small OPOs are particularly concerned with staffing costs. Two OPOs felt that there was no need for these professionals. This raises concerns about the implementation of AI in smaller OPOs with fewer employee resources. As AI products develop, more high-quality off-the-shelf products will be available. But investments may be required during the transition period.

In addition to the availability of budget and the opposing visions for AI expertise, concerns regarding data sharing and inconsistency in terminology in the available data were raised. The following statement shows that all OPOs and transplant centers are required to use electronic medical records (EMRs) for consistency, but this requirement is not sufficient for data availability since donor hospitals avoid sharing and transferring this information:

There's no requirement that are really currently placed on donor hospitals to upload or share data electronically. We're required to have an EMR. They're required to have an EMR. But there's nothing that says that they transfer data to us electronically. (ID 5)

In addition to data sharing avoidance, there is an inconsistency in technical terminology in the transplant system:

I think one of the real problems on a national level is inconsistency in terminology and definitions. So different organizations, both in the OPO world and in the transplant world and in the government world, define a donor differently. (ID 5)

Competitive pressure

OPO performance is evaluated by CMS metrics to maintain a high donation and transplant rate. This creates competition amongst OPOs because the metrics are used to sort them into tiers (quartiles). OPOs in the top quartile can take over the territory of lower performing OPOs:

With the way the new CMS metrics have come out with donation rate and transplant rate and their tier [quartile] system, for that question [asking for naming competitors or benchmarks], the answer is always yes, there are competitors. Because if an OPO is in a tier two [second quartile] or tier three [bottom half] other OPOs or outside entities could apply for their territory. That being said, I think it's important for each individual OPO to drive excellence. (ID 4)

This competitive pressure and relative ranking system may drive AI adoption as each OPO has a desire to out-perform others. AI adoption is more likely when it offers a strategic advantage. This incentive scheme can drive innovation, but there are also concerns about equity implications if some OPOs lack resources for AI adoption.

Government regulations

Government regulations are likely to prevent OPOs from adopting AI systems. All OPOs mentioned that they are required to maintain compliance with regulations such as the Health Insurance Portability & Accountability Act (HIPAA), which includes data privacy, security, and confidentiality:

A second influencer from the environment is the medical legal aspects of Protected Health Information, […]. It's tightly regulated, and any kind of effort to hand to utilize that data certainly has to be evaluated against what the limitations and the allowances are for using it by HIPAA. The penalties for HIPAA violations are financially catastrophic in some cases for healthcare organizations. (ID 5)

Even if an AI system offers operational or clinical benefits, the legal and financial vulnerabilities and reputational risks create a barrier for AI adoption. Clear and consistent regulatory guidelines regarding AI use in healthcare need to balance promoting innovation with safeguarding patient confidentiality.

Industry requirements

The conflict between competitive and regulatory forces drives two opposing visions for AI adoption: (1) top-down adoption, where a regulatory agency facilitates all OPOs adopting AI at the same time, and (2) bottom-up adoption, where individual OPOs decide to adopt AI. While five OPOs preferred a top-down approach and three preferred a bottom-up approach, two OPOs identified value in both strategies.

The OPOs that favor a top-down approach expect to face strict regulatory boundaries that prevent the use of AI. This is partly related to concern about violating regulations and potential consequences. However, the need for consistency, unity, and fairness across OPOs is also a driving factor in preferring top-down adoption (see Table 3 for quotes). There is a sense that it is inappropriate to deviate from the traditional process without pre-authorization and without all OPOs implementing a new process at the same time.

OPEN IN VIEWER

Table 3.

Example quotes for the top-down versus bottom-up tension.

Top-down approach	Bottom-up approach
“[We] would be pleased to put this into place as long as the entire country that all the OPO's are doing it. So, everything I'm saying here is, it is predicated on the fact that no individual OPO could step out and do any of this stuff on their own without a national, organ procurement transplant network approved approach.” (ID 6)	“I think that the new metrics that OPOs are being held accountable by CMS certainly promote the adoption of AI to increase our transplant rate and our donation rate, […]. I think the pressure that those metrics put upon the OPO industry, that creates the space needed to adopt AI to improve practices.” (ID 1)
“The other thing you have to remember right now is allocation has very specific policies and procedures around it to ensure the justice and equity in the system the best we can. So, anytime an organ procurement organization goes outside that allocation system that's currently set up, there has to be evidence variation provided to UNOS to ensure that the best process was maintained for that. So, if you're talking about applying AI to change the list or change how you offer, you’d have to fix the system. Otherwise, every OPO is going to have to write a letter every time they allocated a kidney using AI if the current system is not adopted for it.” (ID 4)	“So, since we don't know whether AI is going to prove successful in terms of more organs are transplanted, and those transplants do well, we're going to go for it, we're going to be competitive. We're going to stay on the cutting edge of all technology.” (ID 7)

However, this approach to technology adoption is at odds with the competitive pressure that is also present. As described in the example quote in Table 3, there is also an expectation for OPOs to pursue innovative strategies to increase their donation and transplant rates. A bottom-up approach where individual OPOs are first adopters is consistent with the five stages of the diffusion of innovation theory46:

It is the same thing with everything else. There's going to be the diffusion of innovation curve. My organization would be on the front side of it. There are other organizations that would be on the last part of it, and then most of them will be in the middle. (ID 6)

Collectively, these opposing visions, top-down and bottom-up adoption, demonstrate the tension between regulatory constraints and competitive pressure. While the top-down approach is driven by concerns about equity implications, the bottom-up approach emphasizes innovation to maximize performance.

Customer readiness

Donor families, transplant candidates, and transplant centers are customers for OPOs. Donor families and transplant candidates suffer from the time-consuming allocation process:

Sometimes families walk away because they know it's going to take a while. So, if we could improve and speed up the process, I'd be curious to see if that would be impacted as well. (ID 10)

Improving and speeding up the allocation process can have significant benefits for both donor families and transplant candidates.

However, transplant centers are managing a high volume of work and need AI systems to facilitate processing. Transplant centers do not want AI to be an additional burden:

In organ allocation our customers are the transplant centers and their readiness. They are highly over capacity […]. Putting some integrated steps that would benefit them, I'm sure they would appreciate that. But the idea of new implementation into their workflow would be highly stressful. (ID 10)

AI should reduce unnecessary offers and speed up interpreting the valid offers. AI interventions offering a quicker and more effective allocation process are more likely to be adopted.