Is Watson for Oncology per se Unreasonably Dangerous?: Making A Case for How to Prove Products Liability Based on a Flawed Artificial Intelligence Design

A. Watson for Oncology

Doctors, hospitals, and healthcare providers are beginning to standardize the use of artificially intelligent medical devices in the treatment of patients. Examples of artificially intelligent medical devices include DeepMind by Google, ¹ Inner Eye by Microsoft, ² and Watson by IBM. ³ IBM's breakthrough technology, Watson, was thrust into the spotlight following its defeat of two longstanding Jeopardy! champions. ⁴ Although Watson has many uses, it has established a reputation for itself within the healthcare industry. ⁵

Watson Health is the subdivision of IBM that is responsible for developing and acquiring AI that can process massive amounts of health-related data—from clinical studies to skin images to genome sequencing—and creating tools that aid medical professionals. ⁶ WO was developed by Watson Health and is an AI device that makes cancer treatment recommendations. ⁷

A supercomputer, WO also employs AI to make cancer treatment recommendations through a multi-step process. ⁸ The process begins with a physician or other medical professional feeding patient data into WO. ⁹ After checking the accuracy of the data, the physician will “Ask Watson,” which prompts WO to analyze the patient data in order to make cancer treatment recommendations. ¹⁰ WO makes six color-coded recommendations. ¹¹ Two recommendations are highlighted with green, which means they are the preferred cancer treatments. ¹² The next two recommendations are highlighted with yellow, which means they are treatments that should be considered but are not preferred. ¹³ The final two recommendations are highlighted with red, which indicates that they are not recommended courses of treatment. ¹⁴

B. Behind the Black Box

On the surface, WO's decision-making process is simple. WO scans a fixed list of cancer treatments, from which it then eliminates the obvious courses of treatment that would be unsuccessful. ¹⁵ WO then arranges the remaining treatments according to the treatments' appropriateness for the patient and displays the treatments that made the final cut to the physician. ¹⁶ WO will, in conjunction with recommended treatments, offer supporting literature. ¹⁷ But what is unclear is how WO decides which cancer treatments to recommend – this is referred to as the black box effect. ¹⁸ An investigation into WO's decision-making capabilities reveals two flaws: its narrow scope of training and variable concordance rates.

It is important to uncover how WO is trained – in other words, making the black box effect less opaque – because patients deserve to know whether WO makes well-reasoned, supportable decisions and if those decisions are the same decisions a practicing oncologist would make. In configuring WO, IBM consulted Memorial Sloan Kettering Cancer Center in New York City to be WO's exclusive trainer. ¹⁹ As previously mentioned, WO scans a fixed list of cancer treatments before it provides the physician with the list of color-coded courses of treatment. WO scans a fixed list from which it chooses the final six recommended courses of treatment based on a multi-factor assessment. It is the doctors of Memorial Sloane who provided WO a set of factors to consider when forming a list of cancer treatments. Some factors include the patient's age, gender, treatment history, and tumor grade. ²⁰ However, WO's education did not include learning from patient outcomes. ²¹ According to Hogan and Swetlitz, WO is “Memorial Sloan Kettering in a box,” due to the fact that it received no other training. ²² All of the treatments WO can recommend are based entirely on the training provided by Memorial Sloane Kettering doctors - they determined what information WO needs to devise its guidance, which cancer treatment WO will recommend, and hand-selected the literature and statistics that would justify a given recommendation. ²³ The narrow breadth of training WO received seems to heighten the importance of clinical testing; the need to verify WO works well, is safe and recommends the best strategies for treatment. ²⁴

The effectiveness of WO is unproven because it makes suboptimal recommendations and it was not subject to clinical testing. ²⁵ If WO had been subject to clinical testing, it would be able to make treatment recommendations in a manner similar to real oncologists. But it does not. Three of the problems with WO's recommendations is that they are based on a limited set of data, the research WO relies on to make recommendations is not current, and WO cannot learn which recommendations effectively treat certain types of cancer since it is not on a network. ²⁶ Oncologists, however, can consult both past and present medical research and are not only able to rely on their past experiences but can also draw from their colleagues' experiences as well. Even though WO does not make the final decision on how to treat a patient's cancer, the effect of introducing WO into a patient's care is that it constricts the range of treatments and research the oncologist may have taken account without WO. ²⁷

Moreover, since WO was not subject to clinical testing, the only proof of how it performs is its concordance rates. A concordance rate is a study where an oncologist will “Ask Watson” for advice about how to treat a set of patients, and then compare WO's recommendation to those of real oncologists. ²⁸ WO concordance rate studies, with the exception of one study, have been conference abstracts that are paid for by a customer or are conducted or authored by IBM; this means that the vast majority of studies on WO are not verified. ²⁹ In addition, WO's concordance rates are widely divergent. WO's concordance rate ranges between 33 percent and 96 percent for some cancers. ³⁰ This means that, if a cancer patient enters a doctor's office who uses WO to make cancer treatment recommendations, there is thirty-three to ninety-six percent chance the course of treatment WO recommends is one an actual oncologist would recommend. If WO made recommendations that were either highly consistent with oncologists (a ninety-six percent concordance rate) or highly inconsistent with oncologists (a thirty-three percent concordance rate), then an oncologist could make up for any deficiencies in WO's decision making. But oncologists do not have this opportunity because the concordance studies are unverified. WO also has no connection to a network so it cannot identify trends in the treatment of specific types of cancer or patient outcomes. ³¹ In short, it cannot learn – leaving patients to roll the dice and hope WO recommends an optimal course of treatment.

The final issue with WO is that its design creates practice-setting biases as the treatment recommendations WO makes do not always comport with the practices of doctors in the United States or globally. ³² For instance, Denmark reported a concordance rate of 33 percent ³³ while the American Society of Clinical Oncology reported a 96 percent concordance rate. ³⁴ The concordance rate for oncologists in Korea is 41.5 percent. ³⁵ It is likely that, as a result of the discrepancy in WO's concordance rate, a patient could endure a reduction in life expectancy or die due to a WO recommendation. This note recommends that the patient should bring a products liability claim against IBM based on WO's faulty design. ³⁶

A. Ways in which AI Challenges the Existing Legal Framework

AI machines challenge the traditional legal doctrines of negligence, such as products liability, vicarious liability, and medical malpractice, because AI makes it difficult to determine who caused the injury in question. The 2010 Flash Crash exemplifies how AI makes it difficult to discern causation. Navinder Sarao was a securities trader who relied on an algorithm to place and cancel $200 million worth of trades in seconds, a practice referred to as “spoofing.” ³⁷ This resulted in the orders trading 19,000 times, which is alleged to have caused a trillion-dollar market crash. ³⁸ There are at least three competing theories that purport to identify the cause of the Flash Crash. Fist, the SEC found that Sarao was at least significantly responsible for causing the crash. ³⁹ The next agency that set out to explain the Flash Crash is the Commodities and Futures Trading Commission (CFTC). Initially, the CTC did claim high-frequency traders like Sarao, did cause the crash, but later came to the conclusion that Sarao was not the sole or a significant cause of the crash. ⁴⁰ Other theorists suggest that a lack of circuit breakers in the market is to blame while others claim the exchanges simply could not handle the volume of trades and thus malfunctioned. ⁴¹ The array of explanations for how and why the 2010 Flash Crash occurred signifies how difficult it is to point to a particular instance of misfeasance that caused the harm in question.

In the end, only Sarao faced civil charges. ⁴² But it is not obvious that his actions caused the Flash Crash. The divergent accounts of what caused the Flash Crash perfectly demonstrates how difficult it is to discern who caused the crash – trading algorithms or individual actors. The use of artificially intelligent medical devices forces judges to confront the same daunting task: how to identify the cause(s) of a patient's injuries and appropriately attribute blame.

B. Solutions Proposed by Other Scholars

Scholars have proposed various solutions that aim to remedy the challenges artificially intelligent medical devices pose in the context of negligence suits. In From Jeopardy to Jaundice: The Medical Liability Implications of Dr. Watson and Other Artificial Intelligence System, Jessica Allain offers a streamlined approach to assessing liability against AI systems. ⁴³ In particular, Allain recommends combining elements from medical malpractice, vicarious liability, products liability, and enterprise liability as a means to address liability that arises from AI systems. ⁴⁴ Her claim is that this model will eliminate any inequities, like litigants receiving disparate dispositions or manufacturers facing the brunt of liability, that may arise from courts applying different theories of liability. ⁴⁵ She notes that patients cannot bring product liability claims against WO for informational defects because the learned intermediary doctrine would bar litigants from suing manufacturers. ⁴⁶ Allain also states that successfully suing for products liability due to design defect is complicated by the fact that the theory only addresses liability that results from hardware defects, which makes making deciphering WO's complex design an even more formidable challenge for courts. ⁴⁷ The approach Allain recommends for handling causes of action that arise from AI systems would entail courts deciphering the direct cause of the patient's injuries, then classifying the matter as a medical malpractice or products liability case. ⁴⁸ If the direct cause is the AI hardware, then the claim would proceed as a products liability case. ⁴⁹ Otherwise, it would proceed as a medical malpractice claim under the theory of enterprise liability. ⁵⁰ Allain claims her two-step approach encourages the use of AI systems, while minimizing ambiguities and equally distributing losses. ⁵¹ For example, if a case proceeds under enterprise liability and not Allain's combined theory, an economic disincentive for WO's use would be created since the head of the business organization in question would face a disproportionate amount of liability. ⁵² Minimizing ambiguities and equally distributing loses are important goals because they will encourage the adoption of emerging AI technology. ⁵³

In Machines without Principals: Liability Rules and Artificial Intelligence, David Vladeck focuses on theories of liability as they relate to machines that are fully autonomous, like self-driving planes and cars. ⁵⁴ Vladeck fleshes his argument out through analogy. He imagines Google's self-driving car acting in ways that the manufacturers, designers, and others did not foresee, ultimately harming individuals. Vladeck proposes that the doctrine of res ipsa loquitor (“RIL”) should apply when the accident can, in some way, be traced back to the human hand. ⁵⁵ Thus, in the instant analogy, Google would be held liable on the presumption that the accident is prima facie evidence of a design error. ⁵⁶ Vladeck then goes on to state that, as it relates to truly autonomous technology that acts outside of the rules provided to it by its manufacturers and designers, a strict liability rule should apply. ⁵⁷ Proceeding under Vladeck's proposed theory eliminates any unfairness to plaintiffs who are injured through no fault of their own, places the burden on AI manufacturers since they presumably in the best position to absorb the costs, reduces the expenses associated with litigating liability, and encourages innovation by creating a predictable liability regime. ⁵⁸

Sjur Dyrkolbotn, in A Typology of Liability Rules for Robot Harms, argues that the existing legal doctrine of products liability can sufficiently address harms attributed to AI systems. ⁵⁹ Dyrkolbotn explains that the real challenge to successfully proving a products liability claim against an AI system will be litigants' ability to provide enough evidence that helps the judge and jury decipher the causal chain in question and whether the harm was actually caused by the artificially intelligent machine. ⁶⁰ To aid in this difficult task, Dyrkolbotn recommends that AI be leveraged to understand the deontic workings of autonomous technology. ⁶¹ Dyrkolbotn is suggesting that AI responsibility tracking tools can help courts distill legally and ethically relevant information from complex causal chains involving intelligent systems. ⁶² He envisions an AI algorithm that can analyze the log of computations a medical device performs and map the human chain of events, distill the information, and the AI algorithm would then explain the data in a manner that would help humans assign liability. ⁶³

C. Redefining Unreasonable Danger

Vladeck's and Allain's theories aim to resolve the tension between tort law and technology humans have yet to encounter – fully autonomous AI systems. This article, on the other hand, offers a solution for today; one that provides litigants a means of suing manufacturers if they are injured by a system like WO that is only partially autonomous. ⁶⁴ While Dyrkolbotn's theory does not target fully autonomous AI systems and he does believe the existing products liability regime can support a claim against an AI system, he argues that only AI responsibility tracking can help plaintiffs prove causation. I disagree that plaintiffs should have to wait until some future date when AI can help them prove the causation, they need a solution now. Both Dyrkolbotn and I agree that plaintiffs face a heavy burden because of how cumbersome it is to obtain evidence that demonstrates the causal chain of events that lead to the harm and what flaw in the hardware of the AI proximately caused the plaintiff's injuries. ⁶⁵

The evidentiary burden a plaintiff must overcome to prevail in a products liability suit against an AI system that proceeds on the theory of defective design is eloquently presented by Dyrkolbotn. Given the fact that most plaintiffs' have limited resources and lack access to key information, the evidentiary burden they face is palpably unfair. Some would argue that, in a case where WO led to a patient's death or reduced life expectancy, a plaintiff tort lawyer would take the case on a contingent fee basis. Even if a plaintiff is lucky enough to retain an attorney on the contingent fee basis, there is still an informational hurdle to overcome. As described in Part I, subpart c, the effectiveness of WO remains unproven because it was not subject to clinical testing. The only public data available are concordance rate studies. But all of the concordance studies, except one, are controlled or influenced by IBM and published in conference abstracts. The effect is that no one has data that is objectively reliable and explains how WO makes decisions. In essence, IBM has a monopoly on information plaintiffs need.

This article proposes that, when proceeding in a products liability suit on the theory of defective design, WO presents an opportunity for the legal community to adopt a broader definition of what it means for a device to be unreasonably dangerous for ordinary or foreseeable use. ⁶⁶ Courts should adopt a rebuttable presumption or per se rule that holds that an artificially intelligent medical device is unreasonably dangerous when a manufacturer intentionally designs the product in a manner that causes it to yield disparate results simply by virtue of the medical setting in which it operates. WO presents an opportunity to apply this rebuttable presumption. As previously stated, WO was trained by a single hospital, is able to draw only from a limited number of cancer treatments recommendations, and is not designed to learn from patient outcomes or identify trends. ⁶⁷ WO's varying concordance rates are proof that it was designed in such a way as to render it unreasonably dangerous, as explained below. ⁶⁸

A. Why Products Liability?

In the event a plaintiff is injured by WO, she could sue under one of several liability regimes. A plaintiff may seek recompense for the injuries she suffered by suing the United States Federal Drug Administration (FDA), suing the hospital or healthcare institution under a theory of vicarious liability, suing the doctor or team of doctors for medical malpractice, or suing IBM for products liability. As is explained below, a plaintiff is most likely to recover by suing under the theory of products liability.

One way that the United States minimizes injuries to consumers of products is through regulation. ⁶⁹ The Federal Food, Drug, and Cosmetic Act (FDCA) endows the FDA with regulatory authority over medical devices. ⁷⁰ Since WO is considered a type of medical device, ⁷¹ it falls within the FDA's authority. ⁷² A medical device is defined as an instrument, apparatus, implement, machine, contrivance, implant, in vitro reagent, or other similar or related article, including any component, part, or accessory, which is–

Federal Food, Drug, and Cosmetic Act, 21 U.S.C. 321(h). Because WO recommends cancer treatments, it would fall squarely under subsection (2) of the FDCA's definition. An AI developer would then seek to classify their device as a Mobile Medical App, Medical Device Data System, Software as Medical Device, or Clinical Decision Support Software (CDSS). ⁷³ A device's classification dictates whether it is considered a medical device, subject to enforcement discretion, or is a Class I, II, or III device. ⁷⁴ Normally, WO's classification as a CDSS device ⁷⁵ would mean it is subject to regulation, but it was exempted from regulation in 2016. The 21^st Century Cures Act legislatively exempts certain devices by clarifying that the term “device” does not include software that supports or “provides recommendations to a healthcare professional about prevention, diagnosis, or treatment of a disease or condition.” ⁷⁶ In short, WO is not subject to regulation. Further, even if WO were regulated, a plaintiff could not sue the FDA for her injuries. ⁷⁷ It is well established that a tort case can only be brought against an individual government actor, but not against the government itself. ⁷⁸ Therefore, absent a named individual and corroborating evidence, a plaintiff is not likely to recover by suing the government.

In search of another means of recovery, the plaintiff may sue the hospital or healthcare institution that employs the oncologist. In the medical context, vicarious liability focuses on institutions that employ healthcare providers ⁷⁹ and is a liability regime that holds the institution responsible if an oncologist violates her duty of care. ⁸⁰ The healthcare institution can be held liable under the theory of “respondeat superior,” which perceives the oncologist as an agent of the institution. ⁸¹ Therefore, when a physician acts on behalf of and in advancement of the institution, the institution can be held liable for the agent's negligent acts. ⁸²

A plaintiff's ability to recover against the healthcare institution depends on several factors: (1) the type of hospital, (2) the type of employment relationship, ⁸³ and (3) establishing the oncologist committed malpractice. ⁸⁴ Hospitals are either privatized or partially or wholly owned by the federal government. ⁸⁵ However, hospitals that are either partially or wholly owned by the government are generally exempt from liability due to sovereign immunity. ⁸⁶

Presuming the hospital in question is privately owned, ⁸⁷ a plaintiff would then have to demonstrate the employment relationship between the oncologist and institution. If the oncologist was a “servant” or “agent” of the hospital, then liability can be found. ⁸⁸ This prong is not unnecessarily difficult to prove since many jurisdictions do not require a plaintiff provide positive proof that the physician in question was an employee of the hospital if a reasonable person would presume the physician was providing medical treatment on behalf of the hospital ⁸⁹ or if the patient sought services from the hospital and not an individual physician. ⁹⁰

The final, and most difficult, hurdle the plaintiff has to overcome is proving the oncologist committed malpractice. ⁹¹ A plaintiff is not likely to prove that the doctor committed malpractice because it is nearly impossible to prove that it was the doctor who, and not WO, that caused the plaintiff's injuries; this is due to the fact that WO does not disclose its decision making and was not subject to clinical testing, so it would be difficult for a plaintiff to find evidence of how WO operates or its common flaws. ⁹² Even when it is clear either that the physician or the AI device caused the harm, courts still have refused to find liability. ⁹³ For example, the Mracek court declined to find a manufacturer liable, despite ample evidence that the surgical AI device malfunctioned, simply because the claimant did not submit expert testimony critical of the AI device. ⁹⁴

Another option an injured plaintiff may contemplate is a medical malpractice suit. The legal theory of medical malpractice is a subset of negligence law that requires physicians to exercise reasonable care when treating patients. ⁹⁵ In the context of WO, a claim might arise if an oncologist or other medical professional failed to act as a reasonably prudent physician. ⁹⁶ In a traditional medical malpractice case, a plaintiff would need to show there was a physician-patient relationship ⁹⁷ and that they either did or did not provide their informed consent. ⁹⁸ If those elements are satisfied, liability then turns on whether the oncologist conformed with certain acceptable standards of reasonable care during treatment and whether the oncologist is the cause of the plaintiff's injuries. ⁹⁹

To understand why products liability is the best theory to recover against WO one needs to understand how a medical malpractice suit would proceed if WO were involved. To be clear, it is more difficult for the plaintiff to prevail when an artificially intelligent medical device is involved. Because WO is designed to make cancer treatment recommendations for the treating oncologist's consideration and does not independently treat patients, courts would likely analogize WO to a consulting physician. ¹⁰⁰ A medical malpractice claim can be established against a consulting physician only if a physician-patient relationship existed. ¹⁰¹ Therefore, a plaintiff would have to prove she was in a physician-patient relationship with WO to win her suit. Current jurisprudence suggests that because WO does not interact with patients, WO would not owe any duty to the plaintiff; ¹⁰² and thus, a plaintiff could not establish the requisite relationship necessary to prevail.

Another issue a plaintiff would have to confront when bringing a medical malpractice suit against WO is proving that WO is capable of committing a negligent act. ¹⁰³ The court would have to try and engraft the prevailing standard of care onto a sophisticated, artificially intelligent machine that only makes decisions with minimal risk based on a system of algorithms. ¹⁰⁴ With this in mind, the court would have to try and understand whether its systems of algorithms would permit it to act unreasonably. Because the black-box effect obscures WO's inner workings, ¹⁰⁵ a court may not be uncomfortable finding that WO did commit a negligent act since there is no clear way to know – further diminishing the plaintiff's ability to prevail in a medical malpractice suit.

In sum, a plaintiff who has been injured by WO does not have legal recourse to sue the FDA, is unlikely to recover against the hospital through vicarious liability because proving the oncologist caused the plaintiff's injuries is a virtual impossibility and is unlikely to prevail in a medical malpractice suit against WO because a physician-patient relationship cannot be shown. The final option available to an injured plaintiff is to bring a products liability suit against IBM. There are several reasons why bringing a products liability suit is preferable. First and foremost, a plaintiff is best positioned to recover for her injuries by bringing a products liability claim because it is an actual legal possibility as compared to suing the FDA. ¹⁰⁶ A plaintiff also would not have to establish a physician-patient relationship as would be the case in a medical malpractice suit. ¹⁰⁷ When comparing the likelihood of recovery by proceeding under the theory of products liability to vicarious liability, it is arguable that they are equally disadvantageous. Allain would likely contest that both products liability and vicarious liability suits present the same burden – namely, that a plaintiff must be able to articulate and convince the court that the artificially intelligent device caused her injuries. ¹⁰⁸ While it is true that in both scenarios a plaintiff would have to prove her injuries were caused by WO, the conceptual difficulty in convincing the court of such is lower in a products liability case. In a medical malpractice action, the plaintiff would have to explain exactly how WO caused her injuries despite its obscure nature ¹⁰⁹ and that it was WO's hardware and not the software that failed. ¹¹⁰

But the plaintiff is more likely to prevail if she brings a products liability suit pursuant to my proposed theory that WO is unreasonably dangerous because it was designed in such a way as to defy consumers' expectations that it is uniformly used or applied by medical practitioners. Through shifting the focus of the causality inquiry to the methods used to train and design WO, the plaintiff avoids the need to explain WO's decision-making process. Courts and juries can understand the consequences of process and methodology, and therefore would presumably be more willing to find liability because those are concepts that are within their everyday experiences. It is also reasonable to presume the court would be comfortable finding liability based on the aforementioned theory because doing so would be similar to finding liability based on the doctrine of RIL. ¹¹¹ The only mystery in the plaintiff's case is WO. Presuming the treating oncologist can prove they are not liable, all signs point to WO even though the plaintiff would be unlikely to identify a particular failure; consider it - there is no evidence that WO is effective, it operates disparately in different settings, and cannot learn from its mistakes – what else could have caused the plaintiff's injuries except WO. Moreover, if the court is willing to take notice of the great disparity between WO's concordance rates, ¹¹² it would have some evidence that WO does not perform uniformly and thus violates consumer's expectations. The final advantage of bringing a suit under the proposed theory is that it would save both the plaintiff and defendant litigation cost by eliminating reliance on expert testimony and avoiding the same fate as the plaintiff in the Mracek case. ¹¹³

B. Making the Case: Suing IBM for Products Liability

The theory of products liability makes entities that engage in the business of selling or distributing products legally responsible for harms to persons or property caused by defects. ¹¹⁴ There are three ways a product is considered to be “defective.” A product is defective if, at the time of sale or distribution, it contains a manufacturing defect, design defect, or was operated improperly due to inadequate instructions or warnings. ¹¹⁵ A plaintiff suing IBM is not likely to recover if she predicates her case on a manufacturing defect because the manner in which WO performs is consistent with its intended design. It is also unwise for a plaintiff to sue IBM for their failure to adequately warn or provide instruction because the “learned intermediary” doctrine shields manufacturers from this type of liability. ¹¹⁶ As it applies to medical devices, manufacturers are considered to have discharged their duty once they adequately warn the treating physician about the risks the medical device may pose. ¹¹⁷ Thus, the plaintiff must predicate her products liability suit against IBM for WO's defective design. ¹¹⁸ It is elaborated below that, to prevail on her products liability claim sounding in negligence, ¹¹⁹ the plaintiff must prove that WO's design is defective, ¹²⁰ that the injuries the plaintiff suffered were reasonably foreseeable, ¹²¹ and that WO's defective design is what caused her injuries. ¹²²

C. Is Watson for Oncology Defective?

In bringing a products liability claim against IBM for WO's faulty design, the essential question the plaintiff must answer is whether the design of WO creates an unreasonable risk or risks. ¹²³ The plaintiff can satisfy this inquiry by alleging and proving (i) that WO was defective at the time it left the control of IBM, ¹²⁴ (ii) a reasonable alternative design would have reduced the foreseeable risks of harm posed by WO, (iii) and that IBM's failure to adopt an alternative design rendered the product not reasonably safe. ¹²⁵

The first element the plaintiff must prove, by a preponderance of the evidence, is that the product was defective when it left the control of the defendant-manufacturer. ¹²⁶ This means that our plaintiff must prove that WO became defective while in the control of IBM, ¹²⁷ and that the defect did not arise after IBM relinquished control of WO. ¹²⁸ To date, IBM nor any physician has been sued due to WO injuring a patient; thus, our plaintiff is unlikely to have circumstantial evidence that the defect occurred while under IBM's control. However, the court should infer that WO's defect existed when it was in IBM's control based on its nature. ¹²⁹ The court should infer that WO was defective when it left IBM due to its design. Because WO is designed as a closed system – i.e. the information upon which WO relies to make decisions is finite and cannot be altered by the treating physician. ¹³⁰ Courts have applied this logic in cases involving products that are sealed when packaged. In Darryl v. Ford, the plaintiff sued Ford Motor Company for damages suffered as a result of a faulty brake system. ¹³¹ The court made the inference that the braking system was defective at the time it was under the control of the defendant-manufacturer and that its condition was not substantially changed because the braking system was in a sealed container. ¹³² Courts have also made the same inference in the case of packaged pork infested with trichinae based on the well-known fact that trichinae inhabits the live animal, and therefore the pork could not have been contaminated later. ¹³³ Applying the same logic to WO, the court should find WO became defective while under the control of IBM because it is a closed, static system that therefore could not have been altered once it left IBM's auspices.

Next, the plaintiff would need to establish the basis for declaring that the design of WO is defective. The Restatement Third of Torts describes a product's design as defective

To satisfy her burden, the plaintiff would normally have to present evidence of a reasonable alternative design that would have reduced or avoided the injuries she suffered. ¹³⁴ There is no definition of reasonable alternative design, but courts may consider different factors like industry standards, regulatory compliance, or remediation efforts. ¹³⁵ However, courts also permit plaintiffs to rely on circumstantial evidence, as opposed to providing a reasonable alternative design, to prove that WO's design is defective. ¹³⁶ The evidence presented would need to support the inference that the harm the plaintiff suffered was of a kind that ordinarily occurs as a result of a product defect, and was not solely caused by sources other than the product defect. ¹³⁷

D. FORESEEABILITY

To satisfy the second prong of her claim, our plaintiff must prove that the dangers posed by WO were foreseeable. ¹⁴⁹ The Restatement Third of Torts goes on to explain that because a reasonable design still presents risks, “the balancing of risks and benefits in judging product design and marketing must be done in light of the knowledge of risks and risk-avoidance techniques reasonably attainable at the time of distribution.” ¹⁵⁰ In essence, the plaintiff needs to convince the trier of fact that, after balancing the costs and benefits of an alternative design, the risk of harm posed was foreseeable and could have been reduced or avoided. ¹⁵¹

Assuming that IBM will deny that the injuries the plaintiff suffered were foreseeable and could have been avoided, the plaintiff can introduce the concordance rates of WO and the STAT study to show that IBM was on notice of the harm WO could cause. ¹⁵² IBM took WO on a world tour – oncologists in 50 hospitals on 5 continents are using the supercomputer. ¹⁵³ The way a medical institution acquires WO is through a consultation, where the entity will attempt to integrate WO into their systems to further understand if it is capable of what has been purported. ¹⁵⁴ Whether or not the outcome is positive, the medical institution will typically report its findings. ¹⁵⁵ Therefore, it can reasonably be presumed that both the general public and IBM are on notice of how WO is performing. A problem exists, however, with which of those studies are published and available to the public. IBM controls the information that is released in regard to WO's success and is the only entity that truly knows WO's design, and as such it has the power to selectively release and cite data. ¹⁵⁶ The result is that IBM has withheld studies that show WO has a low concordance rate. For example, when WO was tested in Denmark its concordance rate was 33%, yet that study cannot be accessed. ¹⁵⁷ Due to the fact that these studies are inaccessible, the plaintiff's use of WO's concordance rate is limited to relying on the concordance rate itself, as opposed to an explanation of how WO's design led to those results. The concordance rates can only be used to demonstrate that IBM knew that WO did not have a consistent standard level of agreement with oncologists, which suggests WO is unreliable.

Due to the fact that the concordance rate only shows that IBM was on notice that WO is unreliable, the plaintiff must introduce the STAT study to demonstrate what risks were foreseeable. STAT is a news outlet that specializes in investigative journalism focused on the life sciences and pharmaceuticals. ¹⁵⁸ In IBM pitched its Watson supercomputer as a revolution in cancer care. It's nowhere close, STAT uncovered the shortcomings of WO in the context of IBM's marketing scheme. ¹⁵⁹ The reality is that WO is still “struggling with the basic step of learning about different forms of cancer” ¹⁶⁰ three years after being introduced into the market, yet IBM continues to only publish reports that cast WO in a glowing light. ¹⁶¹

There are several facts that, at a minimum, prove IBM was on notice that WO posed a risk to patients. For example, it is well documented that it is difficult to constantly update WO with new studies and courses of treatment; that the extent to which a hospital relies on the treatment recommendation WO provides varies by hospital staff; and that WO does not properly abstract patient data from medical records. ¹⁶² Our plaintiff would argue that this means that, after receiving incomplete patient information, WO recommends a course of treatment based on incomplete and outdated research and studies. The effect of this is that oncologists will not consider a course of treatment they likely would have if they researched on their own. Moreover, not all of the treatments WO recommends are backed by evidence i.e. scientific journals or other research. ¹⁶³ Therefore, our plaintiff can argue IBM is at least on notice that WO deprives oncologists of the panoply of available cancer treatments and does so, not on the basis of clinical trials that track survival rate, but the guesswork of a single group of oncologists.

E. CAUSATION

Although the Restatement Third of Torts did not draft a set of causation standards unique to products liability, ¹⁶⁴ case law is instructive as to how causation standards apply to AI machines. One leading case involving an artificially intelligent machine is Mracek v. Bryn Mawr Hosp., which is a products liability action brought against Intuitive Surgical, Inc. (Intuitive). The plaintiff in Mracek brought an action against Intuitive for damages arising out of da Vinci's malfunction. ¹⁶⁵ During Mracek's prostatectomy, da Vinci displayed multiple error messages and was rebooted numerous times, which he alleges caused him to suffer from erectile dysfunction (ED). ¹⁶⁶ At trial, Mracek contended that it was obvious that da Vinci caused his injuries because the treating physician would testify that da Vinci displayed error messages and had to be rebooted. ¹⁶⁷ The court found this testimony insufficient for two reasons: (1) the technical complexity of da Vinci is not within the common knowledge of the jury and (2) the physician's testimony is the only narrative, did not opine as to a defect in da Vinci, and did not link da Vinci's malfunctioning to the plaintiff's injuries. ¹⁶⁸ Thus, our plaintiff would need to produce a causation expert to explain how WO was the actual and proximate cause of her injuries.