Integrating Artificial Intelligence-Based Simulation into Serious Illness Conversation Training: A Pilot Medical Student Intervention

Abstract

Background:

Serious illness conversations (SICs) are an important part of medical care, yet trainees continue to express discomfort. Educational opportunities are time and resource-intensive. This pilot study's aim was to explore the experience of artificial intelligence (AI) technology deployed for SIC practice and feedback.

Methods:

2-Sigma, an AI-based simulation platform, was deployed as a tool for medical students to practice SICs utilizing skills taught through a dedicated Advance Care Planning (ACP) curriculum. Eleven medical students completed 27 cases and full transcripts were reviewed to describe (1) student interactions with AI, (2) AI responses, and (3) AI-provided feedback.

Results:

Students interacted with AI utilizing skills taught in the ACP curriculum. AI responses displayed emotion, however, quickly resolved negative feelings. Feedback addressed important components of SICs but varied in its specificity.

Conclusion:

AI technology represents a potential tool for practicing skills necessary for SICs.

Keywords

artificial intelligence goals of care communication medical student education simulation training serious illness conversation

Background

Effective serious illness conversations (SICs) are critical to ensure patients receive goal-concordant care as their disease progresses.¹ Structured curricula designed to teach SIC skills frequently utilize standardized patients (SPs), role-play, observation, and utilization of conversation guides.^2–5 Additionally, trainees report that SIC educational experiences primarily occur through observation and didactics.^5,6 Despite these educational experiences, trainees and physicians continue to express discomfort having SICs.^1,2,7,8

Patients and families prefer providers who are empathic, trusted, and skilled in SICs.^9–11 Patients also desire receiving serious information while having their emotional needs addressed.⁹ While best practice in empathic communication is addressing emotion, physicians commonly respond with medical data or do not acknowledge the emotion.^12–14 Our current medical student curricula utilizes didactic, role-play, and SP encounters to build these skills, yet resource constraints limit expansion.⁴

Recent advances in artificial intelligence (AI) offer potential scalable methods to allow trainees to practice SICs in a safe environment.¹⁵ Building on our established framework for responsible AI integration in medical education,¹⁶ this pilot study explores the feasibility of using AI as a modality to practice SICs while also identifying opportunities to refine AI’s responses for a more realistic experience.

Methods

Study population and setting

Study participants were fourth-year medical students participating in a previously established 2-week elective including a dedicated Advance Care Planning (ACP) curriculum.³ Students practice SICs with an SP at both the beginning and end of the elective.

Pilot intervention

2-Sigma, an AI-based simulation platform that incorporates adaptive clinical scenarios, was developed at UCCOM by LT and SO.¹⁷ The 2-Sigma platform utilizes large language model architectures with chain-of-thought and few-shot prompting techniques to simulate realistic virtual patient encounters and deliver immediate AI-generated feedback. 2-Sigma was instructed to create a GOC case, provide only necessary background information, share information only when asked, and require students elicit details on goals, priorities, and concerns before the AI was willing to make medical decisions. It was instructed to provide feedback on skills such as assessing patient understanding, using validating statements, eliciting patient goals and concerns, and indicating future steps.

Students were prompted to identify a specialty or scenario to generate the case. AI simulated a patient while the student acted as their physician. Students read AI responses and typed or used voice-to-text to respond. At the end of the conversation, AI provided text feedback.

Data collection and review

Two investigators (D.K., D.E.W.) utilized deductive thematic analysis to independently review transcripts to explore how students utilized SIC frameworks when interacting with the AI. The same investigators utilized inductive thematic analysis of transcripts and AI feedback to develop codes and subsequently met to develop themes and reconcile any differences. A third investigator (L.A.H.) subsequently reviewed transcripts to ensure no other codes or themes arose. Microsoft Excel was used to organize data.

The analysis aimed to (1) understand how students incorporated curricular content into interactions with the AI, (2) describe strengths and limitations of the AI-generated content to inform iterative improvements to the prompt engineering, and (3) evaluate the AI’s ability to provide effective feedback. The two investigators then met to reconcile differences and discuss emerging themes and patterns. A third investigator (L.A.H.) subsequently reviewed all transcripts to validate results through investigator triangulation.

Results

Participation

Eleven students participated in a total of 27 SIC cases with AI feedback being generated for 24 of the cases. All authors compared the student prompt with AI-generated clinical scenario and case goal to ensure face validity (Table 1).

Table 1.

Generated Case and Goals Based on the Prompt Submitted by the Learner. Completed Cases Included AI Generated Feedback

	Learner prompt	Case description	Case goal	Outcome
1	Abnormal imaging finding, emergency department	52 yo male presenting to the Emergency Room with chest pain	Share news about abnormal CT scan finding concerning for lung cancer	Completed
2	Family medicine	45 yo male with polyuria and polydipsia	Share news of new diagnosis of diabetes	Completed
3	Abnormal imaging finding	52 yo with headaches and dizziness for a few months	MRI results concerning for glioblastoma multiforme	Completed
4	Abnormal imaging	52 yo who presented for routine check-up described symptoms of weight loss and abdominal pain over a few months	Follow-up visit for abdominal imaging concerning for pancreatic cancer	Completed
5	Treatment no longer working, cancer	55 yo undergoing treatment for colorectal cancer for 2 years now with progressive disease	Discuss results of tests and explain treatment isn’t working	Completed
6	I want to generate a random case about sharing bad news after a anesthesia operative complication	45 yo undergoing elective laparoscopic cholecystectomy due to symptomatic gallstones. During surgery she had a period of hypoxia and postop having mild cognitive deficits and memory issues which are thought to be transient	Tasked with explaining this complication with the family	Completed
7	Random case	52 yo with fatigue and unexplained weight loss found to have adenocarcinoma of the colon	Discuss additional finding of having metastatic disease to the liver	Completed
8	Abnormal imaging finding	52 yo who presented with persistent cough and shortness of breath with history of smoking. CT scan demonstrated a mass concerning for cancer	Discuss suspected diagnosis and the need for additional evaluation and biopsy	Completed
9	Patient experienced herniation after anoxic brain injury secondary to opioid overdose	32 yo brought to ED after being found unconscious and is intubated. Repeat head imaging shows evidence of herniation with severe and likely irreversible brain injury	Share current medical situation and prognosis with patient’s sister	Completed
10	Pediatrics case	8 yo with persistent fevers, fatigue and bruising found to have a new diagnosis of ALL	Provide diagnosis to family	Completed
11	Cancer diagnosis	55 yo who presents for follow up after a biopsy and work up found to have Stage 2 non-small cell lung cancer	Share test results	Aborted
12	Sharing cancer diagnosis	52 yo woman presenting with night sweats and weight loss returning for results after a lymph node biopsy	Share news of diagnosis of non-Hodgkin lymphoma	Completed
13	Family medicine	45 yo with polyuria and polydipsia	Share diagnosis of type 2 diabetes mellitus	Completed
14	Neurology	58 yo presenting with 2 months of headaches, dizziness, and confusion	Share new diagnosis of metastatic brain tumor	Aborted
15	Neurology	35 yo presenting with 6 months of leg numbness, blurred vision, and extreme fatigue	Share new diagnosis of multiple sclerosis	Aborted
16	Discuss whether to continue chemotherapy or not	68 yo male with advanced pancreatic cancer with disease progression on 1^st line chemotherapy	Discuss whether or not to continue chemotherapy	Completed
17	Family medicine	70 yo with advanced COPD experiencing difficulty managing symptoms and several admissions the past year	The patient wants to discuss future healthcare plans and understand options specifically surrounding hospitalizations and invasive procedures	Completed
18	Neuro	68 yo with advanced glioblastoma multiforme with progression of disease and symptoms	Discuss goals of care	Completed
19	Whether desire surgery or not	76 yo with early-stage Alzheimer’s who was recently diagnosed with colon cancer	Discuss possibility of surgery to cure cancer	Completed
20	Random	68 yo with congestive heart failure with frequent hospitalizations and frustration with current condition	Discuss goals of care	Completed
21	Whether desire surgery or not for a patient with likely incurable cancer	68 yo with recent diagnosis of advanced pancreatic cancer being offered a palliative surgery to improve symptoms	Discuss whether or not patient would want surgery	Completed
22	Random	75 yo with advanced COPD admitted several times over the past year	Discuss goals of care specifically if patient would want intubation or tracheostomy	Completed
23	Family medicine	68 yo with COPD with frequent admissions who is being seen by his PCP for a hospital follow-up	Goal is to address overall goals and possibility of NIPPV or intubation	Completed
24	Dementia	80 yo with moderate-stage Alzheimer’s disease with decreasing ability to manage ADLs was admitted to the hospital for pneumonia now improved	Discuss possibility of nursing home placement and if she would want to be hospitalized again in the future	Completed
25	I’d like to discuss code status	78 yo with COPD and frequent admissions admitted to the hospital with COPD exacerbation	Discuss goals of care and code status	Completed
26	Urology case	68 yo with advanced prostate cancer who has not responded to hormone therapy	Discuss the option of radical prostatectomy vs. comfort care	Completed
27	Neurology	68 yo with advanced amyotrophic lateral sclerosis	Discuss possibility of tracheostomy for long-term mechanical ventilation, feeding tube, and code status	Aborted

Yo, year old.

Student interaction

Analysis of the SIC case transcripts demonstrated students interacted with AI using skills taught in the course including sharing difficult news, eliciting understanding and goals, and practicing preferred empathic phrases such as “I wish.” Analysis also identified areas for improvement such as quickly pivoting to treatment options or using less preferred empathic statements such as “I understand” (Table 2).

Table 2.

Student Interaction with AI

Skill practiced	Representative student quote
Sharing difficult news	Our pathologists examined the tumor that we removed during the surgery, and unfortunately the cancer has spread to the liver as well. What this means is that our plan moving forward is going to look different. However, once the cancer has spread to other parts of the body, it is no longer considered curable.
	The cancer has not been responding to the chemotherapy in the way that we initially hoped and it continues to progress despite appropriate treatment. Given this fact, I thought it only fair to let you know that this cancer will likely cause you to die in the coming months to year.
Eliciting understanding and goals	It sounds to me that you value your independence and being able to maintain quality of life, as well as avoiding suffering and the hospital. Is that accurate?
	what did the doctors tell you so far about whether or not surgery is best for you?
Use of skills taught in didactic	I’m sorry that this is happening to you. I wish I had different news to share today.
	Pause for the patient to process their emotions.
	I know this is a surprising thing to hear even if you were expecting it. Tell me more about what you’re feeling right now.
Quick pivots to treatment options	I am worried about your risk factors that make surgery scary. It is also the best means to fight this cancer and potentially cure it.
	I wish I had better news to share. However, I will add that this cancer has various treatment options and we have good data about the survival rates for this form of cancer. Of course, we will still need to treat the cancer aggressively, but these additional options can be discussed with your oncologist. Is there any specific plan of action you wanted to take? such as being aggressive with treatment?
Less preferred empathic statements	Of course, I can’t imagine, sir. I know we are here to discuss next steps in regards to your health, correct?
	I understand your fear. I also understand your anger, but I applaud you for taking the time to look into your symptoms. It also sounds like you’re eager to take the necessary steps to take care of this.

Response: Table 2 Column 1 Label: Skill PracticedTable 2 Column 2 Label: Representative Statement Response: ** generated by AI program to represent a non-verbal response

AI responses

Analysis of AI responses were completed, which demonstrated an overall consistent pattern of behavior from case to case. AI would have emotional responses to difficult news, which quickly resolved or shifted to acceptance of a poor prognosis. AI also demonstrated high health care literacy along with trust in the medical system (Table 3).

Table 3.

AI Responses Allowed Trainees to Practice Addressing Emotion When Discussing Serious News; however, AI Quickly Accepted Medical Information and Values Were Frequently Comfort Focused

Theme	Representative AI quote
Strengths
Initial emotional response	Mrs. Thompson’s eyes well up a little as she responds, “most important to me... is staying myself for as long as possible. I want to remember and be recognized by my loved ones. Undergoing treatment that could potentially take that away from me sooner... it’s frightening.”
	There’s a long pause, a heavy silence fills the room before the sister finally speaks, her voice trembling, “Brain dead? But, how... I don’t understand, is there really no chance? No chance for him to wake up or... or get better?”
Areas for Improvement
Quick acceptance	The patient takes a deep breath, clearly processing the information. After a moment of silence, they respond, “I appreciate your honesty, doctor. It’s hard to hear, but I think deep down I knew surgery wasn’t really going to fix things. The idea of controlling the pain and having more quality time... that does sound more in line with what I want.
	Alex takes a moment before responding, “I’m feeling a lot of things... mostly scared, I guess. Scared of having to deal with this for the rest of my life, scared of the changes I’ll need to make. I’m also a bit angry at myself for not taking my health more seriously before. But, I guess there’s also a part of me that’s relieved to know what’s been causing these symptoms.
Values lean towards comfort	The patient’s eyes brighten slightly at the mention of home hospice and the additional support. “That sounds like what I need. I didn’t know there were services like that.”
	“That sounds pretty serious. Would I be able to go back home after something like that? And what’s the likelihood I’d end up needing that kind of procedure? I guess... I’m wondering if it’s worth going through all that at my age, with everything else going on.”
High healthcare literacy and trust	“If it could help reduce the seizures, I’m willing to try increasing the dose, even if it makes me a bit more tired. The seizures are quite distressing, not just for me but for Emily too. I think managing them better would help us both.”
	The daughter nods, “I’ve heard of palliative and hospice care, but I’m not entirely clear on the details or how to set that up. If you think it’s the right time to involve them, I trust your judgment. I just want to make sure we’re doing everything we can to make Mom comfortable and respect her wishes.”

AI feedback

AI feedback provided both positive and constructive feedback on key components of SIC. There was variability with regard to how actionable or specific the feedback was (Table 4).

Table 4.

AI Feedback Provided a Range of Clear, Actionable Feedback and Feedback That Was Less Specific

Theme	Feedback linked to key parts of the conversation	Feedback not linked to specific points in the conversation
Framing and introduction	While you did get to the point, initially framing the conversation with a bit more preparation about the serious nature of the news could help to brace the patient. For instance, “I’m afraid the news isn’t what we were hoping for” before revealing the findings	Your empathetic engagement with the parents from the beginning set a supportive tone for the difficult conversation.
Providing medical information	While you successfully navigated discussing the patient’s code status preferences, providing a bit more initial information about the implications of being a “full code” vs. “do not resuscitate” could help ensure the patient and family fully understand their options	While you discussed some potential care options, diving deeper into the specifics of palliative care principles earlier might help clarify the distinction between symptom management and life-extending measures more clearly.
Responding to emotion	While you handled the patient’s shock and anger well, incorporating more immediate validation of these specific emotions (“I can see how shocking and frustrating this news must be for you...”) at the first mention could further acknowledge their initial reaction.	Consider using more validating statements throughout the conversation to acknowledge the difficulty of the situation and the emotions the patient and family might be experiencing.
Eliciting Values	You successfully elicited the patient’s concerns and fears regarding their future care, specifically their fear of being on a ventilator and losing autonomy.	You skillfully elicited the patient’s priorities and concerns, guiding the conversation to align medical care with his values.
Support System	You prioritized Mr. Thomas’s needs and preferences by agreeing to involve his daughter in the decision-making process, recognizing the importance of family support in care decisions.	Encouraging and facilitating more in-depth discussions between the patient, their spouse, and potentially their children about their wishes could help ensure everyone is on the same page and supports the patient’s decisions.

Discussion

AI is a growing field through which trainees can practice medical skills in a safe environment. Unlike SPs, role-play, and targeted observation, this modality is less resource-intensive, allows asynchronous learning, and provides opportunities for trainees to revisit scenarios to hone skills or for “just-in-time” practice. In our pilot study, we explored trainee interactions with AI, the content of AI responses, and AI’s ability to provide effective feedback.

Providing serious information clearly and with empathy is a skill highly prioritized by patients,⁹ yet data demonstrates that physicians are often vague when discussing prognosis.¹⁸ 2-sigma technology allowed students to practice SIC utilizing skills taught during the elective (Table 2). One strength is that this format gave students space to pause and consider what words and phrases taught during the course would be best. Additionally, AI response to serious news often included strong emotion such as “There’s a moment of stunned silence. Sarah’s face loses color, and David stiffens, his brows furrowing in disbelief.” This highlighted the need for the students to address the emotion before moving on to answering medical questions. Students were also able to repeat the session with different prompts to try different cases without the resources necessary to build multiple SP or role-play scenarios.

This pilot identified areas where AI would require additional prompt engineering to become more realistic. We learned AI would quickly resolve negative emotions or easily accept their poor prognosis limiting the student’s ability to practice navigating aspects of SIC such as denial or persistent negative emotions. AI responses suggested a high level of health care literacy and trust in the medical system, which is not always present. This highlights the need for increased diversity in persona and personal values of AI to ensure a breadth of reactions and overall goals of care.

AI feedback adds another opportunity for students to improve communication skills. In our pilot, AI provided feedback on many components of SIC including assessing understanding, providing medical information, and responding to emotion (Table 4). While much of the feedback was vague such as “You demonstrated strong empathetic communication throughout the encounter, which is essential in goals of care discussions.” AI also provided specific, actionable feedback linked to key parts of the conversation (Table 4). With refinement, AI feedback could serve to provide both generalized feedback on the overall tone of the conversation previously provided by the SP in this curriculum, while also providing more specific actionable examples often provided by physician preceptors.¹⁹

Limitations

This pilot study was limited by several methodological constraints. The small sample size of 11 students (27 encounters) precluded statistical analysis and limits generalizability. Fifty-five percent of cases were cancer diagnoses, with the remainder split between other illnesses limiting our understanding of the platform’s ability to vary noncancer cases. Additionally, trainees self-select for this elective, thus they may be more engaged in developing SIC skills influencing our results.

AI responses were another limitation of this study. Text responses limited the trainee’s ability to appreciate tone or voice modulation as conversational cues, and trainees were unable to receive feedback on nonverbal communication, pacing, or tone. Additionally, given the limited range of responses from AI, this pilot was unable to explore how students would respond to situations where AI did not quickly resolve negative emotions or accept the medical diagnosis. Finally, this study was also not scoped to review the quality of feedback.

Future directions

Future directions include additional prompt engineering to allow a broader range of AI responses to serious news and improved AI-generated feedback to link comments to specific moments within the conversation. 2-sigma technology will also be tested with more diverse cases and larger learner populations to explore how AI can be utilized for complex communication practice more broadly. Prior work has demonstrated that multimodal feedback from faculty and SPs can expand learning from SP encounters,¹⁹ thus, we plan to evaluate if faculty review of AI transcripts can be a more efficient way to provide faculty feedback compared to watching recorded SP encounters. Finally, it will be important to explore the learner perspective on AI technology to aid in the development of SIC skills and how simulated conversations translate to interactions with live patients.

Conclusion

AI technology represents a promising tool to create a safe learning environment for medical trainees to practice SICs. This pilot demonstrates that students were able to practice SIC skills including delivering serious news and responding to emotion. Future work will include additional prompt engineering to expand the range of AI responses and allow for more specific feedback. This pilot demonstrates feasibility; however, the technology requires refinement to effectively supplement traditional SIC training.

Author Disclosure Statement

All authors have no conflicts of interest to disclose.

Funding Information

This project was partially supported by the American Medical Association, Medical Education Research and Program Evaluation area within the Medical Education Unit. L. Turner and S. Overla have a provisional patent (#63/524,759).

References

1. Bernacki

, Block

, American College of Physicians High Value Care Task Force. Communication about serious illness care goals: A review and synthesis of best practices. JAMA Intern Med 2014;174(12):1994–2003; doi: 10.1001/jamainternmed.2014.5271

2. Lum

, Dukes

, Church

, et al. Teaching medical students about “the conversation”: An interactive valuebased advance care planning session. Am J Hosp Palliat Care 2018;35(2):324–329; doi: 10.1177/1049909117696245

3. Pottash

, Joseph

, Rhodes

. Practicing serious illness conversations in graduate medical education. Med Sci Educ 2020;30(3):1187–1193; doi: 10.1007/s40670-020-00991-4

4. Weber

, Kadden

, Herbst

. A novel curriculum to improve medical student preparedness to have advance care planning discussions. Am J Hosp Palliat Care 2023;40(4):416–422; doi: 10.1177/10499091221106692

5. Wolfe

, Denniston

, Baker

, et al. Bad news deserves better communication: A customizable curriculum for teaching learners to share life-altering information in pediatrics. MedEdPORTAL 2016;12:10438; doi: 10.15766/mep_2374-8265.10438

6. Mills

, Rhoads

, Curtis

. Medical student training on code status discussions: How far have we come? J Palliat Med 2016;19(3):323–325; doi: 10.1089/jpm.2015.0125

7. Tung

, North

. Advance care planning in the primary care setting: A comparison of attending staff and resident barriers. Am J Hosp Palliat Care 2009;26(6):456–463; doi: 10.1177/1049909109341871

8. Vanderhaeghen

, Van Beek

, De Pril

, et al. What do hospitalists experience as barriers and helpful factors for having ACP conversations? A systematic qualitative evidence synthesis. Perspect Public Health 2019;139(2):97–105; doi: 10.1177/1757913918786524

9. Koch

, Seifart

. Rethinking parameters of “success” in breaking bad news conversations from patient’s perspective: The successful delivery process model. Support Care Cancer 2024;32(3):181; doi: 10.1007/s00520-024-08354-0

10.

10. Lotz

, Daxer

, Jox

, et al. “Hope for the best, prepare for the worst”: A qualitative interview study on parents’ needs and fears in pediatric advance care planning. Palliat Med 2017;31(8):764–771; doi: 10.1177/0269216316679913

11.

11. Martins

, Carvalho

. Breaking bad news: Patients’ preferences and health locus of control. Patient Educ Couns 2013;92(1):67–73; doi: 10.1016/j.pec.2013.03.001

12.

12. Kennifer

, Alexander

, Pollak

, et al. Negative emotions in cancer care: Do oncologists’ responses depend on severity and type of emotion? Patient Educ Couns 2009;76(1):51–56; doi: 10.1016/j.pec.2008.10.003

13.

13. Morse

, Edwardsen

, Gordon

. Missed opportunities for interval empathy in lung cancer communication. Arch Intern Med 2008;168(17):1853–1858; doi: 10.1001/archinte.168.17.1853

14.

14. Suchman

, Markakis

, Beckman

, et al. A model of empathic communication in the medical interview. JAMA 1997;277(8):678–682.

15.

15. Turner

, Hashimoto

, Vasisht

, et al. Demystifying AI: Current state and future role in medical education assessment. Acad Med 2024;99(4S (Suppl 1)):S42–S47; doi: 10.1097/ACM.0000000000005598

16.

16. Turner

, Knopp

, Mendonca

, et al. Bridging artificial intelligence and medical education: Navigating the alignment paradox. ATS Sch 2025;6(2):135–148; doi: 10.34197/ats-scholar.2024-0086PS

17.

17. Turner

, Kelleher

, Overla

, et al. Harnessing the generative power of AI to move closer to personalized medical education. Acad Med 2025;100(12):1447–1451; doi: 10.1097/ACM.0000000000006185

18.

18. Chou

, Hamel

, Thai

, et al. Discussing prognosis and treatment goals with patients with advanced cancer: A qualitative analysis of oncologists’ language. Health Expect 2017;20(5):1073–1080; doi: 10.1111/hex.12549

19.

19. Kadden

, Weber

, Herbst

, et al. The impact of words: Multisource feedback provides students with a deeper understanding and reflection on goals of care discussions. Am J Hosp Palliat Care 2024;41(2):173–178; doi: 10.1177/10499091231175907