Comparative Effectiveness and Harms of Intraoperative Transesophageal Echocardiography in Noncardiac Surgery: A Systematic Review

Abstract

Intraoperative use of transesophageal echocardiography (TEE) has become commonplace in high-risk noncardiac surgeries but the balance of benefits and harms remains unclear. This systematic review investigated the comparative effectiveness and harms of intraoperative TEE in noncardiac surgery. We searched Ovid MEDLINE, PubMed, EMBASE, and the Cochrane Library from 1946 to March 2017. Two reviewers independently screened the literature for eligibility. Studies were assessed for the risk of selection bias, confounding, measurement bias, and reporting bias. Three comparative and 13 noncomparative studies were included. Intraoperative TEE was employed in a total of 1912 of 3837 patients. Studies had important design limitations. Data were not amenable to quantitative synthesis due to clinical and methodological diversity. Reported incidence of TEE complications ranged from 0% to 1.7% in patients undergoing various procedures (5 studies, 540 patients). No serious adverse events were observed for mixed surgeries (2 studies, 197 patients). Changes in surgical or medical management attributable to the use of TEE were noted in 17% to 81% of patients (7 studies, 558 patients). The only randomized trial of intraoperative TEE was grossly underpowered to detect meaningful differences in 30-day postoperative outcomes. There is lack of high-quality evidence of effectiveness and harms of intraoperative TEE in the management of non-cardiac surgeries. Evidence, however, indicates timely evaluation of cardiac function and structure, and hemodynamics. Future studies should be comparative evaluating confounder-adjusted impact on both intraoperative and 30-day postoperative clinical outcomes.

Keywords

intraoperative transesophageal echocardiography transesophageal echocardiography noncardiac surgery systematic review effectiveness

Introduction

Transesophageal echocardiography (TEE) in cardiac surgery has evolved beyond its original use to assess global ventricular systolic function, regional wall motion abnormalities, and severity of valvular lesions.^1-3 It is now increasingly used intraoperatively to monitor hemodynamic status, examine cardiac structures, confirm diagnoses, and assess early results of surgical interventions. Furthermore, the technology has now expanded into non-cardiac perioperative settings as a natural extension of its use in cardiac surgery.

With the diminishing use of the pulmonary artery catheter,⁴ anesthesiologists need real-time dynamic monitoring modalities to optimize perioperative hemodynamic management and minimize perioperative morbidity and mortality. Mortality rates following elective noncardiac surgery have not significantly improved over the past 25 years despite advances in preoperative risk assessment, introduction of enhanced recovery programs, and advances in minimally invasive surgical procedures.^5-7 As such, both transthoracic echocardiography (TTE) and TEE are now integrated widely into daily anesthesia practice.^8-11 However, the sterile surgical field, positive pressure ventilation, patient’s position, and inaccessibility of the standard acoustic windows limit the intraoperative use of TTE. TEE, therefore, becomes a more practical modality for intraoperative cardiac imaging and hemodynamic monitoring.

The only systematic review on the topic investigated the frequency distribution of various pathologies detected with point of care echocardiography in patients undergoing noncardiac surgery.¹¹ The review, however, neither performed a critical appraisal of the literature nor investigated the effectiveness and harms of TEE.

To investigate the comparative effectiveness and harms of the intraoperative use of TEE in noncardiac surgery and identify subgroups of patients most likely to benefit, we performed a systematic review of the literature. Initially, when we searched and screened the literature, the scope of literature review was wider including the evidence for TEE training requirements, certification and maintenance of competence. In this article, however, we report evidence synthesis for comparative effectiveness and harms of intraoperative TEE in noncardiac surgery.

Methods

We followed the Cochrane and PRISMA standards for conducting and reporting our systematic review.^12,13

Data Sources

An experienced information specialist searched Ovid MEDLINE, PubMed, EMBASE, and the Cochrane Library from 1946 to March 2017. An additional search was conducted using the Google search engine. Keywords and medical subject headings related to TEE, perioperative/intraoperative period and noncardiac surgery were used. Retrieved records were limited to English language and human studies. The full search strategy is provided in the supplementary material (available in the online version of the article).

Study Selection

We included both experimental or observational studies that employed TEE intraoperatively to manage noncardiac surgery. To detect publication bias, we did not exclude studies when they did not report outcomes of unintended harms of TEE, intraoperative clinical events (mortality, cardiac arrest, myocardial infraction), change in patient management, and postoperative 30-day clinical events (mortality, cardiovascular mortality, myocardial infraction, stroke/transient ischemic attack, arrhythmia, hypotension requiring circulatory support, acute kidney injury and length of hospital stay) which were considered the review outcomes of interest.

We excluded records:

with study participants that were not adult noncardiac surgical patients

when intervention was not TEE

when effectiveness or harms of intraoperative TEE, reporting of TEE findings, or TEE training, assessment and certification were not investigated

when they were editorials, commentaries, narrative reviews, case reports, or surveys of views of health care providers

when they were reported in a language other than English

for other reasons as documented in the PRISMA flow diagram (Figure 1)

One reviewer screened titles (AF) and excluded ineligible records. Another reviewer independently confirmed exclusions (MTA). Full texts of included records were further assessed for eligibility independently by 2 reviewers (AF and MTA). Disagreements were resolved with consensus. General study characteristics and outcomes data were extracted by one reviewer (MTA) and verified by another reviewer (AF). Between reviewers, both methods and content expertise were represented.

Critical Appraisal

Because much methodological diversity was anticipated, we employed a generic assessment of methodological limitations when critically appraising the studies (risk of bias assessment). We evaluated studies for selection bias (including attrition bias), confounding, measurement bias, and obvious outcome reporting bias. Inferences about effectiveness and harms of therapeutic or diagnostic interventions require assessment of the validity of studies for causal inference; in other words, establishing that the exposure or intervention and outcome associational risk estimate represents the causal risk estimate. Noncomparative single arm before-after studies have serious limitations in this regard unless effect sizes are very large and there are no other plausible competing explanations (eg, confounder effect). Plausible and competing explanations for observed effects in single-arm before-after trials, cohorts, and case-series can be attributed to time-dependent natural variation in disease severity, patient prognostic factors, co-exposures and co-interventions. We could not undertake a formal assessment of outcome reporting bias because study protocols did not exist or were not accessible. Publication bias could also not be detected because data were not amenable to quantitative synthesis (see below).

Evidence Synthesis

Clinical and methodological diversity across studies and sparse data precluded data meta-analysis. Furthermore, most studies were noncomparative and descriptive in design rather than analytic from which no estimates of effects could be generated. We, therefore, adopted a qualitative approach to evidence synthesis.

Results

We screened 1329 records and included 16 studies of effectiveness and harms of intraoperative TEE in non-cardiac surgery (Figure 1). Intraoperative TEE was used in 1912 of 3837 patients included across the entire body of evidence (Table 1). Only three studies were comparative analyses: a randomized controlled trial and 2 cohort studies, with the remaining studies employing single-arm cohort or experimental design (Table 2).^14-16

Table 1.

Included Study Characteristics.

Study/Year	Country	Funding	Surgery	Design	Sample size	Analysis
Ball et al²⁸/2013	NR	None	Renal cell carcinoma	Cohort	36	Noncomparative
Black et al²⁰/1990	USA	NR	Neurosurgery	Cohort	101	Noncomparative
Burger-Klepp et al¹⁹/2012	Austria	None	Liver transplant	Cohort	396	Noncomparative
Chatterjee et al³⁷/2012	India	None	Neurosurgery	Single-arm trial	15	Noncomparative
D’Angelo et al²⁵/1997	USA	NR	Vascular	Cohort	8	Noncomparative
Denault et al²⁷/2002	Canada	Nonprofit	Mixed^a	Cohort	214	Noncomparative
Hofer et al²¹/2004	Switzerland	NR	Mixed^a	Cohort	99	Noncomparative
Lin et al²⁶/2006	Taiwan	NR	Mixed^a	Case series	10	Noncomparative
Markin et al²²/2015	USA	NR	Liver transplant	Cohort	120	Noncomparative
Memtsoudis et al¹⁷/2006	USA	NR	Mixed^a	Case series	22	Noncomparative
Myo Bui et al ¹⁶/2015	NR	NR	Liver transplant	Cohort	656	Comparative
Pai et al¹⁴/2015	USA	None	Liver transplant	Cohort	1811	Comparative
Schulmeyer et al²³/2006	Chile	NR	Abdominal surgery	Single-arm trial	98	Noncomparative
Shillcutt et al¹⁵/2014	USA	NR	Mixed^a	RCT	28	Comparative
Shillcutt et al¹⁸/2016	USA	NR	Liver transplant	Cohort	100	Noncomparative
Suriani et al²⁴/1998	USA	NR	Mixed^a	Cohort	123	Noncomparative

Abbreviations: NR, not reported; RCT, randomized controlled trial.

“Mixed” indicates that different types of surgery that may have included obstetrics, gynecological procedures, general surgery, orthopedic procedures, neurosurgery, lung transplant, urology, or plastic procedures.

Table 2.

Included Study Designs and Reported Outcomes.

Study/Year	Study Design	TEE Proficiency	TEE Indication	Supported by ASA and SCA 2010 Guidelines	Reported Outcomes
Study/Year	Study Design	TEE Proficiency	TEE Indication	Supported by ASA and SCA 2010 Guidelines	Impact on Patient Management	TEE Complications	Postoperative Outcomes
Ball et al²⁸/2013	Noncomparative retrospective cohort	NR	Monitoring and management	No	Yes	NR	NR
Black et al²⁰/1990	Noncomparative cohort	Experienced	Monitoring and management	Yes	Yes	Yes	NR
Burger-Klepp et al¹⁹/2012	Noncomparative retrospective cohort	NR	Monitoring and management	Equivocal	NR	Yes	Yes
Chatterjee et al³⁷/2012	Single-arm trial	NR	Monitoring	Equivocal	NR	NR	NR
D’Angelo et al²⁵/1997	Noncomparative cohort	NR	Monitoring	Equivocal	NR	NR	NR
Denault et al²⁷/2002	Noncomparative cohort study	Basic or advanced	Diagnosis, monitoring, and management	Yes	Yes	NR	NR
Hofer et al²¹/2004	Noncomparative retrospective cohort	Unclear	Diagnosis, monitoring, and management	Yes	Yes	Yes	NR
Lin et al²⁶/2006	Case series	NR	Diagnosis and management	Yes	Yes	NR	NR
Markin et al²²/2015	Noncomparative retrospective cohort	50% formally trained	Monitoring and management	Mixed yes and equivocal	NR	Yes	NR
Memtsoudis et al¹⁷/2006	Case series	Advance	Diagnosis and management	Yes	Yes	NR	Yes
Myo Bui et al¹⁶/2015	Comparative retrospective cohort study	NR	Monitoring and management	Equivocal	NR	Yes	NR
Pai et al¹⁴/2015	Comparative retrospective cohort study	Trained	Monitoring and management	Equivocal	NR	Yes	NR
Schulmeyer et al²³/2006	Single-arm trial	Advance	Monitoring and management	Yes	Yes	Yes	NR
Shillcutt et al¹⁵/2014	Randomized controlled trial	Trained	Monitoring and management	Yes	Yes	Yes	Yes
Shillcutt et al¹⁸/2016	Noncomparative retrospective cohort study	Trained	Diagnosis, monitoring, and management	Equivocal	NR	NR	Yes
Suriani et al²⁴/1998	Retrospective chart review	Basic or advanced	Diagnosis, monitoring, and management	Mixed yes and equivocal	Yes	Yes	NR

Abbreviations: NR, not reported; TEE, transesophageal echocardiography; ASA, American Society of Anesthesiologists; SCA, Society of Cardiovascular Anesthesiologists.

Study Risk of Bias

We found serious study limitations in all non-comparative evidence. Even the three comparative studies were not free of important design limitations. Important concerns were confounding by indication and detection or selection bias (Table 3). Furthermore, only 4 studies examined postoperative adverse clinical outcomes (see section “Thirty-Day Postoperative Clinical Outcomes”), indicating a shortage of empiric evidence addressing the topic of interest.^15,17-19

Table 3.

Effectiveness and Harms of Intraoperative Transesophageal Echocardiography (TEE) in Noncardiac Surgery.

Study/Year	Study Objective, Population, and Selection Criteria	Study Design (Sample Size)	Results	Authors’ Conclusions	Limitations
Ball et al²⁸/2013	To investigate intraoperative TEE-guided changes in surgical approach in patients with renal cell carcinoma undergoing nephrectomy and inferior vena cava tumor thrombectomy—excluded cases when TEE could not be used	Noncomparative retrospective cohort (N = 36)	Changes in surgical approach were noted in 17% of patients mostly in those with right atrium extension	TEE affects surgical decision making particularly in patients with intracardiac tumor thrombus extension	Noncomparative study Unclear risk of selection and outcome measurement bias because of poor reporting (study available only as an abstract)
Black et al²⁰/1990	Efficacy of intraoperative TEE in the management of paradoxical air embolism associated with right to left shunting due to patent foramen ovale in adult patients undergoing neurosurgical procedure in the sitting position	Noncomparative cohort (N = 101)	Clinical management was altered in all patients in whom right to left shunting was detected No adverse intraoperative clinical outcomes or complications were noted	TEE is effective in managing neurosurgical procedures by identifying right to left shunts and paradoxical air embolism in the sitting position and has no major adverse events	Noncomparative study design Unclear random selection of surgical patients; not blinded for prior TTE findings; and limited generalizability because the observed incidence of right to left shunts was much less than that expected for the general population Underpowered to determine efficacy because only 6 right to left shunts were identified
Burger-Klepp et al¹⁹/2012	To study intraoperative TEE related complications in patients undergoing orthotopic liver transplant Median MELD (model of end-stage liver disease) score = 16 (12-20); 72% of patients had varices Excluded liver failure with no pretransplantation esophagogastroduodenoscopy screening or incomplete medical records	Noncomparative retrospective cohort (N = 396)	One case of variceal hemorrhage requiring balloon tamponade was documented 4% of patients died within 30 days of transplantation	TEE is a relatively safe method for monitoring cardiac performance in patients with documented esophagogastric varices undergoing liver transplant	Noncomparative study Patients with incomplete records were excluded (attrition bias) Unclear risk of outcome measurement bias due to nonblinding and uncertain data transcription accuracy
Chatterjee et al³⁷/2012	Intraoperative TEE to evaluate hemodynamic and echocardiographic parameters in a pilot study on adult patients with no cardiovascular disease undergoing elective supratentorial craniotomy and requiring mannitol intraoperatively Patients with contraindications for placement of the TEE probe or surgical positioning (prone position or requirement of extreme neck flexion) were excluded	Single-arm trial (N = 15) Arterial and central venous catheters were also placed		Hemodynamic changes were quickly and reliably demonstrated using TEE in elective supratentorial craniotomies, but clinical effectiveness and harms were not investigated	Noncomparative study designUnclear random selection of patients; unclear whether central venous and mean arterial pressure readings influenced echocardiographic parameters. The study did not aim to investigate effectiveness or harms of intraoperative TEE.Limited generalizability because patients with cardiovascular conditions were not recruited.
D’Angelo et al²⁵/1997	A qualitative comparison of diagnostic performance of intraoperative TEE and pulmonary artery catheter monitoring in patients undergoing laparoscopic assisted abdominal aortic aneurysm repair under pneumoperitoneumPatients were excluded if they had a contraindication to laparoscopy or refused to participate in the study	Diagnostic cohort study, included here as noncomparative diagnostic efficacy study because reported intraoperative mortality data (N = 8)Hemodynamic changes were monitored with pulmonary artery catheters (n = 8) and TEE (n = 7)		TEE may be a better alternative to monitor cardiac status and should be considered in patients with coexisting cardiac disease undergoing laparoscopic procedures	Seriously underpowered noncomparative cohort study designed as a comparative diagnostic performance evaluationThe only clinical event reported was surgical mortality which was zero
Denault et al²⁷/2002	To evaluate patient management impact of perioperative TEE in noncardiac surgeryExcluded TEE performed in cardiac operating room	Non-comparative cohort study (N=214)Of 214 consecutive TEEs, 155 were in the operating room for mostly general, thoracic or vascular procedures. Other TEE exams were in the surgical recovery room or surgical intensive care unit	Medical or surgical management was changed as per TEE findings in 28% of patients in the OR (and in 40% of patients overall)Therapy was more commonly modified in general surgery compared with vascular and thoracic surgeries	Intraoperative TEE has an impact in non-cardiac surgical procedures, but harms were not reported	Non-comparative study.Outcome was subjective non-blinded judgement and it is unclear what other hemodynamic monitoring was also available. Further, cardiac surgery patients in the intensive care unit were not excluded (potential confounding)
Hofer et al²¹/2004	To investigate the impact of intraoperative TEE on hemodynamic management during elective noncardiac surgery in patients at risk of intraoperative myocardial ischemia or hemodynamic instability	Noncomparative retrospective cohort study (N = 99)Patients also underwent routine monitoring with electrocardiography, arterial, central venous or pulmonary artery catheterAttending anesthesiologist judged the incremental impact of TEE	Changes in drug therapy were made in 47% and changes in fluid therapy in 24% of patients. TEE predominantly influenced intraoperative management in patients with segmental wall motion abnormalities and history of left heart failureInfluence of TEE on fluid therapy during lung or liver transplantations was significantly greaterNo TEE complications	Intraoperative TEE has an impact in noncardiac surgical procedures with no major procedure-related adverse events	Noncomparative study.Judgments of changes in management decisions due to TEE findings were subjective and nonblinded. Unclear how much management decisions were based on TEE findings as opposed to routine intraoperative hemodynamic monitoring
Lin et al²⁶/2006	Efficacy and utility of TEE in the diagnosis and management of intraoperative cardiac arrest in patients undergoing elective noncardiac surgeryCardiac arrests due to obvious causes or significant blood loss were excluded	Case series (N = 10)Cause of cardiac arrest was confirmed by postmortem, surgical, or other standard diagnostic studies	Three deaths were observedTEE explanation for cardiac arrest was myocardial infarction (n = 5), pulmonary embolism (n = 2), hypovolemia due to hemorrhage (n = 1), dilated ventricles with sudden fibrillation (n = 1) and unknown etiology (n = 1)	TEE is effective in managing noncardiac surgical patients with intraoperative cardiac arrest	Noncomparative studySelection bias could not be ruled out because of poor reporting. Unclear how much other intra- and postoperative diagnostic evaluations contributed to diagnostic assessment and patient managementSmall sample size
Markin et al²²/2015	To evaluate safety of intraoperative TEE in patients undergoing orthotopic liver transplantationData were excluded for incompleteness or no intraoperative TEE intervention	Noncomparative retrospective cohort (N = 116)TEE was performed at the discretion of anesthesiologists	Total complication rate was 1.7% (variceal hemorrhage and ventricular fibrillation)	TEE is a relatively safe method to monitor patients undergoing liver transplantation	Noncomparative study.Attrition bias could not be ruled out because of missing data from 4 patients. Postoperative bleeds and TEE complications were not assessed
Memtsoudis et al¹⁷/2006	Patients experiencing cardiac arrest during noncardiac surgical procedures	Case series (N = 22)	Suspected diagnosis for cardiac arrest was made in 19 patients.Management was guided by TEE in 82% of patients68% perioperative mortality were observed (36% intraoperative and 32% postoperative)	TEE is effective in managing noncardiac surgical patients with intraoperative cardiac arrest	Noncomparative study.Selection bias could not be ruled out because of inadequate reporting and because patients who arrived with cardiac arrest from outside the operating room were excluded. Judgments of changes in management decisions due to TEE findings were subjective and nonblinded.Small sample size
Myo Bui et al¹⁶/2015	To investigate the gastrointestinal complications of intraoperative TEE in liver transplantation patients with MELD score ≥25Excluded: active gastrointestinal bleeding and recent esophageal banding	Comparative retrospective cohort study (N = 656)TEE was inserted in 433 patientsProbe insertion was at the discretion of anesthesiologist	Postoperative gastrointestinal consult (6 vs 5)Major gastrointestinal complications (1 vs 0)Bloody nasogastric output >500 mL/d (11 vs 7)	Intraoperative TEE was not associated with major gastroesophageal and hemorrhagic complications in patients with MELD score ≥25 during liver transplantation	Retrospective hospital records—data accuracy unclear (eg, excluded patients when TEE status could not be confirmed, n = 244)Unclear risk of attrition bias and high risk of bias due to confounding by TEE contraindicationOutcome measurement bias unclearUnclear risk of bias due to level of care differencesUnderpowered study
Pai et al¹⁴/2015	To evaluate safety of intraoperative TEE in patients undergoing orthotopic liver transplantationPatients had preoperative variceal banding for grade II and grade III varices	Retrospective cohort comparing TEE related outcomes with no intraoperative TEE use (N = 1811; 232 underwent TEE and 1579 did not)	Two cases of variceal bleeds in either group	TEE is a relatively safe method to monitor patients undergoing liver transplantation	Confounding by contraindication is an important limitation of this studyPostoperative gastrointestinal bleeds were not assessed
Schulmeyer et al²³/2006	Impact of intraoperative TEE on hemodynamic management during elective or emergent abdominal surgery at high risk for myocardial ischemia or hemodynamic instabilityExcluded patients with recent esophageal or gastric surgery, active bleeding of esophageal varices, or those post radiation therapy of the esophageal areaPatients undergoing vascular abdominal procedures were also excluded	Single-arm trial (N = 98)Management was guided by intraoperative TEE as per protocolPulmonary artery catheter was placed in 24 of 90 patients undergoing TEE to compare their relative utility	TEE did not inform management in 2%, led to important hemodynamic management changes in 48%, postoperative management decisions due to new diagnoses in 25%. As per the judgment of anesthesiologist, TEE could substitute for a pulmonary artery catheter in all 24 patientsNo serious complications relating to TEE insertion were observed	TEE is effective in managing high-risk patients undergoing noncardiac surgery and has no major adverse events. TEE can substitute for pulmonary artery catheter	Noncomparative study designSubjective nonblinded assessment of outcomes
Shillcutt et al¹⁵/2014	To study the feasibility and safety of intraoperative TEE in patients with left ventricular diastolic dysfunction at increased risk of postoperative adverse eventsExcluded surgical emergencies, preoperative shock, anesthesia protocol other than general, patients with elevated intracranial pressure	Randomized controlled trial of surgical patients with baseline left ventricular dysfunction (N = 28)TEE-guided, goal-directed, algorithmic fluid and vasoactive drug management was compared with conventional management (noninvasive blood pressure monitoring or invasive arterial line or central venous line).64% of TEE group also had invasive arterial line or central venous monitoringNo pulmonary artery catheters were employed	The study did not detect any significant differences in duration of surgeryNo procedure-related adverse events were observedNo differences were found in 30-day clinical outcomes of length of hospital stay, congestive heart failure, and atrial fibrillation	Echocardiography-guided hemodynamic management of patients with diastolic dysfunction during noncardiac surgery may be a safe alternative to conventional management	Inadequate reporting of sequence generation and allocation concealment (unclear risk of confounding bias—baseline imbalance in ventricular dysfunction was not reported)Unclear risk of performance and detection bias because of lack of blinding of interventionHigh risk of outcome reporting bias for the mortality outcome and worsening of preexisting symptomsUnderpowered study
Shillcutt et al¹⁸/2016	To determine if abnormal intraoperative TEE findings are associated with major postoperative adverse cardiac events in patients undergoing orthotopic liver transplantationExcluded patients with incomplete data	Noncomparative retrospective cohort study (N = 100)TEE findings were recorded after blind review by experienced echocardiographersMACE (major adverse cardiovascular event) was defined as: atrial fibrillation, nonfatal myocardial infarction, congestive heart failure, pulmonary embolism, stroke, and death	32% of patients had abnormal preoperative resting TTE examinations9 intraoperative cardiac arrests and 1 intraoperative death were observedBiventricular dysfunction and intracardiac thromboemboli were associated with decreased overall 1-year survival and 30-day MACE	Specific intraoperative TEE detected findings are predictive of postoperative cardiac events	Retrospective noncomparative study at unclear risk of selection bias (46 patients had incomplete TEE examination)Moderate risk of confounding bias because of unadjusted univariate analyses and unclear risk of outcome measurement bias because clinical events were obtained from electronic medical records
Suriani et al²⁴/1998	To evaluate the impact of intraoperative TEE in noncardiac surgery over a 4-year period	Retrospective chart review (N = 123)TEE was either planned or undertaken per requirementA random subset of TEE videotapes were verified by advance trained echocardiographers blinded to original findings57% of patients also had invasive hemodynamic monitoring (pulmonary artery catheter or central venous line)	Intraoperative TEE had an impact on patient management in 81% of patientsTEE resulted in major impact in 15% (eg, life-threatening treatment, altered surgical approach, altered anesthetic management, required further postoperative evaluation), minor impact in 48% (eg, monitoring of cardiac function or change in drug therapy), limited impact in 17%, and no impact in 20% of patientsPatients in whom TEE had any impact were significantly olderNew findings were observed in 33% of patients; in others TEE either verified suspected findings (25%) or did not result in new findings (42%)Zero complications were reported	Particularly in patients older than 65 years undergoing noncardiac surgery, TEE is effective and safe for intraoperative hemodynamic monitoring	Noncomparative chart review (reporting bias)Subjective assessment of impact on outcomesOn retrospective blinded verification, discrepancies in at least one TEE finding was noted for 42% of patientsPotential for confounding bias by alternative hemodynamic monitoring could not be excluded

TEE Probe–Related Complications

Nine studies reported complications associated with intraoperative TEE in noncardiac surgical populations.^14-16,19-24 In 2 studies, only serious TEE complications were measured.^21,23 Zero cases were reported for 197 patients. Of 5 studies investigating all TEE-related complications, 3 cases were reported across 540 patients with incidence ranging from 0% to 1.7%.^{14,15,20,22,24} Incidence of gastrointestinal bleeds associated with probe insertion in patients undergoing liver transplantation ranged from 0.3% to 2.5%.^14,16,19,22 Complications such as odynophagia, oropharyngeal and dental trauma, esophageal perforation, and endotracheal tube displacement requiring repositioning were not found.

No adverse TEE probe–related adverse events were observed in the single randomized controlled trial in which 14 surgical patients were assigned to TEE-guided intraoperative hemodynamic management versus 14 to routine care.¹⁵ Two other comparative observational studies were also underpowered to detect any significant differences in specific probe-related complications between patients managed with intraoperative TEE versus patients who were not.^14,16

Intraoperative Clinical Events

Changes in or emerging clinical events were rarely reported across studies. Twelve intraoperative deaths were reported in 140 surgical patients in four non-comparative studies.^17,18,25,26 These numbers, however, do not lend themselves to even a crude estimation of TEE-associated risk of death because 2 of the 4 studies were case series of patients who had experienced cardiac arrest intraoperatively.^17,26

Change in Intraoperative Management

Seven noncomparative studies including a total of 558 patients contributed relevant data.^{17,20,21,23,24,27,28} The intraoperative surgical or medical management was altered because of TEE probe placement was a subjective nonblinded assessment of investigators. After excluding 2 studies, the study by Memtsoudis et al¹⁷ (which was restricted to a series of patients experiencing intraoperative cardiac arrest) and the study by Black et al²⁰ (which exclusively focused on the management of paradoxical air embolism in neurosurgical patients), changes in surgical or medical management attributable to the use of TEE were noted in 17% to 81% of heterogeneous samples of noncardiac surgical patient population. All contributing studies had serious study design limitations for causal inference (Table 3).

Thirty-Day Postoperative Clinical Outcomes

Four studies observed the postoperative course of patients (Table 3)^15,17-19 The study by Memtsoudis et al¹⁷ observed that 32% of 22 noncardiac surgical patients who underwent intraoperative TEE examination because of unexpected hemodynamic collapse requiring advanced cardiac life support subsequently died in the immediate postoperative period. The study was a retrospective and noncomparative evaluation of a departmental TEE database for which it was difficult to make judgements about the presence or absence of selection bias due to inadequate reporting. Furthermore, generalizability of the findings is very limited. No other postoperative clinical events were reported in this study. In another noncomparative study of 396 patients undergoing liver transplantation under intraoperative TEE monitoring, 4% of patients died within 30 days.¹⁹

Shillcutt et al¹⁸ conducted a retrospective cohort study of patients undergoing orthotopic liver transplantation. Authors aimed to investigate associations between intraoperative TEE findings and postoperative clinical outcomes, but the study was not designed to evaluate effectiveness of TEE. Most common intraoperative TEE findings were microemboli (44% of patients), right ventricular dysfunction (31%), and thromboembolism (27%). Biventricular dysfunction and intracardiac thromboemboli were associated with decreased overall 1-year survival and 30-day major adverse cardiovascular events.

Shillcutt et al¹⁵ reported a randomized controlled trial in 28 patients with left ventricular diastolic dysfunction at increased risk of postoperative adverse events. The study was clearly underpowered for clinical outcomes. Within 30 days after surgery, 2 patients randomized to intraoperative TEE developed congestive heart failure or atrial fibrillation. In the conventional management arm (intraoperative noninvasive blood pressure monitoring or invasive arterial line or central venous line), the corresponding number of patients was 6 (relative risk = 0.33, 95% CI 0.08-1.38). Postoperative myocardial infarction, stroke/transient ischemic attack, and acute kidney injury were not observed. Length of hospital stay did not significantly differ between the randomized groups. Mortality outcome was not reported.

Discussion

Practice guidelines and expert opinion recommend the use of TEE to manage patients undergoing cardiac surgical procedures.²⁹ For noncardiac surgeries, although early reports indicated TEE’s incremental clinical value in noncardiac surgical patients and superiority over other monitoring modalities, a critical review of existing evidence has been lacking.^30-32

To our knowledge, this is the first systematic and critical evaluation of the literature addressing the question of effectiveness and harms of the use of intraoperative TEE exclusively in patients undergoing noncardiac surgical procedures when compared with management without the use of this technology. While authors of included studies concluded that the use of TEE is effective and safe for intraoperative management of high-risk patents, our findings could not confirm this claim for a number of reasons. Despite the fact that our systematic review included evidence from well more than 3000 patients, we conclude that there is a lack of high-quality evidence addressing the comparative effectiveness and harms of intraoperative TEE in the management of noncardiac surgical procedures versus routine intraoperative hemodynamic monitoring. Extant evidence does however suggest that the technology aids in a timely evaluation of functional and morphological cardiac parameters, and hemodynamics.

A number of factors contribute to the inconclusive findings of this review. With rare exceptions, generally studies were not analytically designed and analyzed to study effectiveness and harms of TEE in comparison with routine intraoperative hemodynamic management—only 3 of 16 studies employed a comparative design.^14-16 As such, various design and analytic biases were noted precluding confident conclusions about the relative effectiveness and safety of the intervention. Also, a number of studies were restricted to specific noncardiac surgical populations, for example, posterior fossa neurosurgical procedures in the sitting position, orthotopic liver transplantation, or patients experiencing intraoperative cardiac arrest, limiting generalizability of findings. Furthermore, postoperative outcomes were rarely investigated, and when they were, studies lacked statistical power to detect important differences.

Jasudavisius et al¹¹ recently published a systematic review of TEE in noncardiac surgery. Comparative effectiveness in terms of differences in risk of adverse surgical outcomes between the use and nonuse of TEE intraoperatively was not investigated. Rather, authors provided a prevalence map of the various diagnoses reported across the studies employing TTE or TEE in the perioperative period. Across 13 included studies, they found that the most common intraoperative findings were low ejection fraction, right ventricular failure, wall motion abnormalities, and pulmonary embolism. Our findings confirm their observation that currently there is lack of evidence demonstrating the effectiveness of intraoperative TEE in improving patient-oriented outcomes.

Of the 12 practice guidelines on the use of perioperative TEE published to date (Table 4), only 3 examined existing evidence and found it to be of limited quality with recommendations largely founded on expert opinion.^29,33,34 In our view, despite uncertainty about the comparative effectiveness and harms of TEE, existing guidance, even when largely based on expert opinion, is necessary as the technology has already diffused into routine noncardiac surgical practice. In this light, current guideline recommendations limiting the use of TEE to high-risk noncardiac surgeries and patients with intraoperative hemodynamic instability appear to be reasonable. Our review highlights the limitations of evidence and calls for high-quality research on the topic to establish empirically that benefits of TEE outweigh harms and costs in managing high-risk noncardiac surgical procedures. Ever increasing scarcity of health care resources makes it imperative that recommendations for intraoperative use of TEE in noncardiac surgeries are based on high certainty evidence of benefits outweighing risks and burden to the patient, the operating team, or both, and the cost to the health system and payers.

Table 4.

Published TEE Guidelines.

Guidelines	Country	Level of Evidence	Findings	Inference	Limitations
ASA/SCA (1996)³³	USA	Stated and defined. Evidences divided into 3 categoriesCategory I: Supported by strongest evidence or expert opinionCategory II: Supported by weaker evidence and expert consensusCategory III: Have little current scientific or expert support	Limited evidence in all categories with strong influence of the opinion of expertsMore emphasis on cardiac surgical procedures	Stronger evidence on TEE utilization in intraoperative acute, persistence and life-threatening hemodynamic disturbance with no obvious causeDiagnosis of thoracic aneurysmAir embolism during neurosurgical procedures	Was not specific for noncardiac surgical patientsMainly based on expert opinions
ASE/SCA (1999)³⁸	USA	Not stated	Expert opinionDescription of a comprehensive intraoperative TEE examination based on a series of 20 anatomically referenced cross-sectional views	Standardization of TEE exam (quality)	The recommended views targeted patients undergoing cardiac surgery procedures
ASE/SCA (2002)³⁹	USA	Not stated	Expert opinionDescription of cognitive and technical skills requirements for basic and advanced training on perioperative echocardiographyClassified as level I, II, and III	Training and expertise	No evidence on clinical benefit and harms was described
ASE/SCA (2006)⁴⁰	USA	Not stated	Expert opinionContinuous quality improvement (CQI)	Quality assuranceincorporating CQI into practice may be beneficial	Very limited evidence. One study included a group of 10 cardiac anesthesiologists practicing in an academic environment may benefit from offline interpretation provided by 2 echocardiographers to enhance interpretation and define area that require further training
Canadian Anesthesiologists Society (2006)⁴¹	Canada	Not stated	Expert opinion	Canadian guidelines focused on training, CQI and TEE exam	No evidence on clinical benefit and harms was described
ASE/SCA (2010)²⁹	USA	Stated and identifiedBased on the strength of the literature and divided into:Category A: SupportiveCategory B: SuggestiveCategory C: EquivocalCategory D: Insufficient	Suggested utilization of TEE in:Cardiac and thoracic aorta surgical patients.Noncardiac surgical patients with intracardiac embolism, air embolism, and patent foramen ovale, mitral regurgitation, left ventricular hypertrophy, left ventricular outflow tract obstruction, regional wall motion abnormalityPatients undergoing:Vascular surgery and other major surgeryLiver transplantCritical care patients.Level of evidence were found to be category B	TEE utilization when the nature of surgery or the patient’s known or suspected cardiovascular pathology that might result in severe hemodynamic, pulmonary, or neurologic compromiseAlso, if expertise and equipment are available, TEE should be used when unexplained life-threatening circulatory instability persists despite corrective therapy	Moderate level of evidence (category B). No category A evidence existed
European Association of Cardiothoracic Anaesthesiologists (2010)⁴²	Europe	Not stated	Expert opinionGuidelines on TEE, which include perioperative management	TEE indications in different settings, training requirement, examination required	Very limited data on perioperative TEENo evidence on clinical benefit and harms was described
ASE/SCA (2013)⁴³	USA	Not stated	Expert opinionGuidelines on basic TEE	Basic TEE examCognitive and technical skills requiredTraining and certification requirementQuality assurance	No evidence on clinical benefit and harms was described
ASE/SCA (2013)⁴⁴	USA	Not stated	Expert opinionAn update of previous guidelines on training and competencyIndicationsComprehensive TEE exam and specific structures	Standardization, competency, and quality assurance	No evidence on clinical benefit and harms was described
ASE (2015)³⁴	USA	Stated and the following levels of evidence are identified:B2, observational (noncomparative) studiesB3 observation studies/case reportsD1 lack of evidence to show benefits	Evidence related to noncardiac surgery (B2):Major vascular surgeryOrthopedic/spinal surgeryNeurosurgery/sitting positionLiver/lung transplantProsthetic valve thrombusLevel of evidence B3:Kidney transplantLevel of evidence D1:Trauma	Highest level of evidence is B2	No level A evidence was listed
Chinese Anesthesiologists (2015)⁴⁵	China	Not stated	Narrative review more than guidelines	Nonspecific	No evidence on clinical benefit and harms was described
British Society of Echocardiography (2015)⁴⁶	UK	Not stated	Expert opinionFocused on standards of obtaining a full TEE exam	Obtaining details full exam ensures nothing is missed	Not specific for perioperative TEE.No evidence on clinical benefit and harms was described

Abbreviations: ASA, American Society of Anesthesiologists; ASE, American Society of Echocardiography; SCA, Society of Cardiovascular Anesthesiologists; TEE, Transesophageal echocardiography.

Clinical thinking at the point of care is focused on structural, morphological and hemodynamic changes. While there is validity of TEE in intraoperative settings for identifying hemodynamic instability and its etiology, our review shows that there is dearth of high-quality empiric evidence of the effectiveness and cost-effectiveness of TEE in improving outcomes of surgery such as length of hospital stay and 30-day postoperative events. Despite the passage of time, unfortunately, the observation by Catena and Mele³⁵ that little literature exists on patient outcome, logistics, financial impact, medicolegal implications, and safety of TEE in noncardiac surgeries remain true today.

Limitations of our review include language restriction as one of the study eligibility criteria and the lack of formally grading our certainty in findings as per the GRADE (Grading of Recommendations, Assessment, Development and Evaluations) approach.³⁶ The latter, however, was a deliberate decision of ours as the limitations in the design, statistical power, validity and applicability of studies were quite obvious, obviating a formal GRADE evaluation.

Ideally, adequately randomized controlled trials are best suited to study effectiveness, cost-effectiveness, and harms of interventions. However, given the quality of existing studies on the topic we have observed in this systematic review of the literature, large administrative database analyses, including interrupted time series, that adequately control for confounding and patient selection bias appear to be the most practical way forward as future research agenda. Furthermore, prospective studies should not only investigate intraoperative clinical outcomes but also 30-day postoperative clinical events.

Footnotes

Acknowledgements

We would like to thank Alexandra (Sascha) Davis BA, MLIS, Librarian, Library and Learning Centre, Civic Campus, The Ottawa Hospital for developing the search strategy for this review.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article:

Dr. Fayad is an author for UpToDate and receives authorship bonus.

Dr. Shillcutt is an author for , has funding from the National Institute of Aging (1R03 AG045103-01A1), and is owner of Brave Enough, LLC.

Dr. Ansari received payment from lead author for his work as a senior meta-research methodologist on this project.

None declared for other co-authors.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplemental Material

Supplementary material is available for this article online.

References

Lang

Vignon

Weinert

, et al. Echocardiographic quantification of regional left ventricular wall motion with color kinesis. Circulation. 1996;93:1877-1885.

Clements

de Bruijn

. Perioperative evaluation of regional wall motion by transesophageal two-dimensional echocardiography. Anesth Analg. 1987;66:249-261.

Ellis

Shah

Briller

Roizen

Aronson

Feinstein

. A comparison of methods for the detection of myocardial ischemia during noncardiac surgery: automated ST-segment analysis systems, electrocardiography, and transesophageal echocardiography. Anesth Analg. 1992;75:764-772.

Sandham

Hull

Brant

, et al; Canadian Critical Care Clinical Trials Group. A randomized, controlled trial of the use of pulmonary-artery catheters in high-risk surgical patients. N Engl J Med. 2003;348:5-14.

Noordzij

Poldermans

Schouten

Bax

Schreiner

Boersma

. Postoperative mortality in The Netherlands: a population-based analysis of surgery-specific risk in adults. Anesthesiology. 2010;112:1105-1115.

Bilimoria

Liu

Paruch

, et al. Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013;217:833-842.

Spanjersberg

van Sambeeck

Bremers

Rosman

van Laarhoven

. Systematic review and meta-analysis for laparoscopic versus open colon surgery with or without an ERAS programme. Surg Endosc. 2015;29:3443-3453.

Denault

Fayad

Chen

. Focused ultrasound is the next step in perioperative care. Can J Anaesth. 2013;60:741-747.

Sharma

Fletcher

. A review of echocardiography in anaesthetic and peri-operative practice. Part 2: training and accreditation. Anaesthesia. 2014;69:919-927.

10.

Terkawi

Karakitsos

Elbarbary

Blaivas

Durieux

. Ultrasound for the anesthesiologists: present and future. ScientificWorldJournal. 2013;2013:683685.

11.

Jasudavisius

Arellano

Martin

McConnell

Bainbridge

. A systematic review of transthoracic and transesophageal echocardiography in non-cardiac surgery: implications for point-of-care ultrasound education in the operating room. Can J Anaesth. 2016;63:480-487.

12.

Higgins

JPT

Green

eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration; 2011.

13.

Moher

Liberati

Tetzlaff

Altman

; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62:1006-1012.

14.

Pai

Aniskevich

3rd Feinglass

, et al. Complications related to intraoperative transesophageal echocardiography in liver transplantation. Springerplus. 2015;4:480.

15.

Shillcutt

Montzingo

Agrawal

, et al. Echocardiography-based hemodynamic management of left ventricular diastolic dysfunction: a feasibility and safety study. Echocardiography. 2014;31:1189-1198.

16.

Myo Bui

Worapot

Xia

, et al. Gastroesophageal and hemorrhagic complications associated with intraoperative transesophageal echocardiography in patients with model for end-stage liver disease score 25 or higher. J Cardiothorac Vasc Anesth. 2015;29:594-597.

17.

Memtsoudis

Rosenberger

Loffler

, et al. The usefulness of transesophageal echocardiography during intraoperative cardiac arrest in noncardiac surgery. Anesth Analg. 2006;102:1653-1657.

18.

Shillcutt

Ringenberg

Chacon

, et al. Liver transplantation: intraoperative transesophageal echocardiography findings and relationship to major postoperative adverse cardiac events. J Cardiothorac Vasc Anesth. 2016;30:107-114.

19.

Burger-Klepp

Karatosic

Thum

, et al. Transesophageal echocardiography during orthotopic liver transplantation in patients with esophagoastric varices. Transplantation. 2012;94:192-196.

20.

Black

Muzzi

Nishimura

Cucchiara

. Preoperative and intraoperative echocardiography to detect right-to-left shunt in patients undergoing neurosurgical procedures in the sitting position. Anesthesiology. 1990;72:436-438.

21.

Hofer

Zollinger

Rak

, et al. Therapeutic impact of intra-operative transoesophageal echocardiography during noncardiac surgery. Anaesthesia. 2004;59:3-9.

22.

Markin

Sharma

Grant

Shillcutt

. The safety of transesophageal echocardiography in patients undergoing orthotopic liver transplantation. J Cardiothorac Vasc Anesth. 2015;29:588-593.

23.

Schulmeyer

Santelices

Vega

Schmied

. Impact of intraoperative transesophageal echocardiography during noncardiac surgery. J Cardiothorac Vasc Anesth. 2006;20:768-771.

24.

Suriani

Neustein

Shore-Lesserson

Konstadt

. Intraoperative transesophageal echocardiography during noncardiac surgery. J Cardiothorac Vasc Anesth. 1998;12:274-280.

25.

D’Angelo

Kline

Chen

Halpern

Cohen

. Utility of transesophageal echocardiography and pulmonary artery catheterization during laparoscopic assisted abdominal aortic aneurysm repair. Surg Endosc. 1997;11:1099-1101.

26.

Lin

Chen

Wang

. Use of transoesophageal echocardiography during cardiac arrest in patients undergoing elective non-cardiac surgery. Br J Anaesth. 2006;96:167-170.

27.

Denault

Couture

McKenty

, et al. Perioperative use of transesophageal echocardiography by anesthesiologists: impact in noncardiac surgery and in the intensive care unit. Can J Anaesth. 2002;49:287-293.

28.

Ball

Arora

Brady

Shah

Black

Allaf

. Defining the role of intraoperative transesophageal echocardiography during inferior vena caval tumor thrombectomy in renal cell carcinoma. Scientific Program for 31st World Congress of Endourology & SWL Program Book. J Endourol. 2013;27:A1-A470. doi:10.1089/end.2013.2001.

29.

American Society of Anesthesiologists and Society of Cardiovascular Anesthesiologists Task Force on Transesophageal Echocardiography. Practice guidelines for perioperative transesophageal echocardiography. An updated report by the American Society of Anesthesiologists and the Society of Cardiovascular Anesthesiologists Task Force on Transesophageal Echocardiography. Anesthesiology. 2010;112:1084-1096.

30.

Eisenberg

London

Leung

, et al. Monitoring for myocardial ischemia during noncardiac surgery. A technology assessment of transesophageal echocardiography and 12-lead electrocardiography. The Study of Perioperative Ischemia Research Group. JAMA. 1992;268:210-216.

31.

Fontes

Bellows

Ngo

Mangano

. Assessment of ventricular function in critically ill patients: limitations of pulmonary artery catheterization. Institutions of the McSPI Research Group. J Cardiothorac Vasc Anesth. 1999;13:521-527.

32.

Labovitz

Lewen

Kern

Vandormael

Deligonal

Kennedy

. Evaluation of left ventricular systolic and diastolic dysfunction during transient myocardial ischemia produced by angioplasty. J Am Coll Cardiol. 1987;10:748-755.

33.

Practice guidelines for perioperative transesophageal echocardiography. A report by the American Society of Anesthesiologists and the Society of Cardiovascular Anesthesiologists Task Force on Transesophageal Echocardiography. Anesthesiology. 1996;84:986-1006.

34.

Porter

Shillcutt

Adams

, et al. Guidelines for the use of echocardiography as a monitor for therapeutic intervention in adults: a report from the American Society of Echocardiography. J Am Soc Echocardiogr. 2015;28:40-56.

35.

Catena

Mele

. Role of intraoperative transesophageal echocardiography in patients undergoing noncardiac surgery. J Cardiovasc Med (Hagerstown). 2008;9:993-1003.

36.

Balshem

Helfand

Schunemann

, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64:401-406.

37.

Chatterjee

Koshy

Misra

Suparna

. Changes in left ventricular preload, afterload, and cardiac output in response to a single dose of mannitol in neurosurgical patients undergoing craniotomy: a transesophageal echocardiographic study. J Neurosurg Anesthesiol. 2012;24:25-29.

38.

Shanewise

Cheung

Aronson

, et al. ASE/SCA guidelines for performing a comprehensive intraoperative multiplane transesophageal echocardiography examination: recommendations of the American Society of Echocardiography Council for Intraoperative Echocardiography and the Society of Cardiovascular Anesthesiologists Task Force for Certification in Perioperative Transesophageal Echocardiography. Anesth Analg. 1999;89:870-884.

39.

Cahalan

Stewart

Pearlman

, et al. American Society of Echocardiography and Society of Cardiovascular Anesthesiologists task force guidelines for training in perioperative echocardiography. J Am Soc Echocardiogr. 2002;15:647-652.

40.

Mathew

Glas

Troianos

, et al. American Society of Echocardiography/Society of Cardiovascular Anesthesiologists recommendations and guidelines for continuous quality improvement in perioperative echocardiography. J Am Soc Echocardiogr. 2006;19:1303-1313.

41.

Beique

Ali

Hynes

, et al. Canadian guidelines for training in adult perioperative transesophageal echocardiography. Recommendations of the Cardiovascular Section of the Canadian Anesthesiologists’ Society and the Canadian Society of Echocardiography. Can J Anaesth. 2006;53:1044-1060.

42.

Flachskampf

Badano

Daniel

, et al. Recommendations for transoesophageal echocardiography: update 2010. Eur J Echocardiogr. 2010;11:557-576.

43.

Reeves

Finley

Skubas

, et al. Basic perioperative transesophageal echocardiography examination: a consensus statement of the American Society of Echocardiography and the Society of Cardiovascular Anesthesiologists. J Am Soc Echocardiogr. 2013;26:443-456.

44.

Hahn

Abraham

Adams

, et al. Guidelines for performing a comprehensive transesophageal echocardiographic examination: recommendations from the American Society of Echocardiography and the Society of Cardiovascular Anesthesiologists. J Am Soc Echocardiogr. 2013;26:921-964.

45.

Peng

Song

Wang

Liu

. Essential training steps to achieving competency in the basic intraoperative transesophageal echocardiography examination for Chinese anesthesiologists. Front Med. 2015;9:123-128.

46.

Wheeler

Steeds

Rana

, et al. A minimum dataset for a standard transoesphageal echocardiogram: a guideline protocol from the British Society of Echocardiography. Echo Res Pract. 2015;2:G29-G45.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB