Abstract
Introduction
Critically ill ECMO patients face high bleeding risks from anticoagulation and other factors. Although viscoelastic tests, like ROTEM, are widely used to assess hemostasis, evidence for their superiority over conventional coagulation tests (CCTs) is unclear. We assessed the prevalence of coagulopathy defined by ROTEM and CCT transfusion triggers, test concordance, and blood product use in bleeding ECMO patients.
Methods
This retrospective study included patients treated with ECMO at the Intensive Care Unit from April 1, 2020, to September 30, 2022, for whom ROTEM and CCTs were tested simultaneously to identify coagulopathy during bleeding. Coagulopathy and transfusion needs were assessed using the cut-offs from both methods.
Results
Of 88 patients, 34 (39%) had Clauss fibrinogen <1.5 g/L, and 21 (24%) had INR ≥1.8. Low FIBTEM A10, low EXTEM A10, and prolonged EXTEM CT were identified in 49 (56%), 43 (49%), and 36 patients (41%). EXTEM CT and INR were poorly associated, particularly for patients on VV-ECMO. FIBTEM A10 and Clauss fibrinogen showed a stronger relationship regardless of circuit type. For plasma and fibrinogen transfusions, ROTEM-based thresholds were more frequently met than CCT thresholds. Patients with abnormal ROTEM received more plasma and fibrinogen than those without [26/36 (72%) versus 21/52 (40%) and 45/49 (92%) versus 13/39 (33%), all p-values <0.01].
Conclusion
Abnormal ROTEM results were more frequent, often discordant with CCTs, and were associated with more plasma and fibrinogen transfusions in bleeding patients on ECMO. Further prospective studies are required to assess the potential clinical utility of ROTEM in ECMO settings.
Introduction
Viscoelastic assays (VEAs), including rotational thromboelastometry (ROTEM), are gaining popularity for detecting global hemostatic abnormalities and enabling rapid transfusion management. VEAs offer faster results as point-of-care tests when performed in clinical areas. However, it is still uncertain whether transfusion strategies guided by VEAs are more effective than conventional coagulation tests (CCTs) in aiding bleeding management, optimizing blood usage, or lowering bleeding-related mortality.1,2
Although CCTs, such as prothrombin time (PT)/international normalized ratio (INR), partial thromboplastin time (PTT), activated clotting time (ACT), and anti-Xa assays, are more widely available than VEAs, they were not specifically created to support transfusion resuscitation. 3 PT/INR can be prolonged in individuals with chronic liver disease and those taking vitamin K antagonists, and it is not recommended for guiding plasma transfusions in these settings.4,5 Likewise, prolonged PTT can occur due to various factors, including deficiencies in coagulation factors and the presence of both endogenous and exogenous factor inhibitors. This includes the use of commonly administered in-hospital anticoagulants, such as unfractionated heparin (UFH). 6
Vancouver General Hospital (VGH) is a quaternary care center providing critical care to severely ill patients from the Lower Mainland, Vancouver, and other hospitals across British Columbia. VGH introduced ROTEM in 2016 to aid transfusion management for major bleeding at the trauma bay and in operating theatres. The service was subsequently extended to the intensive care units (ICUs), where it is used for a broader range of surgical and medical patients. At VGH, only EXTEM and FIBTEM are utilized, while HEPTEM and APTEM are not available. Hence, heparin titration is performed using the PTT (nomogram for high-risk or low-risk bleeding), or in specific scenarios, such as heparin resistance, liver disease, or the presence of a lupus anticoagulant, anti-Xa activity is measured.
Between spring 2020 and fall 2022, there was a notable increase in the number of critical care patients admitted to the VGH ICU who required extracorporeal membrane oxygenation (ECMO) treatment for COVID-19-related acute respiratory distress syndrome. These patients often experienced clinically significant bleeding, defined as grade 2 or higher on the WHO bleeding scale, at least in part due to UFH used. 7 As a result, the VGH Hematology Laboratory observed a significant surge in ROTEM test volume, which impacted operational costs and staff workload. Notably, ROTEM and CCTs have comparable analytical turnaround times at VGH, with a median of 10 min, and are often performed in parallel at our institution.
Here, we report a retrospective study of ROTEM use to evaluate coagulopathy in patients requiring ECMO with bleeding. We identified patients who had concurrent ROTEM and CCT testing performed while receiving ECMO and reviewed their clinical features and ROTEM/CCT results. Our primary objective was to assess the frequency of ROTEM and CCT abnormalities in this population and the relationship between ROTEM and CCT results. We also compared the association of ROTEM and CCT abnormalities with blood component and product needs for this patient population.
Patients and methods
Study design, setting, and population
This retrospective observational study was conducted in the VGH ICU and included all patients aged 18 years or older who required ECMO treatment and experienced at least one episode of bleeding of WHO grade 2 or higher from April 1, 2020, to September 30, 2022. Only those who underwent both CCTs and ROTEM testing simultaneously during bleeding management were included. The majority of patients were anticoagulated with a low target UFH infusion, using a weight-based nomogram. Variance in this strategy occurred in the presence of bleeding, in diagnoses requiring full anticoagulation, such as venous thromboembolism, and at the attending physician’s discretion. Anticoagulants were paused or discontinued in some cases during bleeding. There was no difference in anticoagulation management protocols between venovenous (VV)- and venoarterial (VA)-ECMO circuits.
The University of British Columbia Institutional Research Ethics Board waived the requirement for informed consent, considering the retrospective nature of this study.
The primary objectives were to determine (1) the prevalence of laboratory-detected coagulopathy defined by ROTEM and CCT abnormalities, and (2) the relationship between CCT and ROTEM results in the above patient population.
The secondary objective was to assess the association between blood product usage and abnormal CCT and ROTEM values, as defined by our institution’s transfusion triggers for massive hemorrhage protocol in trauma and perioperative settings.
Data collection and measurement
Patient characteristics, including age, sex, ECMO treatment indications, bleeding manifestations, CCTs and ROTEM test indications, were obtained from patient charts. Laboratory results, including platelet count, INR, PTT, Clauss fibrinogen level, and ROTEM results, including EXTEM A10, EXTEM CT, EXTEM ML30, and FIBTEM A10, were collected from the hematology laboratory information system. Transfusion data were collected from the blood bank disposition records and patient charts.
Conventional coagulation tests (CCTs)
For CCTs, PT/INR, PTT, and Clauss fibrinogen levels were analyzed using HemosIL RecombiPlasTin 2G and HemosIL, SynthASiL, and HemosIL, Q.F.A. Thrombin (Bovine) [IL ACL, Werfen, Bedford, MA], respectively. Hemoglobin levels and platelet counts were analyzed using Cellpack DFL, Fluorocell WNR, and Fluorocell PLT [Sysmex Corporation, Kobe, Japan], and Cellpack DST, Sulfolyser SLS, and Lysercell WNR [Sysmex Reagents America Inc., Mundelein, IL, USA], respectively.
ROTEM analysis
ROTEM was analyzed using ROTEM-Delta with star-temR, ex-temR, fib-temRTem Innovations GmbH, Munich, Germany. For EXTEM ROTEM, the following variables were measured: clotting time [CT (s)]; clot formation time [CFT (s)]; A10 (clot amplitude 10 min after the end of CT); maximum clot firmness [MCF (mm)]; and maximum lysis time at 30 min [ML30 (%)]. For the FIBTEM assay, A10 and MCF were measured.
All sample testing was performed by medical laboratory technologists using laboratory-based instruments in accordance with institutional standard operating procedures. ROTEM devices, complete blood count (CBC) analyzers, and CCT analyzers are located in the central hematology laboratory, primarily to enable laboratory personnel to maintain quality control processes. The average analytical time for CCTs, CBC, platelet count, and Clauss Fibrinogen was also 10 min, identical to the first available ROTEM EXTEM A10 parameter, which allows clinicians to visualize the clot curve and numerical values (with online real-time results viewable via a hospital web portal).
All blood samples collected from the ICU were handled as STAT samples, requiring immediate delivery to the laboratory for processing, which takes less than 5 min. As a result, the turnaround times for CBC, CCTs, and ROTEM tests were consistently under 15 min from blood collection to result.
ROTEM versus CCT-based cut-offs for coagulopathy
Diagnostic thresholds for coagulopathy in bleeding patients were established institutionally in line with widely adopted recommendations, using ROTEM and CCT parameters as follows: ROTEM-based: EXTEM A10 <40 mm, EXTEM CT >100 s, EXTEM ML30 >10%, or FIBTEM A10 <10 mm. CCT-based: INR ≥1.8, PTT ≥1.5 times the upper normal limit, Clauss fibrinogen level <1.5 g/L, platelets <50 × 109/L. The ROTEM thresholds were derived from trauma-massive hemorrhage protocol (MHP) cut-offs in the literature and aligned with local VGH guidelines to inform transfusion decisions.8,9 The INR, PTT, and Clauss fibrinogen thresholds were established based on the Ontario regional MHP, developed through a modified Delphi process, which was subsequently adopted for use in British Columbia.10,11 Although the designed algorithms recommend blood product transfusion, the final decision lies with the ICU physicians.
Statistical analysis
Comparing the two testing platforms, we assessed the frequencies of coagulopathy defined by CCTs and ROTEM cut-offs and the number of allogeneic blood components and products transfused within 24 h of the initial ROTEM and CCT tests during bleeding events.
Comparisons between ROTEM and CCT tests were carried out using linear mixed-effects models with a random intercept term to account for multiple testing within individual patients. CCT and ROTEM values were normalized prior to performing the regression analysis and associations are reported as standardized regression coefficients (β) with 95% confidence intervals calculated using the profile likelihood method (see supplemental methods for additional details). The Mann-Whitney U-Test was performed to detect differences in blood transfusion requirement between the two test algorithms (significant deviations from a normal distribution identified). For categorical variables, frequencies were collected, and Fisher’s exact test was used based on sample size considerations. Cohen’s kappa and McNemar’s test were used to assess the discordance of ROTEM-based FIBTEM A10 (<10 mm) versus Clauss fibrinogen level (<1.5 g/L) and ROTEM-based EXTEM CT (>100 s) versus CCT-based INR (≥1.8). A p-value <0.05 was considered significant for all statistical tests. R software (version 4.5.2) was used for statistical analysis. 12
Results
Patient characteristics
Indications for ECMO treatment and bleeding manifestations.
Abbreviation: ARDS: acute respiratory distress syndrome.
Frequencies of ROTEM/CCT abnormalities
Of 88 patients, 46 (48%), 34 (39%), and 21 (24%) had PTT ≥1.5 times the upper normal limit, Clauss fibrinogen <1.5 g/L, and INR ≥1.8. FIBTEM A10 <10 mm, EXTEM A10 <40 mm, and EXTEM CT >100 s were identified in 49 (56%), 43 (49%), and 36 patients (41%). Seventeen patients (19%) had a platelet count <50 × 109/L.
Relationship between ROTEM and CCTs and the effects of ECMO circuit type.
Comparisons between paired CCT and ROTEM results were carried out using linear mixed-effects modeling to account for repeated sampling of the same patients, with results shown in Figure 1. The strongest associations based on standardized regression coefficients (β) were between Clauss fibrinogen versus FIBTEM A10 (β = 0.76, 95% CI = 0.69–0.82) and platelet count versus EXTEM A10 (β = 0.68, 95% CI = 0.59–0.77). These relationships were both independent of ECMO circuit type (Figure 1(a) and (b)). The association between INR and EXTEM CT was weaker compared to the other tests (β = 0.41, 95% CI = 0.31–0.50), with a significant interaction with the ECMO circuit type (Figure 1(C); p = 0.002 for interaction between circuit type and INR): while there was a modest relationship between INR and EXTEM CT for patients on VA-ECMO (β = 0.52, 95% CI = 0.35–0.70), there was a lower regression coefficient for patients on VV-ECMO (β = 0.20, 95% CI = 0.08–0.33). A notable difference between the VV- and VA-ECMO groups was that patients with severe COVID-19 accounted for a large proportion of the VV-ECMO (56%), but none of the VA-ECMO patients; however, excluding COVID-19 patients from the analysis did not alter the regression results, indicating that COVID-19 status was not driving the difference between the VA- and VV-ECMO patients (Figure S1). PTT acts as a proxy for heparin levels, which, in principle, could differentially affect EXTEM CT and INR. Including PTT as a covariate into the INR versus EXTEM CT models had no significant impact for both the full dataset and circuit type subgroups (no meaningful change to the INR coefficients and p-values > 0.05 for all likelihood ratio tests/PTT coefficients), indicating that heparinization was not affecting the association between the two tests. Relationship between ROTEM and conventional coagulation tests (CCTs) in bleeding patients on VA- and VV-ECMO. (a) Paired Clauss fibrinogen and FIBTEM A10. (b) Paired platelet count and EXTEM A10 measurements (for the subset of patients with normal FIBTEM A10). (C) Paired INR and EXTEM CT measurements. For all plots, black horizontal and vertical lines correspond to transfusion threshold values and colored lines to linear mixed-effects model regression fits (shaded regions = 95% confidence intervals). Standardized regression coefficients are reported in the main text.
Concordance between ROTEM and CCTs based on clinical transfusion thresholds
There was limited concordance between the proportion of patients meeting transfusion thresholds identified by ROTEM and CCT results at least once during bleeding, as shown in Table S1. For coagulopathy identified by EXTEM CT >100 s versus INR ≥1.8, there was 40% discordance (35/88, kappa = 0.12). Discordant hypofibrinogenemia was also identified, but to a lesser degree, when comparing FIBTEM A10 <10 mm and Clauss fibrinogen <1.5 g/L [19/88 (22%) discordant, kappa = 0.58]. In both cases, abnormalities were more frequently detected by ROTEM compared to the corresponding CCT (McNemar p = 0.02 and 0.001 for EXTEM CT vs INR and FIBTEM vs Clauss fibrinogen, respectively; see Table S1). Of note, when a Clauss fibrinogen cutoff of 1.7 g/L (derived from regression analysis) is used instead of the 1.5 g/L cutoff, there is improved concordance with FIBTEM A10 (16% discordant, kappa = 0.68, McNemar p = 0.42). In contrast, an INR of 1.8 was already optimal based off regression indicating that the discordance between INR and EXTEM CT was not due to the applied thresholds. Comparing EXTEM A10 versus platelet count in the subset of patients with normal FIBTEM A10 (n = 39) showed 23% discordance (kappa = 0.18, McNemar p = 0.51).
Blood component and product requirement
Plasma product transfusion frequency categorized by ROTEM and CCT abnormalities during bleeding episodes.
Total red blood cell (RBC) and plasma product usage based on ROTEM/CCT abnormality profiles are shown in Figure 2. Patients with both abnormal EXTEM CT and INR required significantly more RBC units and received more FP than patients without abnormalities (Figure 2(a) and (b)). On the other hand, patients who had only abnormal EXTEM CT received more FP but did not receive significantly more RBCs than those with both normal tests, whereas patients with only an elevated INR did not receive more RBCs or FP (Figure 2(a) and (b)). Patients with abnormal FIBTEM A10, with or without a concordantly abnormal Clauss fibrinogen, received significantly more RBCs and FC compared to patients with both normal, whereas patients with only abnormal Clauss fibrinogen did not receive more of either RBCs or FC (Figure 2(c) and (d)). RBC and plasma product utilization categorized by ROTEM and CCT abnormalities. (a) Total frozen plasma (FP) and (b) total RBCs transfused per patient by EXTEM CT and INR result status. (c) Total fibrinogen concentrate (FC) and (d) total RBCs transfused per patient by FIBTEM A10 and Clauss fibrinogen result status. For all categories, abnormal (Abn) indicates that the patient had ≥1 test result that was at an abnormal level as defined in Table 2. All data is displayed using default boxplot parameters from R with horizontal lines corresponding to medians and boxes showing the interquartile range. p-values calculated by the Mann-Whitney U test by comparing to the “Both Normal” group as reference.
Discussion
Extracorporeal life support, such as ECMO, is a critical treatment for cardiac or respiratory failure, but it is known to be highly associated with bleeding complications. Defining optimal strategies to assess coagulopathy to guide transfusion in this population remains a challenge. Here, we have analyzed a cohort of ECMO patients with significant bleeding who had simultaneous CCT and ROTEM testing performed. We found that ROTEM abnormalities were significantly more frequent than CCT abnormalities in this population and were associated with more plasma product transfusions.
The pathophysiology of coagulopathy in ECMO patients is multifactorial, involving anticoagulants used to prevent circuit thrombosis, platelet dysfunction, increased fibrinolysis, and local injuries at cannula sites. 13 Due to the UFH commonly used to maintain the ECMO circuit, certain CCTs, such as PTT, cannot reliably detect coagulation factor deficiencies. VEAs, such as ROTEM, are being increasingly utilized to provide rapid global hemostasis assessment and goal-directed treatment in ECMO patients, despite uncertain clinical effectiveness based on the available evidence. Functional VEAs, such as ROTEM, did not provide additional information to CCTs or predict bleeding events during VV-ECMO, as described by Hellmann et al. 14 On the other hand, ROTEM or thromboelastography (TEG)-guided strategy led to more cryoprecipitate transfusions compared to those managed by CCTs in pediatric ECMO patients. 15 Also, a systematic review by Jiritino et al. did not demonstrate an improvement in primary clinical outcomes, including bleeding, thrombosis, or ECMO circuit changes, by the viscoelastic point-of-care-guided algorithm. 16 However, these studies were often limited in terms of not sufficiently capturing the bleeding events or being underpowered for significance.
ROTEM was widely employed to assess coagulation status in ECMO-treated patients at our institution throughout the study period, with frequent abnormalities identified. Patients with ROTEM abnormalities tended to receive more plasma and fibrinogen transfusions, even when INR or Clauss fibrinogen values were within normal ranges. These findings raise the possibility that intensivists at our center considered ROTEM findings in diagnosing coagulopathy and guiding the management of hemostatic blood components and products. However, it also remains possible that patients with isolated ROTEM abnormalities, in some instances, had more clinically significant coagulopathy that triggered transfusions.
ROTEM EXTEM CT and INR were often discordant, with ROTEM more frequently detecting abnormalities than INR in our cohort. Surprisingly, patients on VV-ECMO showed a significantly weaker relationship between INR and EXTEM CT, and a higher rate of isolated EXTEM CT abnormalities compared with patients on VA-ECMO. COVID-19 status alone did not explain the observed patterns, but it remains possible that other patient factors and/or factors inherent to the circuits could play a role (e.g., inflammatory state and longer average run times for VV-ECMO patients). Patients with isolated EXTEM CT abnormalities received more FP but had similar RBC transfusions to those without EXTEM CT abnormalities. In the ECMO setting, it remains to be seen whether patients with an isolated, prolonged EXTEM CT have a higher bleeding risk and will benefit from transfusion of hemostatic blood products. Our data suggests that the interpretation may also depend on the clinical context or underlying medical conditions.
ROTEM FIBTEM A10 showed a higher positivity than Clauss fibrinogen for detecting hypofibrinogenemia using the selected thresholds; however, based off linear regression analysis a higher Clauss fibrinogen cutoff of 1.7 g/L instead of 1.5 g/L is more analogous to the ROTEM threshold. Using a higher Clauss fibrinogen cutoff could therefore largely negate differences in sensitivity; however, there is no well-established evidence to support maintaining a fibrinogen level above 1.7 g/L for bleeding control outside the context of postpartum hemorrhage. 17 Patients with at least one FIBTEM A10 abnormality were more frequently given fibrinogen replacement than those without, and this remained true even among patients who never met the Clauss fibrinogen transfusion threshold. Notably, a considerable proportion of patients without any FIBTEM A10 or Clauss fibrinogen abnormalities still received at least one fibrinogen transfusion (70% of this group). This could, in part, be due to patients receiving empirical fibrinogen to achieve balanced transfusion resuscitation at the beginning of bleeding episodes, as supported by the observation that this group received the least total fibrinogen.
ROTEM EXTEM A10 is, in principle, affected by abnormalities in platelet concentration and function, providing a theoretical advantage compared to relying solely on CBC to guide platelet transfusion. However, in practice, relying on EXTEM A10 to assess platelets is limited to patients with concurrently normal FIBTEM A10 values due to the confounding effect of fibrinogen. In our cohort, there was a fairly strong association between the EXTEM A10 values and platelet count from CBC analysis, suggesting a limited contribution of platelet dysfunction to the ROTEM results. Furthermore, only a handful of patients met the ROTEM-based transfusion threshold (i.e., low EXTEM A10 with normal FIBTEM A10), preventing detailed analysis of whether ROTEM influenced platelet transfusion decisions.
ROTEM is more expensive than CCTs. Using the calculated consumable costs of ROTEM and CCTs from 2025, PTT, PT/INR, Clauss fibrinogen, and ROTEM cost 0.06, 0.10, 0.52, and 46.04 Canadian dollars per test, respectively. ROTEM-Delta also requires more hands-on time from technologists for tasks such as reagent warming and pipetting than CCTs, which should be factored into operational costs. Utilizing ROTEM alongside CCT may not be essential in settings without evidence that it provides advantages over CCTs in transfusion management, particularly at centres where the turnaround times are comparable. An exception to this is when the ROTEM analyzer is positioned at the bedside, reducing transportation delays and workload, thereby improving overall workflow efficiency.
The primary limitation of this study was its retrospective nature, which could lead to confounding from sources such as the indication for ECMO, underlying medical conditions, and test-ordering biases. These factors could affect the relationship between paired CCT and ROTEM results, including the observed differences between VV- and VA-circuits. Controlling additional factors, such as disease severity/indication for ECMO, was not possible due to sample size/statistical power considerations. Larger prospective studies would be of interest to further assess these and other variables that might differentially influence CCT and ROTEM results in the ECMO setting, and in cases of discordant CCT/ROTEM results, determine which is more predictive of true coagulopathy. With respect to the relationship between CCT/ROTEM results and transfusions, an important limitation is that the temporal relationship between abnormal test results and transfusions could not be reliably assessed. Furthermore, clinical considerations could have influenced whether intensivists relied on ROTEM or CCT results when deciding to transfuse, e.g., relying on bleeding severity/clinical signs of coagulopathy irrespective of test results. Conversely, some clinicians may avoid FP transfusion to prevent fluid overload. In addition, the inability to precisely assess bleeding control post-transfusion precluded evaluation of the effectiveness of bleeding management.
ROTEM transfusion thresholds are also varied across institutions, where ours are derived from trauma-based massive hemorrhage protocols and may not be optimal in the ECMO setting, where a unique pathophysiology is at play. Finally, this study did not assess adverse events related to increased plasma product use and thrombosis resulting from more blood transfusions guided by ROTEM.
In conclusion, we found discordant ROTEM and CCT results in bleeding patients undergoing ECMO treatment, particularly between INR and EXTEM CT, with more frequent ROTEM abnormalities and higher rates of plasma product transfusion in patients with isolated ROTEM abnormalities. It remains unknown whether isolated ROTEM abnormalities truly reflect coagulopathy and justify blood product transfusion, and what the optimal ROTEM-based transfusion thresholds are for this unique patient population. The additional expense of ROTEM and the potential risks associated with increased transfusion rates warrant further prospective outcome studies in targeted ECMO populations.
Supplemental material
Supplemental material - ROTEM versus conventional coagulation tests in evaluating coagulopathy and transfusion requirement in ECMO patients: A retrospective study
Supplemental material for ROTEM versus conventional coagulation tests in evaluating coagulopathy and transfusion requirement in ECMO patients: A retrospective study by Kevin E Shopsowitz, Robert Guo, Geoffrey Chan, Jian Mi, Tyler Smith, Kristine Roland, Jacqueline Trudeau, Gordon Finlayson, Hussein D Kanji, Sakara Hutspardol in Perfusion
Footnotes
Acknowledgments
K.E.S. and S.H. designed the research study, acquired and analyzed the data, wrote the first draft and edited the manuscript. R.G. and K.R. designed the research study, acquired the data, and reviewed and edited the manuscript. T.S. designed the research study and reviewed and edited the manuscript. J.M., G.C., and G.F. acquired the data and reviewed and edited the manuscript. J.T. and H.D.K. reviewed and edited the manuscript.
Ethical considerations
The University of British Columbia Institutional Research Ethics Board waived the requirement for informed consent, given the study’s retrospective nature.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author, S.H., upon reasonable request.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
