Towards prompt electrocardiogram acquisition in triage: Preliminary testing of a symptom-based clinical prediction rule for the Android tablet

Abstract

Background:

For patients with suspected acute coronary syndrome, international guidelines indicate that an Electrocardiogram (ECG) should be performed within 10 min of first medical contact, however success at achieving these guidelines is limited.

Aims:

The purpose of this study was to develop and perform initial testing of a clinical prediction rule embedded in a tablet application, and to expedite the identification of patients who require an electrocardiogram within 10 min.

Methods:

This derivation of the Acute Coronary Syndrome Application (AcSAP) comprised of three local studies, an unpublished audit and literature critique. The AcSAP was prospectively tested over four months in patients presenting to the Emergency Department (ED) of a Dublin teaching hospital. An audit form retrieved data pertaining to times of: registration to the emergency department, triage, first electrocardiogram and diagnosis. The AcSAP was subsequently evaluated by experienced triage nurses (n=18) who had utilised it.

Results:

The AcSAP was activated 379 times. Patients with ST Elevation Myocardial Infarction (STEMI) and non-ST Elevation Myocardial Infarction (NSTEMI) were significantly more likely to return a categorisation of ‘immediate ECG’ or ‘ECG within 10 min’ (p<0.001). There was a significant difference in ‘triage to ECG’ times across categories, the ‘immediate ECG’ categorisation resulting in the shortest time (p=0.002). Evaluations suggest that staff found the tool quick and easy to use and results seemed accurate.

Conclusion:

Testing of the AcSAP suggests that it accurately identifies patients who require an ECG within 10 min. As such, it has the potential to support the meeting of clinical guidelines for ECG acquisition.

Keywords

Door to electrocardiogram acute coronary syndrome emergency department technology nursing

Introduction

The electrocardiogram (ECG) is considered the most important diagnostic tool for patients with suspected acute coronary syndrome (ACS) as the presence of ST segment elevation drives subsequent life-saving treatment algorithms. International guidelines recommend that the ECG should be obtained and interpreted within 10 min of the patient’s arrival to the emergency department.^1,2 However, despite these recommendations, only 17–41% of patients with ACS receive an ECG within 10 min.^3–6 Many factors have been cited for impacting on ECG delays (Figure 1) and these include registration processes;³ patient demographics;⁴ poor staff knowledge of ACS symptoms;⁵ and inappropriate ACS triage acuity levels.⁶ As delays to ECG acquisition may lead to delays in ACS diagnosis and treatment,^7,8 it is imperative that clinical pathways for ACS incorporate a system which supports timely ECG acquisition. The aim of this article is to report on the development and initial testing of a clinical prediction rule, to expedite the identification of patients who require an ECG within 10 min.

Figure 1.

Simplified process map for electrocardiogram (ECG) acquisition in the emergency department (ED) and components of delay. ACS: acute coronary syndrome.

Methodology and results

There is wide consensus concerning the methodology for the development of a clinical prediction rule. There are three stages: stage 1, ‘derivation of the rule; stage 2, ‘internal and external validation’; and stage 3, ‘impact’.^9–11 This preliminary testing addresses the derivation and internal validation stages only. In order to maintain this chronological order the methods and results of each stage will therefore be presented together. The derivation stage, Stage 1(a), will firstly summarise the empirical evidence from studies which informed the development of the clinical predication rule. This will be followed by Stage 1(b), a description of the derivation of the prototype clinical prediction rule itself, the Acute Coronary Syndrome Application (AcSAP). Lastly, the much larger internal validation phase, Stage 2, will be described in which the stability, performance and usability of the AcSAP is tested/addressed.

Stage 1(a) derivation of the rule

Methods

The AcSAP derivation, from which the list of predictors for the clinical predication rule was derived, is outlined here, and comprises three local studies, an unpublished audit of medical charts and an examination of international literature.

Results

In the first study, O’Donnell et al.¹² examined ACS symptom presentation in a cohort of ACS patients (n=1947) from five teaching hospitals in Dublin, Ireland. Patients were interviewed 2–4 days following their ACS event and data were gathered using the ACS response to symptom index.¹³ The study identified that chest pain was the most commonly reported main/worst symptom in both women (68.8%, n=375) and men (72.8%, n=1021). Women reported a significantly greater number of symptoms than their male counterparts (3.4 vs 2.9; χ²=33.43, d.f.=10, p<0.0001). Approximately 30% of patients did not experience chest pain and for the majority of these patients shortness of breath was the main/worst symptom. Figure 2 outlines the most common main/worst symptom reported.

Figure 2.

The main/worst symptom reported in patients with Acute Coronary Syndrome.

In the second study, O’Donnell and Moser¹⁴ qualitatively examined ACS symptom presentation in a cohort of 42 ACS participants from two large teaching hospitals in Dublin, Ireland. Participants were interviewed 2–4 days after their admission to hospital. Analysis revealed two distinct ACS presentations – slow-onset ACS which involved the gradual onset of mild, intermittent atypical/typical symptoms (n=27), and fast-onset ACS which involved the sudden onset of continuous and severe symptoms (n=15).

In the third study, O’Donnell et al.¹⁵ validated these qualitative findings in a dataset from a randomised control trial in Dublin, Ireland. ACS symptoms were recorded (n=892), with 576 (65%) participants experiencing slow-onset ACS presentation and 316 (35%) fast-onset ACS. Patients with myocardial infarction (MI) (38%) were significantly more likely to experience fast-onset MI than other ACS patients (24%) (p<0.0001).

A three-month unpublished retrospective chart audit was performed in the Emergency Department (ED) of the hospital where the clinical prediction rule would be developed. In this audit, the ‘door to ECG’ of ‘walk-in’ patients with confirmed MI was examined, where ‘door’ was recorded as time of registration in the ED. The median ‘door to first ECG’ was reported as 21 min (25^th percentile=12, 75^th percentile=30; n=52).

Previous efforts to address the issue of delayed ECG acquisition have included: nurse education interventions; implementation of different ECG prioritisation processes; the use of resident ECG technicians and greeter nurses; and, finally, the use of decision/prediction rules.^16–21 Several risk scores for ACS have also been published,²² the most reputable of these are the Thrombolysis in Myocardial Infarction score (TIMI),²³ Global Registry of Acute Coronary Events score (GRACE)²⁴ and Heart Score for Major Cardiac Events (HEART).²⁵ Although not designed specifically for the ED or for expediting ECG acquisition, TIMI and GRACE are commonly applied and are recommended in European and American guidelines at the ED.²⁶ The HEART score on the other hand was developed specifically for the ED, however, as with TIMI and GRACE, it relies on the presence of chest pain, an ECG and troponin results to estimate a score. As 30% of patients experience no chest pain, these scores and other prediction rules are somewhat limited, particularly in the face of an ‘atypical’ symptom presentation. Whilst many of the above efforts have succeeded in reducing ECG times, the literature reviewed above recommended further research on developing protocols for ECG prioritisation and decision support; as well as more research on ways to support the implementation of clinical guidelines for prompt ECG acquisition in patients with ACS.^16–21

Stage 1(b) prototype development

The application-based clinical prediction rule for use on an Android tablet – termed the AcSAP was developed. The clinical premise of the application was based on the findings of the derivation studies and supported by international literature. The AcSAP inputs for each patient included demographics (gender, age, cardiac risk), main/worst symptom evaluation (main symptom, onset, intensity and nature) and, finally, all accompanying symptom evaluation. Simplistic weighting was applied to each item, which was determined by assessing the potential influence of this category of the variable on the risk of a cardiac event, such as diabetic/not diabetic.

The weightings were crudely derived from the international risk and risk-scoring literature and the derivation studies in the following ways. Demographic information contained just two items, gender and age. While men are more likely to experience MI, women have an excess mortality rate following their event, even when adjusting for confounding variables such as demographics and cardiovascular profile. Poor adherence to clinical guidelines for female patients has been cited as a possible contributing variable to this finding.²⁷ Therefore, both genders received an equal weighting of one to offset this gendered risk in women. As the risk of ACS increases with age,^25,28 older age-groups were associated with an increased weighting (Table 1). Cardiac risk contained two items, previous history of coronary artery disease (CAD) and diabetes, both of which are recognised as two major risk factors for ACS.²⁸ Each item received a weighting of one, and both items could be ticked as relevant. The HEART, TIMI and GRACE scores adopt similar types of weighting to reflect differences in age and cardiac risk.^22–24

Table 1.

Acute Coronary Syndrome Application (AcSAP) inputs and weighting.

Item		Weighting
Gender	Male or female	1
Age, years	>65	3
	40–65	2
	<40	1
Cardiac risk	Previous CAD	1
	Diabetes	1
Main/worst symptom (one item only ticked)	Chest pain or shortness of breath or collapse	3
	Arm/shoulder pain or indigestion or neck/jaw pain	2
	Sweating or dizziness or palpitations	1
Onset	Sudden	2
Onset	Gradual	1
Intensity	Severe	3
	Moderate	2
	Mild	1
Nature	Continuous	2
Nature	intermittent	1
Accompanying symptoms:	Any amount of items ticked	1 per item

CAD: coronary artery disease.

Symptom weightings were addressed in two subsections: (a) main/worst presenting symptom and (b) other accompanying symptoms. With support from the international literature of ACS patients and the derivation studies, these presentation variables were weighted, on incidence, with ‘typical’ and ‘atypical’ symptom categorisation and the phenomena slow-onset and fast-onset ACS.^14,21 A weighting of three was assigned to the main/worst symptoms of chest pain, shortness of breath or collapse as each has a strong association with ACS.^15,26–29 Chest pain is considered to be the main symptom of ACS, however, in patients who do not experience chest pain (approximately 30% of patients,) shortness of breath is the main symptom.^15,21,29 Collapse with or without cardiac arrest is strongly associated with ACS, especially in the pre-hospital environment.² A weighting of two was assigned to the less frequently cited main symptoms of arm/shoulder or neck or jaw pain,^21,30 and a weighting of one assigned to sweating, dizziness and palpitations given as the main symptom.^14,21 A weighting of one was assigned to each of the accompanying symptoms to reflect the increasing number of symptoms associated with ACS presentation in women and the aged.^14,21,29

The AcSAP then calculated one of the following four ECG priority results: immediate ECG required; ECG within 10 min; ECG within 20 min; routine ECG (within 24 h) (Table 2). Any patient scoring 10 or above was deemed to require an immediate ECG or ECG within 10 min (positive screen). All triage staff from the ED received training in the use of the AcSAP. It was emphasised during training that the AcSAP was merely a tool to aid clinicians in identifying patients who require an urgent ECG.

Table 2.

Acute Coronary Syndrome Application (AcSAP) weighted results.

Weighted results	Score
Immediate ECG	>12
ECG within 10 min	10–12
ECG within 20 min	8–9
Routine ECG (within 24 h)	⩽7

ECG: electrocardiogram.

To test inter-rater reliability, all triage staff were shown four video scenarios of different actors presenting with ACS. Staff were asked to activate the AcSAP and enter relevant data as presented by the actor in each video. Intra-class correlation coefficients (ICCs) were derived. The evaluation items were divided into three: (a) demographic (b) main symptom and (c) accompanying symptoms. The level of inter-rater agreement (and related 95% confidence intervals) were 99.8% (99.5–99.9), 93.8% (88.5–97.2) and 97.2%. (95.9–98.2) respectively, and all were considered to be acceptable scores.

Stage 2 internal validation

Methods

The AcSAP was prospectively validated over a four-month period in the ED of the Dublin teaching hospital. The study conformed with the principles outlined in the Declaration of Helsinki, and ethical approval was received from the local Research Ethics Committee. Eligibility criteria included: (a) all non-trauma ‘walk-in’ patients presenting to the ED; and (b) patients aged >18 years.

Data collection

Three Android tablets with the AcSAP were secured in triage cubicles. Following the usual assessment, the triage nurse activated the application and entered the required information. An audit form was used to retrieve data from the electronic patient information system for each patient and included: registration to the ED; triage; first ECG; and diagnosis (confirmed by serial ECG data and troponin testing).

When the AcSAP testing was completed, an anonymous five-item questionnaire was employed for usability testing of the application. A poster outlining details of the study was displayed in the staffroom of the ED, together with copies of the questionnaires. Triage nurses who had used the AcSAP were invited to complete the questionnaire. A box was provided in the staff room for the collection of completed questionnaires. Three Likert scale type questions asked staff to rate the usability, efficiency and usefulness of the AcSAP. Two additional questions asked staff to comment on (a) benefits to using the AcSAP and (b) challenges encountered when using the AcSAP.

Data analysis

SPSS version 22 was used for analysis. Descriptive statistics, percentages and numerical frequency were used to describe the population and usage of the AcSAP. Triage and door to ECG times were positively skewed (skewness values of 5.26 and 5.32 respectively) and expressed as means for comparison with the literature and medians for statistical correctness. Non-parametric Kruskal-Wallis test and Mann-Whitney tests were used as appropriate. There were no missing data. The significance level was set at 0.05.

The sensitivity and specificity of the AcSAP to correctly identify ACS and cardiac symptoms were tested using 4×4 contingency tables. Strong specificity and sensitivity results should be ⩾0.80. Comments from open-ended questions from the staff evaluation of the AcSAP were reviewed, coded and categorised.

Results

Patient characteristics

The AcSAP was activated 379 times during the four-month testing period, resulting in data for n=379 patients. Most were male (204; 54%) and aged 40–65 years (185; 49%). Chest pain was the main presenting symptom, reported by 172 (45%). Only 98 (26%) patients reported diabetes and/or previous CAD. A total of 201 (53%) received a cardiac diagnosis of which 67 (18%) were diagnosed with ACS. Six of these patients received a diagnosis of ST elevated myocardial infarction (STEMI), 12 had non-STEMI (NSTEMI) and 49 had unstable angina (see Table 3).The remaining 134 (35%) had other cardiac illnesses.

Table 3.

Sample characteristics.

		n=379n (%)	MI(18)n (%)	UA(49)n (%)	Other cardiac(134)n (%)	Non-cardiac(178)n (%)
Age in years	<40	82 (22)	-	2 (4)	19 (14)	61 (34)
	40–65	185 (49)	10 (56)	26 (53)	73 (55)	76 (43)
	>65	112 (29)	8 (44)	21 (43)	42 (31)	41 (23)
Gender	Female	175 (46)	5 (28)	22 (45)	54 (40)	94 (53)
Gender	Male	204 (54)	13 (72)	27 (55)	80 (60)	84 (47)
Cardiac risk	None	281 (74)	11 (61)	32 (65)	94 (70)	144 (81)
	Previous CAD	72 (19)	6 (33)	13 (27)	29 (22)	24 (13)
	Diabetes	17 (5)	-	2 (4)	10 (7)	5 (3)
	Previous CAD and diabetes	9 (2)	1 (6)	2 (4)	1 (1)	5 (3)
Symptoms
Main symptom	No main symptom	65 (17)	-	-	6 (4)	59 (33)
	Chest pain	172 (45)	15 (83)	35 (71)	74 (55)	48 (27)
	Shortness of Breath	41 (11)	-	8 (16)	12 (9)	21 (12)
	Dizziness	24 (6)	-	1 (2)	9 (7)	14 (8)
	Arm/shoulder pain	22 (6)	-	1 (2)	5 (4)	16 (9)
	Palpitations	21 (5)	-	1 (2)	16 (12)	4 (2)
	Collapse	19 (5)	2 (11)	1 (2)	7 (5)	9 (5)
	Indigestion	10 (3)	1 (6)	-	4 (3)	5 (3)
	Sweating	3 (<1)	-	-	1 (1)	-
	Neck/jaw pain	4 (1)	-	2 (4)	-	2 (1)
Onset of main symptom	None reported	63 (16)	-	-	6 (5)	57 (32)
	Sudden	132 (35)	9 (50)	18 (37)	57 (42)	48 (27)
	Gradual	184 (49)	9 (50)	31 (63)	71 (53)	73 (41)
Intensity of main symptom	None reported	65 (17)	-	1 (2)	6 (5)	58 (32)
	Severe	44 (12)	8 (44)	8 (18)	16 (12)	12 (7)
	Moderate	177 (47)	7 (39)	29 (59)	75 (56)	66 (37)
	Mild	93 (24)	3 (17)	11 (22)	37 (27)	42 (24)
Nature of main symptom	None reported	68 (18)	-	2 (4)	7 (5)	59 (33)
	Continuous	120 (32)	11 (61)	22 (45)	39 (29)	48 (27)
	Intermittent	191 (50)	7 (39)	25 (51)	88 (66)	71 (40)
Accompanying symptoms ^a	No accompanying symptoms	101 (27)	-	8 (16)	24 (18)	66 (37)
	Shortness of breath	75 (20)	7 (39)	16 (33)	35 (26)	17 (10)
	Arm/shoulder pain	39 (10)	4 (22)	10 (20)	18 (13)	7 (4)
	Chest pain	29 (8)	0 (0)	4 (8)	13 (10)	12 (7)
	Fatigue	29 (8)	1 (6)	4 (8)	-	9 (5)
	Back pain	28 (7)	0 (0)	3 (6)	5 (4)	20 (11)
	Nausea	27 (7)	1 (6)	-	15 (11)	18 (10)
	Dizziness	26 (7)	1 (6)	1 (2)	5 (4)	7 (4)
	Neck/jaw pain	21 (6)	3 (17)	1 (2)	10 (8)	7 (4)
	Sweating	16 (4)	1 (6)	2 (4)	8 (6)	5 (3)
	Palpitations	14 (4)	-	3 (6)	-	6 (3)
	Collapse	11 (3)	2 (11)	1 (2)	5 (4)	3 (2)
	Indigestion	7 (2)	1 (6)	-	3 (2)	3 (2)
	Fear	4 (1)	-	1 (2)	1 (1)	2 (1)

CAD: coronary artery disease; MI: myocardial infarction; NSTEMI: non-ST elevated myocardial infarction; STEMI: ST elevated myocardial infarction; UA: unstable angina.

Patients could cite more than one accompanying symptom.

AcSAP results

Of the 379 activations, 31% (n=119) received a result of ‘immediate ECG’ whilst 40% (n=151) received ‘ECG within 10 min’. The remaining 29% received a result of ‘ECG within 20 min’ (12%) or ‘routine ECG’ (17%) respectively.

For patients diagnosed with STEMI (n=6) or NSTEMI (n=12), a result of ‘immediate ECG’ or ‘ECG within 10 min’ was returned. For those with unstable angina (n=49), 96% returned a result of ‘immediate ECG’ or ‘ECG within 10 min’ while 4% returned a result of ‘ECG within 20 min’. Of other cardiac presentations (n=135), 88% returned a result of ‘immediate ECG’ or ‘ECG within 10 min’ while 7% and 5% returned results of ‘ECG within 20 min’ and ‘routine ECG’ respectively. Patients with STEMI and NSTEMI were significantly more likely than any other condition, to return a result of ‘immediate ECG’ or ‘ECG within 10 min’ (p<0.001) (Figure 3).

Figure 3.

Acute Coronary Syndrome Application (AcSAP) results and diagnosis.

Door and triage to ECG times

The median ‘door to ECG’ was 18 min (interquartile range 12–29 min) and 15.8 % met the guideline of receiving an ECG in 10 min or less. The median ‘triage to ECG’ was 12 min (interquartile range 8–22 min). Kruskal-Wallis tests showed that there was a statistically significant difference in door and triage to first ECG times between the different categorisations, χ² (9.47, p=0.09) and χ² (12.84, p=0.02) respectively (Table 4). The ‘immediate ECG’ categorisation resulted in the shortest time of both door and triage to ECG compared to the other categories (Table 4).

Table 4.

Acute Coronary Syndrome Application (AcSAP) results and electrocardiogram (ECG) timeframes.

	n	Mean (min)	Median (min)	Kruskal-Wallis (p values)
Door to 1st ECG				0.009
Immediate ECG	116	22.9	16.6
ECG in 10 min	142	27.4	17.5
ECG in 20 min	38	29.2	20.0
Routine ECG	42	77.1	24.5
Triage to 1 ^st ECG				0.002
Immediate ECG	116	16.6	11.0
ECG in 10 min	142	22.8	12.0
ECG in 20 min	38	24.8	15.0
Routine ECG	42	71.1	18.5

AcSAP sensitivity and specificity testing

Sensitivity and specificity tests were first performed on the patient ACS only and non-ACS subgroups. Non-ACS included those patients with other cardiac diseases such as stable angina, valvular heart disease etc. and those with non-cardiac diseases such as gastrointestinal disturbances.

Results of the sensitivity and specificity testing indicate that there were no false negatives for ACS – i.e. no-one with ACS was incorrectly categorised (Table 5). A sensitivity result of 1.00 indicates that the AcSAP has a very strong sensitivity for correctly identifying those with ACS. In non-ACS, there were 64 false positives for ACS, in which 64 patients who had not ACS were diagnosed as having ACS. The specificity results of 0.20 indicate a weak specificity, with a poor ability to correctly identify those without ACS.

Table 5.

Sensitivity and specificity of Acute Coronary Syndrome Application (AcSAP) for acute coronary syndrome (ACS) and non-ACS diagnoses.

	ACS	Non-ACS
Test positive	67	249
Test negative	0	64
Total	67	313

Prevalence=0.176, sensitivity for test positive=1, specificity for test negative=0.204.

These tests were repeated on cardiac (both ACS and non-ACS cardiac disease) and non-cardiac patients. There were 19 false negatives for cardiac symptoms i.e. 19 patients presenting with a cardiac event who were incorrectly identified as having a non-cardiac event (Table 6). In non-cardiac patients there were 90 false positives, therefore 90 patients who were non-cardiac were diagnosed as cardiac. These results were tested using 4×4 contingency tables for specificity and sensitivity. Sensitivity results were 0.906 indicating that the AcSAP has a very strong sensitivity for correctly identifying those with a cardiac event. Specificity results were 0.506, indicating moderate specificity, moderate ability to correctly identify those without a cardiac event.

Table 6.

Sensitivity and specificity of Acute Coronary Syndrome Application (AcSAP) for cardiac and non-cardiac diagnoses.

	Cardiac	Non-cardiac
Test positive	183	88
Test negative	19	90
Total	202	178

Prevalence=0.053, sensitivity for test positive=0.906, specificity for test negative=0.506.

Usability testing

The timing of the entire process was recorded for each activation and was calculated from the time of AcSAP activation to the time the result was retrieved. The mean process time was 43 seconds (standard deviation (SD); 18.3, range 29–90 seconds). Eighteen questionnaires (78%) were returned from a total of 23 triage nurses who had used the AcSAP. Each had activated the application on at least two occasions. Results suggest that staff found the application either very easy (n=17) or easy to use (n=1). The inputting of data and retrieving the result was either very quick (n=12) or quick (n=6). Eight staff found the AcSAP to be very useful whereas n=10 staff found it useful. The qualitative comments about benefits were grouped into two categories: ‘usability and expedience’ and ‘clinical usefulness’. Challenges encountered were grouped into two categories ‘clinical usefulness’ and ‘general’ challenges (Table 7).

Table 7.

Qualitative comments about Acute Coronary Syndrome Application (AcSAP).

Benefit: usability and expedience
• Very easy to use • Takes very little time • Looks great, very neat • Position in cubicles good – secure and very neat • Very cheap solution – very easy to use
Benefit: clinical usefulness
• Results seem accurate and expected • Confirmed the diagnosis I had made in my head • Very intuitive – confirms what you were thinking • Great for providing a prompt about the urgency of an ECG – maybe we’ll get a technician now? • Why don’t ambulance crews have something like this?
Challenges encountered: clinical usefulness
• Great tool for junior staff but not sure experienced staff need this • Maybe good for smaller hospitals or GP practices – not sure it’s needed here. • I didn’t use it when I knew what was wrong with the patient
Challenges encountered: general
• Something extra to do during triage • Less likely to enter data into tablet when busy • Who will do all these ECGs? • Great idea in theory but ECG technician needed if this is to work properly

ECG: electrocardiogram.

Discussion

The AcSAP was developed as a tool to assist triage nurses to expedite the identification of patients who require an ECG within 10 min. To the best of our knowledge, this is the first published study, to develop and test a clinical prediction rule for ECG acquisition in the ED, using the Android tablet.

The AcSAP performed well in testing and discriminated well between patients requiring an ECG and those who did not. All patients with NSTEMI and STEMI received a result of ‘immediate ECG’ or ‘ECG within 10 min’, which is the desired result for this cohort. Results suggest that the AcSAP has a very strong sensitivity for correctly identifying those with a cardiac event and those with an ACS event. The AcSAP also demonstrated moderate specificity for identifying those without a cardiac illness, yet weaker specificity for its ability to identify those without an ACS diagnosis. In this initial validation of the AcSAP, the tool appears to be reliable and dependable. An important measure of any new tool is the reliability of predictor variables.¹⁰ As the AcSAP is highly dependent on findings from clinical examination, ascertaining the consistency or reproducibility of the findings by different clinicians was important to establish. As the level of inter-rater agreement ranged from 93–99%, this suggests that predictor variables were reliable and the tool dependable.

In clinical terms, the AcSAP demonstrates excellent clinical sensitivity insofar as it appears to ‘capture’ all of the desired population. The moderate specificity result suggests that it appears to err on the side of caution by including those patients who do not have a cardiac diagnosis, but who may ‘resemble’ the cardiac population. This is clinically ‘intuitive’ and is reflective of clinical practice, especially in the case of non-cardiac patients who present with chest pain. However, with this specificity result, the AcSAP also demonstrates a moderate ability to correctly identify those without a cardiac event. As such, it is unlikely to create additional ECG workload, to that which currently occurs in practice. Rather, it is more likely to provide assistance in identifying those most likely to require an ECG. Thus, the AcSAP is a reliable and useable tool for clinical practice. A larger study may examine the number of ECG’s generated by the tool versus those performed during usual practice. Further research is also needed to identify how more specific the AcSAP is in correctly identifying cardiac events as compared to current clinical practice.

The median ‘door to ECG’ time was 18 min. Although this demonstrates an improvement on the comparative time of the unpublished medical chart review (21 min), this timing is disappointing. The AcSAP, although accurately identifying patients in need of an ECG, did not result in the attainment of clinical guidelines for 10-minute ECG acquisition.^1,2 Some explanation for these results may lie in the current practice of this ED, to have all ECGs performed by an ECG technician (Figure 3). At the time of testing there was no resident ECG technician in the ED, so if needed he/she would be paged to come to the department. As such, any inherent delays to ECG attainment were subsequently compounded upon. Therefore, although the AcSAP quickly identified patients who required an ECG (mean 43 s), the process of acquiring the ECG in this busy ED, appears to be the main component of delay. International research suggests that ECG acquisition processes can be further streamlined by including an ECG technician within the process,¹⁸ an issue highlighted in feedback from staff within this study.

However, even without an ECG technician in ED, the AcSAP seemed to expedite ECG acquisition. Patients who returned an ‘immediate ECG result’ were statistically more likely than others to have quicker ‘door to ECG’ and ‘triage to ECG’ times. Future validation of the AcSAP is recommended in an ED where an ECG technician is resident, so that the true potential of this application can be realised.¹⁸

Another important measure of a new clinical prediction tool is the measurement of its ‘sensibility’ and describes whether it is deemed by staff to be clinically reasonable, easy to use and whether it provides a course of action.¹⁰ The evaluation of the AcSAP reflects general approval of the tool, suggesting it was very quick and easy to use. The position of the tablet in the triage cubicle was convenient, and staff felt that it was ‘neat and secure’. The AcSAP results seemed ‘intuitive’ to them, they had ‘expected the results’ and therefore deemed the ‘results to be accurate’. In addition, as the tool proposes a course of action for staff to take, the international literature suggests that when tools have action outcomes, they are more likely to be used by staff.¹⁰ As such, this evaluation by staff suggests that the AcSAP has strong sensibility and is a useful support for the implementation of international guidelines for prompt ECG acquisition.

However, staff still considered that it was ‘an extra job to do’ and during busy times would either ‘forget to use it’ or ‘opt not to activate it’. Many staff also felt that, for the majority of patients they assessed, they would already ‘know the answer’. This evaluation by staff is quite predictable when considering that the AcSAP was prospectively validated in a busy ED and by experienced triage staff. Although staff were asked to activate the tool for all non-trauma patients, which included both cardiac and non-cardiac patients, they were not always able to do this during very busy times. Staff were also encouraged to look upon the tool as an aid to practice and, in so doing, opted not to use it when they felt it was not needed or they already knew the answer. This evaluation mimics normal and desirable practice by staff when using any clinical prediction tool.

Some staff suggested that the AcSAP would also be very useful for less experienced nurses, smaller hospitals and/or in the pre-hospital environment e.g. Emergency Medical Services (EMS) and general practitioner (GP) practices. Due to the strong performance and sensibility of the application, it is suggested that the AcSAP would indeed be suitable for areas of practice where assistance is required for expediting ECG acquisition. The literature pertaining to pre-hospital delays associated with ACS presentation suggests that the pre-hospital environment would benefit from a tool which identifies patients needing urgent ECGs. This would extend from GP practices to EMS crews who attend patients with symptoms suggestive of ACS, the AcSAP serving to expedite both ECG acquisition and the subsequent presentation of ACS patients to ED. Similarly, smaller hospitals, those without EDs and units with less experienced staff may also benefit from such an application. Future validation of the AcSAP may include these areas of practice.

Limitations

In this initial testing, simplistic weighting of scores was employed. The weightings were crudely derived from the international risk and risk-scoring literature, the derivation studies and were moderated based on the clinical experience of expert staff. Future validation studies are required to complete the external validation of the tool through the use of statistical modelling. The testing should further determine efficacy, effectiveness, reliability and dependability of the AcSAP.

During pilot testing, the AcSAP was activated for 379 ‘walk-in’ patients and included non-trauma patients admitted to the ED over a four-month period. However, we did not collect data on the number of patients on whom the AcSAP was not activated. Future testing may take account of these numbers and ascertain reasons for not activating the AcSAP.

The AcSAP quickly and accurately identified those patients in need of an ECG and although the door to ECG times improved, they were still beyond the times recommended in international guidelines. Future testing may examine ECG times in EDs where the acquisition of the ECG is further optimised i.e. where ECG technicians are resident to perform ECGs when the need is identified.

Although the AcSAP quickly identified patients in need of an ECG, this study did not examine the components of delay in obtaining the ECG. Future testing of the AcSAP may examine these factors and their impact on meeting international guidelines.

The AcSAP has undergone initial prospective validation with experienced triage staff in an ED of a large teaching hospital. As such, we have no data on the performance of the application in other areas of practice or with less experienced staff. Future testing should also include non-teaching hospitals, rural hospitals, community practices and EMS.

The AcSAP is portable and for this initial testing is limited to the Android tablet. However, in the future the AcSAP may have potential uses for other Android devices such as mobile phones.

In conclusion, this is the first published study, to develop and test a clinical predication rule for ECG acquisition in the ED, using the Android tablet. The initial testing of the AcSAP on ‘walk-in’ patients has highlighted the potential benefits of a simple and cost effective tool for identifying ECG acquisition for ACS patients, not only in the ED but also in other areas of care. As such, the AcSAP has the potential to support the implementation of international guidelines for ECG acquisition within 10 min.

Footnotes

Acknowledgements

The authors would like to thank the staff of St James’s Hospital Emergency Department, Dublin, for their support and commitment to the study.

Declaration of conflicting interests

The authors declare that there is no conflict of interest.

Funding

The Health Research Board, Grattan House, 67–72 Lower Mount Street, Grand Canal Dock, Dublin 2. Ireland.

Implications for practice

The Acute Coronary Syndrome Application (AcSAP) identifies patients requiring an urgent electrocardiogram.

The AcSAP has the potential to support the acquisition of electrocardiograms within 10 min.

As the AcSAP can assist in identifying patients in need of an electrocardiogram, it is useful in expediting European Society of Cardiology treatment algorithms for acute coronary syndrome.

References

Roffi

Patrono

Collet

et al . 2015 ESC guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation: Task Force for the Management of Acute Coronary Syndromes in Patients Presenting Without Persistent ST-Segment Elevation of the European Society of Cardiology (ESC). Eur Heart J 2016; 37: 267–315.

Antman

Anbe

Armstrong

et al . ACC/AHA guidelines for the management of patients with ST-elevation myocardial infarction: A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee to revise the 1999 guidelines for the management of patients with acute myocardial infarction). J Am Coll Cardiol 2004; 44: E1–E211.

Yiadom

Baugh

McWade

et al . Performance of emergency department screening criteria for an early ECG to identify ST-segment elevation myocardial infarction. J Am Heart Assoc 2017; 23: 1–23.

Zegre-Hemsey

Sommargren

Drew

BJ.

Initial ECG acquisition within 10 minutes of arrival at the emergency department in persons with chest pain: Time and gender differences. J Emerg Nurs 2011; 37: 109–112.

Bradley

Roumanis

Radford

et al . Achieving door-to balloon times that meet quality guidelines: How do successful hospitals do it? J Am Coll Cardiol 2005; 46: 1236–1241.

Sanders

DeVon

HA.

Accuracy in ED triage for symptoms of acute myocardial infarction. J Emerg Nurs 2016;42: 331–337.

Diercks

Peakock

Hiestand

et al . Frequency and consequences of recording an electrocardiogram >10 minutes after arrival in an emergency room in non-ST segment elevation acute coronary syndrome (from the crusade initiative). Am J Cardiol 2005; 97: 437–442.

Lambrew

Bowlby

Rogers

et al . Factors influencing the time to thrombolysis in acute myocardial infarction. Time to Thrombolysis substudy of the National Registry of Myocardial Infarction-1. Arch Intern Med 1997; 157: 2577–2582.

Laupacis

Sekar

Stiell

IG.

Clinical prediction rules: A review and suggested modifications of methodological standards. JAMA 1997; 277: 488–494.

10.

Stiell

Wells

GA.

Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med 1999; 33: 437–444.

11.

McGinn

Guyatt

Wyer

et al . Evidence-Based Medicine Working Group. Users’ guides to the medical literature, XXII: How to use articles about clinical decision rules. JAMA 2000; 284: 79–84.

12.

O’Donnell

McKee

O’Brien

et al . Gendered symptom presentation in acute coronary syndrome: A cross sectional analysis. Int J Nurs Stud 2012; 49: 1325–1332.

13.

Riegel

McKinley

Moser

et al . Psychometric evaluation of the Acute Coronary Syndrome (ACS) Response Index. Res Nurs Health 2007; 30: 584–594.

14.

O’Donnell

Moser

DK.

Slow-onset myocardial infarction and its influence on help-seeking behaviors. J Cardiovasc Nurs 2012; 27: 34–44.

15.

O’Donnell

McKee

Mooney

et al . Slow-onset and fast-onset symptom presentations in acute coronary syndrome (ACS): New perspectives on prehospital delay in patients with ACS. J Emerg Med 2014; 46: 507–515.

16.

Ballard

Bairan

Newberry

et al . Effect of education on a chest pain mnemonic on door-to-ECG time. J Emerg Nurs 2011; 37: 220–224.

17.

Marino

Mays

Thompson

EJ.

Bypass rapid assessment triage: How culture change improved one emergency department’s safety, throughput and patient satisfaction. J Emerg Nurs 2015; 41: 213–220.

18.

van Tulder

Roth

Weiser

et al . An electrocardiogram technician improves in-hospital first medical contact-to-electrocardiogram times: A cluster randomized controlled interventional trial. Am J Emerg Med 2012; 30: 1729–1736.

19.

Takakuwa

Burek

Estepa

et al . A method for improving arrival-to-electrocardiogram time in emergency department chest pain patients and the effect on door-to-balloon time for ST-segment elevation myocardial infarction. Acad Emerg Med 2009; 16: 921–927.

20.

Osborne

Ali

Lowery-North

et al . Ability of triage decision rules for rapid electrocardiogram to identify patients with suspected ST-elevation myocardial infarction. Crit Pathw Cardiol 2012; 11: 211–213.

21.

Glickman

Shofer

et al . Development and validation of a prioritization rule for obtaining an immediate 12-lead electrocardiogram in the emergency department to identify ST-elevation myocardial infarction. Am Heart J 2012; 163: 372–382.

22.

de AraújoGonçalves

Ferreira

Aguiar

et al . TIMI, PURSUIT, and GRACE risk scores: Sustained prognostic value and interaction with revascularization in NSTE-ACS. Eur Heart J 2005; 26: 865–868.

23.

Antman

Cohen

Bernink

et al . The TIMI risk score for unstable angina/non-ST elevation MI. JAMA 2000; 284: 835–842.

24.

Granger

Goldberg

Dabbous

et al.;

for the Global Registry of Acute Coronary Events Investigators. Predictors of hospital mortality in the global registry of acute coronary events

Arch Intern Med 2003; 163: 2345–2353.

25.

Six

Backus

Kelder

JC.

Chest pain in the emergency room: Value of the HEART score. Neth Heart J 2008; 16: 191–196.

26.

Lee

Chang

Matsuura

et al . Comparison of cardiac risk scores in ED patients with potential acute coronary syndrome Crit Pathw Cardiol 2011; 10: 64–68.

27.

Alabas

Gale

Hall

et al . Sex differences in treatments, relative survival, and excess mortality following acute myocardial infarction: National cohort study using the SWEDEHEART registry. J Am Heart Assoc. 2017; 6: 1–18.

28.

Piepoli

Hoes

Agewall

et al . 2016 European guidelines on cardiovascular disease prevention in clinical practice: The Sixth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of 10 societies and by invited experts): Developed with the special contribution of the European Association for Cardiovascular Prevention & Rehabilitation (EACPR). Eur Heart J 2016; 37: 2315–2381.

29.

Canto

Goldberg

Hand

et al . Symptom presentation of women with acute coronary syndrome. Arch Intern Med 2007; 167: 2405–2413.

30.

Angerud

Thylén

Sederholm Lawesson

et al . Symptoms and delay times during myocardial infarction in 694 patients with and without diabetes: An explorative cross sectional study. BMC Cardiovasc Disord 2016; 16: 108.