The diagnostic validity of clinical airway assessments for predicting difficult laryngoscopy using a grey zone approach

Abstract

Objectives

The diagnostic validity of clinical airway assessment tests for predicting difficult laryngoscopy in patients requiring endotracheal intubation were evaluated using receiver operating characteristic (ROC) curve analysis and a grey zone approach.

Methods

In this prospective observational study, patients were evaluated during a pre-anaesthetic visit. Predictive airway assessment tests (i.e. Modified Mallampati [MMT] classification; upper lip bite test [ULBT]; mouth opening; sternomental distance; thyromental distance [TMD]; neck circumference; neck mobility; height to thyromental distance [HT/TMD]; neck circumference-to-thyromental distance [NC/TMD]) were performed on each patient and LEMON, Naguib, and MACOCHA scores were also calculated. In addition, laryngeal images were acquired and assessed for percentage of glottic opening (POGO) scores. A POGO score of zero was categorized as difficult laryngoscopy.

Results

The incidence of difficult laryngoscopy was 14.4% (35/243). Although seven predictive airway assessments (i.e. MMT classification, ULBT, mouth opening, HT/TMD, NC/TMD, and the LEMON and Naguib models) predicted difficult laryngoscopy by ROC analyses, a grey zone approach showed that the parameters were inconclusive in approximately 70% of patients. From all the tests, the HT/TMD ratio showed the highest sensitivity (80.0%) and ULBT had the highest specificity (95.2%).

Conclusion

Using the grey zone approach, all predictive airway assessment tests showed large inconclusive zones which may explain previous inconsistent results in the prediction of difficult laryngoscopy. Our results suggest that the usefulness of clinical airway evaluation tests for predicting difficult laryngoscopy remains controversial.

Clinical trial registration

ClinicalTrials.gov (NCT01719848)

Keywords

Airway assessment difficult laryngoscopy grey zone

Introduction

Inability to secure the airways can result in serious complications.^1–3 Difficult laryngoscopy, which has been defined as no visualization of any part of the vocal cords with conventional laryngoscope,⁴ is thought to be associated with difficult intubation,⁵ though the condition of ‘difficult tracheal intubation’ can also occur per se.^6–9 Studies have attempted to delineate the most appropriate clinical airway assessment for predicting difficult laryngoscopy but the outcomes are controversial,^9–12 and wide ranges in discriminative power for all airway examinations have been reported.^10–14 In addition, several meta-analyses have suggested that even a combination of airway examinations has limited ability to predict difficult laryngoscopy because of low sensitivity and low positive predictive value.^11,15,16

To find a more accurate predictive test, some studies have sought to identify a clear definition for difficult laryngoscopy^4,17 and identify risk factors by multivariate analysis from a derivation cohort while applying the same test to a validation cohort.^6,18 However, the usefulness of airway evaluations for predicting difficult laryngoscopy is unclear because of the fundamental problem in identifying a binary cut-off value.¹⁹ It is our opinion that in a clinical situation, the division of patients into those who may, and may not have, a difficult laryngoscopy without an overlap is difficult. Therefore, an inconclusive grey zone has been proposed to simulate the reality of clinical practice; the values in the grey zone correspond to predictions that are not precise enough for binary diagnostic decisions.²⁰ Instead of one cut-off value, the grey zone method proposes two cut-offs that constitute its borders.²¹ The first cut-off allows exclusion of diagnosis with near certainty (i.e. privilege sensitivity and negative predictive value) whereas the second cut-off includes the diagnosis with near certainty (i.e. privilege specificity and positive predictive value).²¹ Intermediate values included in the grey zone correspond to a prediction not precise enough for a diagnostic decision.

The aim of this prospective, observational study was to evaluate clinical airway assessments and scoring systems as predictors of difficult laryngoscopy using receiver operating characteristic (ROC) curve analysis with a grey zone approach. The findings of this study may clarify inconsistencies in the prediction of difficult laryngoscopy reported in previous studies.

Patients and methods

Study design

This prospective observational study was conducted at a tertiary care university hospital (Department of Anaesthesiology, Samsung Medical Centre, Seoul, Korea) between July 2012 and July 2013. Airway assessment examinations were performed preoperatively by an experienced, board certified anaesthesiologist (G.K.). Laryngeal images were obtained during endotracheal intubation using a fibreoptic scope (AV® scope; CAREtec, Kangwon-Do, South Korea) attached to a laryngoscope blade after adjusting the axis to align with the eyes (Figures 1A−C). English-type Macintosh laryngoscope blades, size 3, were used to expose the laryngeal view in the ‘sniffing position’ by using a hard, 7 cm pillow to elevate the head.²² The study was approved by the Institutional Review Board at Samsung Medical Centre, Seoul, Korea (SMC 2012-05-079-002) and registered at ClinicalTrials.gov (NCT01719848). All patients provided written informed consent.

Patient selection

Male and female patients aged 20–80 years undergoing general anaesthesia who required tracheal intubation were assessed for eligibility. Exclusion criteria included a history of difficult intubation, congenital disorders related to difficult laryngoscopy, upper airway disease and cervical spine disease with neurological symptoms.

The following predictive airway assessments were performed on each patient:^4,6

Modified Mallampati classification (MMT): performed with the patient in the sitting position with the head in extension, mouth fully opened, tongue out, and without phonation: Grade I: Faucial pillars, soft palate and uvula visible; Grade II: Faucial pillars, soft palate visible, but uvula masked by the base of the tongue; Grade III: Soft palate only visible; Grade IV: Soft palate not visible.

Upper Lip Bite Test (ULBT): Class I: Lower incisors biting the upper lip, making the mucosa of the upper lip totally invisible; Class II: The same biting manoeuvre revealing a partially visible mucosa; Class III: The lower incisors fail to bite the upper lip.

Sternomental distance (SMD): Distance measured in the seated position with the head fully extended on the neck and with the mouth closed (straight distance between the upper border of the manubrium-sterni and bony point of the mentum).

Thyromental distance (TMD): Distance from the thyroid cartilage to the mental prominence with the neck fully extended.

Mouth opening: interincisor or intergingival distance depending on the presence of teeth.

Neck circumference: measured at the level of the cricoid cartilage.

Neck mobility: estimated as the angle between the maxillary tooth occlusal surface and a horizontal line with the patient’s head fully extended.²³

The ratios of neck circumference-to-thyromental distance (NC/TMD)¹² and height-to-thyromental distance (HT/TMD)²⁴ were also calculated. Laryngeal views were graded by a blinded anaesthesiologist (E.K.) from saved images using percentage of glottic opening (POGO) score (i.e. 0 [glottis is not visible] to 100% [entire glottis is visible]).^17,25 A POGO score of zero was categorized as difficult laryngoscopy.

Three multivariate clinical models (Naguib,²⁶ LEMON method²⁷ and MACOCHA score ¹⁸) were also evaluated in the prediction of unanticipated difficult intubation. The Naguib score is regarded as the best score in operative settings:²⁶ the prediction (l) is determined by the following formula: l = 0.2262 – 0.4621 × TMD + 2.5516 × MMT score – 1.1461 × interincisor distance + 0.0433 × height.

Where the TMD distance, interincisor gap, and height were measured in cm and MMT score was graded as 0 or 1. A positive numerical value for l signifies difficult laryngoscopy and a negative value indicates easy laryngoscopy. The LEMON criteria are commonly used in the emergency department.²⁷ The MACOCHA score was created for patients admitted to intensive care units.¹⁸ Patients with difficult laryngoscopy are more likely to have higher LEMON and MACOCHA scores than those patients with a good laryngoscopic view.^18,27

Statistical analyses

All data were analysed using SPSS software (version 18 for Windows®; IBM Corp., Armonk, NY, USA) and MedCalc 12.7.2 (MedCalc Software bvba, Ostend, Belgium). Comparisons of continuous variables between difficult and easy laryngoscopy groups were conducted with independent t-test or Mann-Whitney U-test according to data normality. The MMT classification and ULBT were analysed using the χ²-test for trend.²⁸

The ability of each clinical airway test to predict difficult laryngoscopy was evaluated by ROC curve analysis.²¹ Sensitivity, specificity, positive predictive value, and negative predictive value were calculated for statistically significant variables on ROC curves. In addition, a grey zone was calculated to encompass values from each clinical airway test for which formal conclusions could not be obtained. The grey zone was defined as previously described.²¹ The optimal threshold was estimated using Youden's index (i.e. Sensitivity + Specificity − 1) to maximize both sensitivity and specificity. The mean and 95% confidence intervals (CIs) of Youden’s index were estimated using 1000 bootstrap replications. Therefore, the grey zone was defined as well as its 95% CI.²¹ In a second analytical approach, three classes of response were defined as negative, inconclusive and positive. Inconclusive grey zone was defined as the range of values with sensitivity < 90% and specificity < 90% (diagnostic tolerance of 10%). A P-value < 0.05 was considered to indicate statistical significance.

Sample size calculation was based on a pilot study of 35 patients that showed that the incidence of difficult laryngoscopy was 14.3% (n = 5) and that the area under the ROC curve for MMT classification was 0.66. To demonstrate that MMT classification could predict difficult laryngoscopy with an area under the ROC curve (AUC) of > 0.66, an alpha risk of 0.05 and a beta risk of 0.2, it was estimated that at least 180 patients with easy laryngoscopy and 30 patients with difficult laryngoscopy would be required for the main study.

Results

This study enrolled 263 patients (Figure 2). Of these, 20 were excluded for the following reasons: withdrew informed consent (n = 4); newly developed cervical spine disorder (n = 4); initially received alternative device (n = 2); laryngeal views not saved as video files due to mechanical error (n = 10). Therefore, 243 patients were included in this observational study.

Thirty-five patients (14.4%) had a POGO score of zero and so were categorized as having difficult laryngoscopy and the remaining 208 patients were categorized as having easy laryngoscopy. Patient characteristics according to easy or difficult laryngoscopy are presented in Table 1. Although there was no difference in body mass index (BMI) between the two groups, patients with difficult laryngoscopy were statistically significantly older (P = 0.009) and there were more men than women (P = 0.001) compared with the easy laryngoscopy group. Patients with difficult laryngoscopy had smaller mouth openings (P = 0.009) and higher NC/TMD (P = 0.028) and HT/TMD ratios (P = 0.011). In addition, the distribution of ULBT was statistically significantly different between the two groups (P = 0.001).

Table 1.

Clinical characteristics of the 243 patients enrolled in this prospective study assessing the diagnostic validity of clinical airway assessments for predicting difficult laryngoscopy.

Characteristic	Easy laryngoscopy group n = 208	Difficult laryngoscopy group n = 35	Statistical significance^a
Age, years	47.3 ± 13.5	53.8 ± 13.3	P = 0.009
Sex, male/female	70/138	23/12	P = 0.001
BMI, kg/m²	23.7 ± 3.2	23.8 ± 3.1	NS
MMT, I/II/III/IV	81/71/47/8 ^b	8/14/10/3	NS
ULBT, I/II/III	73/125/10	6/21/8	P = 0.001
SMD, cm	18.5 (17.0, 20.0)	18.0 (17.0, 19.0)	NS
TMD, cm	8.0 (7.0, 8.5)	8.0 (7.3, 8.5)	NS
NC, cm	36.6 (34.0, 39.2)	38.5 (35.0, 40.5)	NS
Neck mobility, °	45 (37, 60)	45 (30, 55)	NS
Mouth opening, cm	4.5 (4.1, 5.0)	4.2 (4.0, 4.6)	P = 0.009
NC/TMD	4.6 (4.3, 5.1)	5.0 (4.5, 5.4)	P = 0.028
HT/TMD	20.1 (19.1, 22.3)	21.3 (20.0, 22.9)	P = 0.011

Data presented as mean ± SD, n of patients or median (interquartile range).

Between-group comparison; continuous variables were compared using independent t-test or Mann–Whitney U-test according to data normality. The MMT classification and ULBT were analysed using the χ²-test for trend.

Data available from 207 patients.

MMT, Modified Mallampati classification: Grade 1: Faucial pillars, soft palate and uvula visible; Grade II: Faucial pillars, soft palate visible, but uvula masked by the base of the tongue; Grade III: Soft palate only visible; Grade IV: Soft palate not visible.

ULBT, Upper Lip Bite Test: Class I: Lower incisors biting the upper lip, making the mucosa of the upper lip totally invisible; Class II: The same biting manoeuvre revealing a partially visible mucosa; Class III: The lower incisors fail to bite the upper lip.

BMI, body mass index; SMD, sternomental distance; TMD, thyromental distance; NC, neck circumference; NC/TMD, neck circumference-to-thyromental distance; HT/TMD, height-to-thyromental distance; NS, no significant between-group difference (P ≥ 0.05).

Receiver operating characteristic curve analysis of airway assessments and established airway scores was performed for predicting difficult laryngoscopy (Table 2). Statistically significant predictors of difficult laryngoscopy were identified on ROC curve analysis and verified using the grey zone approach. Although seven predictive airway assessments (i.e. MMT classification, ULBT, mouth opening, HT/TMD, NC/TMD, and LEMON and Naguib models) predicted difficult laryngoscopy by ROC analyses, a grey zone approach showed that the parameters were inconclusive in approximately 70% of patients. For example, the grey zone included 174 patients (72%) between grade I or II MMT classification, 225 (93%) between grade I and II on ULBT, 171 (74%) between 4.1 and 5.5 for NC/TMD, 159 (65%) between 3.7 and 4.9 cm for mouth opening, 178 (73%) between 18.4 and 23.5 for HT/TMD, 169 (70%) between 0 and 1 for LEMON score and 168 patients (70%) between 0.93 and 6.95 for Naguib score (Table 2).

Table 2.

Receiver operating characteristic (ROC) curve analysis of airway assessments for predicting difficult laryngoscopy using a grey zone approach.

Parameter (n)	ROC Cut-off value	ROC AUC ± SE	95% CI	Statistical significance^a	Lower, upper limits of grey zone	Patients in the grey zone n (%)	Patients with difficult laryngoscopy in the grey zone n (%)
BMI (242)	> 21.9 kg/m²	0.519 ± 0.050	0.454, 0.583	NS	16.4, 28.0 kg/m²	219 (90%)	33 (94%)
MMT (242)	≥ II	0.596 ± 0.049	0.531, 0.659	P = 0.0480	I, II	174 (72%)	22 (63%)
ULBT (243)	≥ III	0.644 ± 0.046	0.580, 0.704	P = 0.0016	I, II	225 (93%)	27 (77%)
SMD (243)	≤ 19.5 cm	0.566 ± 0.049	0.501, 0.630	NS	15.0, 20.5 cm	204 (84%)	32 (91%)
TMD (243)	≤ 37.6 cm	0.542 ± 0.052	0.477, 0.606	NS	6.2, 8.9 cm	191 (79%)	30 (86%)
NC (230)^b	> 38.4 cm	0.590 ± 0.056	0.524, 0.655	NS	30.7, 41.7 cm	203 (88%)	27 (82%)
Neck mobility (243)	≤ 30°	0.592 ± 0.053	0.527, 0.654	NS	15, 59°	185 (76%)	29 (83%)
Mouth opening (243)	≤ 4.3 cm	0.637 ± 0.051	0.573, 0.697	P = 0.0076	3.7, 4.9 cm	159 (65%)	25 (71%)
NC/TMD (230)^b	> 5.1	0.603 ± 0.050	0.536, 0.666	P = 0.0396	4.1, 5.5	171 (74%)	27 (82%)
HT/TMD (243)	> 19.9	0.636 ± 0.045	0.572, 0.696	P = 0.0027	18.4, 23.5	178 (73%)	29 (83%)
LEMON (243)	≥ 2	0.648 ± 0.052	0.585, 0.708	P = 0.0047	0, 1	169 (70%)	16 (46%)
Naguib (241)	> 4.55	0.678 ± 0.048	0.615, 0.736	P = 0.0002	0.93, 6.95	168 (70%)	25 (71%)
MACOCHA (243)	> 0	0.593 ± 0.049	0.528, 0.655	NS	0, 3	175 (72%)	22 (63%)

ROC curve analysis.

Data available from 33 patients with difficult laryngoscopy.

AUC, area under the ROC curve; SE, standard error; CI, confidence interval; BMI, body mass index; MMT, Modified Mallampati classification; ULBT, upper lip bite test; SMD, sternomental distance; TMD, thyromental distance; NC, neck circumference; SMD, sternomental distance; NC/TMD, neck circumference-to-thyromental distance; HT/TMD, height-to-thyromental distance; LEMON, Naguib and MACOCHA are scoring systems^18,25,26; NS, no significant difference (P ≥ 0.05).

The diagnostic validity profiles from the significantly predictive airway assessments are shown in Table 3. Low positive predictive values were observed for all airway tests. The ratio of HT/TMD showed the highest sensitivity (80.0%) and ULBT had the highest specificity (95.2%).

Table 3.

Diagnostic validity profiles of airway assessments for predicting difficult laryngoscopy (n = 243).

Airway assessment with cut-off value	TP (n)	TN (n)	FP (n)	FN (n)	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
MMT ≥ II	27	81	127	8	77.1 (59.9, 89.6)	39.1 (32.4, 46.1)	17.6 (12.0, 24.6)	91.0 (83.0, 96.1)
ULBT ≥ III	8	198	10	27	22.9 (10.4, 40.1)	95.2 (91.3, 97.7)	44.4 (21.5, 69.2)	88.0 (83.0, 91.9)
Mouth opening ≤ 4.3 cm	22	122	86	13	62.9 (44.9, 78.5)	58.7 (51.6, 65.4)	20.4 (13.2, 29.2)	90.4 (84.1, 94.8)
NC/TMD > 5.1^a	14	152	45	19	42.4 (25.5, 60.8)	77.2 (70.7, 82.8)	23.7 (13.6, 36.6)	88.9 (83.2, 93.2)
HT/TMD > 19.9	28	96	112	7	80.0 (63.1, 91.6	46.15 (39.2, 53.2)	20.0 (13.7, 27.6)	93.2 (86.5, 97.2)
LEMON ≥ 2	19	153	55	16	54.3 (36.6, 71.2)	73.6 (67.0, 79.4)	25.7 (16.2, 37.2)	90.5 (85.1, 94.5)
Naguib > 4.55^b	21	149	57	14	60.0 (42.1, 76.1)	72.3 (65.7, 78.3)	26.9 (17.5, 38.2)	91.4 (86.0, 95.2)

Data presented as values (95% confidence interval) for sensitivity, specificity, PPV, and NPV.

Data available from 230 patients.

Data available from 241 patients.

TP, true positive; TN, true negative; FP, false positive; FN, false negative; PPV, positive predictive value; NPV, negative predictive value; MMT, Modified Mallampati classification; ULBT, upper lip bite test; NC/TMD, neck circumference-to-thyromental distance; HT/TMD, height-to-thyromental distance; LEMON and Naguib are scoring systems.^25,26

Discussion

In this prospective observational study, seven airway examinations and/or scoring systems (i.e. MMT classification, ULBT, mouth opening, NC/TMD, HT/TMD, and the LEMON and Naguib models) were found to be statistically significant predictors of difficult laryngoscopy by ROC analysis. However, on further analysis using the grey zone approach, all variables showed large inconclusive zones with a large proportion of patients in the grey zone, which may explain previous inconsistent results with regard to the prediction of difficult laryngoscopy. Out of 243 eligible patients, 35 (14.4%) were categorized as having difficult laryngoscopy. Although statistically significant differences were observed between the easy (n = 208) and difficult laryngoscopy (n = 35) groups in terms of age and sex, in our opinion these differences were not clinically relevant to the study outcome.

Difficult airways are generally encountered in unexpected clinical situations.¹ Although video laryngoscopic airway devices are increasingly used, direct laryngoscopy is most often used as the first device for intubation. The identification of patients in whom laryngoscopy and intubation may be difficult remains a challenge. Several studies in a variety of clinical fields have attempted to identify predictors of difficult laryngoscopy and intubation.^4,15,16 However, airway examinations for predicting difficult airway have been shown to have diverse reliabilities and various cut-off values have been used for the ROC curve analysis.^{8,10,11,29–31} Moreover, these cut-off values are difficult to establish^32–35 and it has been suggested that perfect prediction of truly difficult intubation is impossible because of its rarity.³⁵

Despite being widely used to evaluate diagnostic tests,¹⁹ ROC curve analysis has a fundamental problem in that it is based on a binary decision with a single cut-off value to differentiate patients with and without difficult laryngoscopy. This ‘black or white’ approach might not be suitable in actual clinical situations.³⁶ Therefore, two cut-off values separated by a grey zone have been proposed.²⁰ Recent studies suggest that defining values within a grey zone requires comprehensive evaluation using other clinical data.^37–39 In the present study, each airway assessment had a wide grey zone, which suggests that these predictive tests are inconclusive for predicting difficult laryngoscopy for approximately 70% of patients. Possible reasons for the wide grey zones include variation in operator expertise in airway assessment, laryngoscopy and/or intubation.^32,35,40 Many factors contribute to the problem of reliably predicting difficult laryngoscopy including differences in airway anatomy and morphology (e.g. large tongue, limited neck movement, limited mouth opening, small TMD distance), complex interactions among the various airway parameters used to determine airway difficulty,⁴¹ and the limited value of some tests for accurately assessing specific parts of the airway.³⁵

In addition to the mathematical issues in establishing predictability of difficult airways, airway assessments can be difficult to perform in an emergency situation.⁴ Indeed, most airway assessment examinations require the cooperation of conscious patients.^6,7 While the American Society of Anesthesiologists Task Force on Management of Difficult Airways generally recommend that a MMT score should be obtained from a cooperative patient in the sitting position with adequate tongue protrusion, only 32% of patients who require endotracheal intubation in the emergency department are able to follow commands.⁷ Furthermore, most patients who required endotracheal intubation in intensive care units are supine or recumbent and uncooperative.¹⁸ Unexpected difficult airways have been reported in patients with undiagnosed lingual tonsillitis, lingual thyroid, laryngeal papillomatosis and laryngeal web even after a thorough physical airway examination.^42–46 Although video laryngoscopic airway devices have been shown to improve laryngeal views, successful endotracheal intubations and successful first attempt intubations, the current findings suggest that the accuracy of airway assessments in emergency situations may be questionable.

This present study had several limitations. First, a POGO score of 0 was used to grade difficult laryngoscopy instead of ‘difficult intubation’. Difficult laryngoscopy has occasionally been considered ‘difficult tracheal intubation’,^6–9 but it is only one of the many reasons for difficult intubation. In addition, in some patients with a high POGO score (i.e. good laryngeal view), a guiding gum elastic bougie or multiple attempts were used in the process of endotracheal intubation. In these patients, it is suggested that the curve of the endotracheal tube may not have fitted with the alignment of the oral, pharyngeal, and laryngeal axes. Secondly, the present study used a single type of laryngoscope blade (Macintosh size 3) in all patients. Therefore, because the upper part of the glottis can potentially be covered by the curved Macintosh blade, the POGO scores might have been underestimated. This underestimation might have led to the higher incidence of difficult laryngoscopy in this present study compared with previous reports.¹⁶. Thirdly, all airway evaluations in the present study showed low positive predictive values because of high false positive rates. Generally, an airway assessment with a high sensitivity and a low false positive rate will predict difficult laryngoscopy with precision.⁴⁷ However, the current results suggest that a single type of airway assessment is not sufficient for determining patients who may have difficult laryngoscopy. Considering that positive and negative predictive values are affected by the prevalence of the condition (i.e. difficult laryngoscopy) in the population,^7,48 it is not possible to exclude the possibility that the relatively high incidence of difficult laryngoscopy in this present study (14.4%) influenced the results. Finally, information about the patients in the grey zone with potentially difficult laryngoscopy was not provided. Since difficult laryngoscopy could not be predicted by a single pre-existing airway test or score in approximately 70% of patients, further studies are required to validate these current results. Nevertheless, the current results are useful in that they reaffirm that there are problems in predicting difficult laryngoscopy.

In conclusion, this prospective observational study found that several airway examinations were statistically significant predictors of difficult laryngoscopy by ROC analysis, but using the grey zone approach all variables showed large inconclusive zones that may explain previous inconsistent results in the prediction of difficult laryngoscopy.

Footnotes

Acknowledgements

Drs Jeong Jin Min and Gahyun Kim contributed equally as co-first authors. Dr Gahyun Kim has moved from Samsung Medical Centre to Bundang Jesaeng Hospital, Gyeonggi-do, Korea.

Declaration of conflicting interests

The authors declare that there are no conflicts of interest.

Funding

This study was supported solely by departmental resources. The authors have no commercial, proprietary, or financial interest in the products or companies described in this article.

References

Cook

MacDougall-Davis

. Complications and failure of airway management. Br J Anaesth 2012; 109(Suppl 1): i68–i85.

Martin

Mhyre

Shanks

. 3,423 emergency tracheal intubations at a university hospital: airway outcomes and complications. Anesthesiology 2011; 114: 42–48.

Mort

. Emergency tracheal intubation: complications associated with repeated laryngoscopic attempts. Anesth Analg 2004; 99: 607–613.

Apfelbaum

Hagberg

Caplan

. Practice guidelines for management of the difficult airway: an updated report by the American society of Anesthesiologists task force on management of the difficult airway. Anesthesiology 2013; 118: 251–270.

Benumof

. Difficult laryngoscopy: obtaining the best view. Can J Anaesth 1994; 41(5 Pt 1): 361–365. [in French, English Abstract].

el-Ganzouri

McCarthy

Tuman

. Preoperative airway assessment: predictive value of a multivariate risk index. Anesth Analg 1996; 82: 1197–1204.

Levitan

Everett

Ochroch

. Limitations of difficult airway prediction in patients intubated in the emergency department. Ann Emerg Med 2004; 44: 307–313.

Rosenstock

Gillesberg

Gätke

. Inter-observer agreement of tests used for prediction of difficult laryngoscopy/tracheal intubation. Acta Anaesthesiol Scand 2005; 49: 1057–1062.

Connor

Segal

. Accurate classification of difficult intubation by computerized facial analysis. Anesth Analg 2011; 112: 84–93.

10.

Eberhart

Arndt

Cierpka

. The reliability and validity of the upper lip bite test compared with the Mallampati classification to predict difficult laryngoscopy: an external prospective evaluation. Anesth Analg 2005; 101: 284–289.

11.

Khan

Mohammadi

Rasouli

. The diagnostic value of the upper lip bite test combined with sternomental distance, thyromental distance, and interincisor distance for prediction of easy laryngoscopy and intubation: a prospective study. Anesth Analg 2009; 109: 822–824.

12.

Kim

Ahn

Lee

. Neck circumference to thyromental distance ratio: a new predictor of difficult intubation in obese patients. Br J Anaesth 2011; 106: 743–748.

13.

Ambesh

Singh

Rao

. A combination of the modified Mallampati score, thyromental distance, anatomical abnormality, and cervical mobility (M-TAC) predicts difficult laryngoscopy better than Mallampati classification. Acta Anaesthesiol Taiwan 2013; 51: 58–62.

14.

Hirmanpour

Safavi

Honarmand

. The predictive value of the ratio of neck circumference to thyromental distance in comparison with four predictive tests for difficult laryngoscopy in obstetric patients scheduled for caesarean delivery. Adv Biomed Res 2014; 3: 200–200.

15.

Cattano

Panicucci

Paolicchi

. Risk factors assessment of the difficult airway: an italian survey of 1956 patients. Anesth Analg 2004; 99: 1774–1779.

16.

Shiga

Wajima

Inoue

. Predicting difficult intubation in apparently normal patients: a meta-analysis of bedside screening test performance. Anesthesiology 2005; 103: 429–437.

17.

Levitan

Ochroch

Kush

. Assessment of airway visualization: validation of the percentage of glottic opening (POGO) scale. Acad Emerg Med 1998; 5: 919–923.

18.

De Jong

Molinari

Terzi

. Early identification of patients at risk for difficult intubation in the intensive care unit: development and validation of the MACOCHA score in a multicenter cohort study. Am J Respir Crit Care Med 2013; 187: 832–839.

19.

Sundaram

. Predicting difficult intubation; useful or what? A response to ‘Predicting difficult intubation–worthwhile exercise or pointless ritual?’, Anaesthesia 2002; 57: 105–109. Anaesthesia 2003; 58: 619–619.

20.

Ray

Le Manach

Riou

. Statistical evaluation of a biomarker. Anesthesiology 2010; 112: 1023–1040.

21.

Cannesson

Le Manach

Hofer

. Assessing the diagnostic accuracy of pulse pressure variations for the prediction of fluid responsiveness: a “gray zone” approach. Anesthesiology 2011; 115: 231–241.

22.

Benumof JL. Comparison of intubating positions: the end point for position should be measured. Anesthesiology 2002; 97: 750; author reply 754.

23.

Henderson J. Airway management in the adult. In: Miller RD (ed) Miller’s anesthesia. 7th ed. Philadelphia: Churchill Livingstone Elsevier, 2009, pp.1573–1610.

24.

Safavi

Honarmand

Zare

. A comparison of the ratio of patient’s height to thyromental distance with the modified Mallampati and the upper lip bite test in predicting difficult laryngoscopy. Saudi J Anaesth 2011; 5: 258–263.

25.

Ochroch

Hollander

Kush

. Assessment of laryngeal view: percentage of glottic opening score vs Cormack and Lehane grading. Can J Anaesth 1999; 46: 987–990.

26.

Naguib

Scamman

O’Sullivan

. Predictive performance of three multivariate difficult tracheal intubation models: a double-blind, case-controlled study. Anesth Analg 2006; 102: 818–824.

27.

Reed

Dunn

McKeown

. Can an airway assessment score predict difficulty at intubation in the emergency department? Emerg Med J 2005; 22: 99–102.

28.

Armitage

. Tests for linear trends in proportions and frequencies. Biometrics 1955; 11: 375–386.

29.

Ayoub

Baraka

el-Khatib

. A new cut-off point of thyromental distance for prediction of difficult airway. Middle East J Anaesthesiol 2000; 15: 619–633.

30.

Baker

Depuydt

Thompson

. Thyromental distance measurement–fingers don’t rule. Anaesthesia 2009; 64: 878–882.

31.

Bair

Caravelli

Tyler

. Feasibility of the preoperative Mallampati airway assessment in emergency department patients. J Emerg Med 2010; 38: 677–680.

32.

Cheater

Jameson

. The value of airway assessment. Anaesthesia 2002; 57: 615–615.

33.

Calder I. Useless ritual? Anaesthesia 2002; 57: 612.

34.

Kristensen

. Predicting difficult intubation 2. Anaesthesia 2002; 57. 612; discussion 612-3.

35.

Yentis

. Predicting difficult intubation–worthwhile exercise or pointless ritual? Anaesthesia 2002; 57: 105–109.

36.

Cannesson

. Arterial pressure variation and goal-directed fluid therapy. J Cardiothorac Vasc Anesth 2010; 24: 487–497.

37.

Coste

Pouchot

. A grey zone for quantitative diagnostic and screening tests. Int J Epidemiol 2003; 32: 304–313.

38.

Petraco

Sen

Nijjer

. Fractional flow reserve-guided revascularization: practical implications of a diagnostic gray zone and measurement variability on clinical decisions. JACC Cardiovasc Interv 2013; 6: 222–225.

39.

Duwat

Zogheib

Guinot

. The gray zone of the qualitative assessment of respiratory changes in inferior vena cava diameter in ICU patients. Crit Care 2014; 18: R14–R14.

40.

Vannucci

Cavallone

. Bedside predictors of difficult intubation: a systematic review. Minerva Anestesiol 2016; 82: 69–83.

41.

Greenland

. A proposed model for direct laryngoscopy and tracheal intubation. Anaesthesia 2008; 63: 156–161.

42.

Basaranoglu

Erden

Kokten

. Laryngeal web as a result of Reinke’s oedema: a cause of difficult endotracheal intubation. Br J Anaesth 2006; 96: 406–407.

43.

Basaranoglu

Columb

Lyons

. Failure to predict difficult tracheal intubation for emergency caesarean section. Eur J Anaesthesiol 2010; 27: 947–949.

44.

Lau

Fang

. Unexpected upper airway obstruction due to disseminated human papilloma virus infection involving the pharynx in a parturient. Acta Anaesthesiol Taiwan 2010; 48: 87–90.

45.

Patel

Davidian

Reebye

. Complicated airway due to unexpected lingual tonsil hypertrophy. Anesth Prog 2012; 59: 82–84.

46.

Pisanu

Deplano

Uccheddu

. Unexpected and asymptomatic lingual thyroid as a cause of difficult intubation. J Clin Anesth 2012; 24: 429–430.

47.

Connor

Segal

. The importance of subjective facial appearance on the ability of anesthesiologists to predict difficult intubation. Anesth Analg 2014; 118: 419–427.

48.

Lalkhen

McCluskey

. Clinical tests: sensitivity and specificity. Contin Educ Anaesth Crit Care Pain 2008; 8: 221–223.