Response to Achieving Consensus in the Measurement of Psychological Adjustment to Cleft Lip and/or Palate at Age 8+ Years

Dear Editor,

We read with interest this issue’s article by (Stock et al., 2020), which reviewed patient-reported measures for psychological adjustment to cleft lip and/or palate in people aged 8 years or older. The authors aimed to appraise available measures, before recommending a core set, presumably for use in both research and clinical practice. We commend the authors for their focus on this issue, which is as important as it is challenging. However, we have some reservations about this article’s methodology and reporting.

In 2017, this journal published an article from an International Consortium for Health Outcomes Measurement (ICHOM) working group that aimed to achieve consensus on measurement across many domains of cleft-related health (Allori et al., 2017). Stock et al may have good reason to repeat part of this process, but this was not explained in their article. A critical discussion of the limitations of ICHOM’s core outcome set would have been interesting and could have helped to contextualize this work.

The authors included measures that had at least “acceptable” levels of validity and reliability. However, what constitutes “acceptable” was not stated. While the authors did report whether studies of a measure’s validity existed, they made no attempt to quantify, qualify, or compare the level of evidence that supports each measure. The authors include the About You (CEN-Q) measure in their recommended set, although according to their paper, there is no evidence to support its validity or reliability. We would like to remind readers that validity is not a binary concept. It is more helpful to qualify the evidence that implies an instrument’s validity than to describe a measure as valid or invalid.

Similarly, normative data are population-specific and details about the normative samples referred to in this paper would help readers to understand why the authors considered them translatable. Ideally, a core measurement set would be of use internationally.

Assessing the quality of measurement instruments is a complex process. However, the consensus-based standards for the selection of health status measurement instruments (COSMIN) does provide comprehensive and robust guidance on conducting systematic reviews of measurement instruments (Mokkink et al., 2018). We would be interested to know why the authors chose not to follow this widely accepted guideline for the methodology and reporting of such studies.

Subscales of the CLEFT-Q were deemed too long to include in the author’s core set, and we found this decision confusing. The CLEFT-Q is a modular measure. It is made up of 13 questionnaires which range from 6 to 12 items in length (Klassen et al., 2018). Each questionnaire functions independently, so the assessor can choose which questionnaires are relevant to the person they are assessing. The development of the CLEFT-Q computerized adaptive test is likely to reduce this response burden even further (Harrison et al., 2019).

Moving forward, we think that consensus in this subject can only be achieved through collaboration. We are concerned that researchers in this field are working independently and in parallel, rather than synergistically. Stock et al have gone to considerable effort to involve patients in their work, and this is highly admirable. But consensus will not be achieved unless a critical mass of stakeholders are involved, and this includes other health care providers and researchers from the international cleft community.