Proxy- and Patient-Reported Outcome Measures in Cleft Lip and/or Palate Under 8 Years of Age: A Scoping Review

Abstract

Objective

This scoping review aimed to identify patient-reported outcome (PRO) and proxy-report outcome (ProxRO) measures administered to children under 8 years of age with cleft lip and/or palate (CL/P) and to describe the age at which proxy or self-report measures were used.

Design

Scoping Review

Setting

When children are unable to self-report, PROs and ProxROs can be used. The age at which proxy-report is appropriate has not been described in patients with CL/P.

Patients, Participants

Children born with CL/P.

Interventions

Ovid MEDLINE, EMBASE, PsycINFO, the Cochrane Register, and Web of Science were searched from inception until July 2024. Title and abstract screening, full text review, and data extraction were done independently in duplicate. Measures were categorized as PROs, ProxROs, or “MultiROs,” where both caregivers and children responded. A narrative synthesis was performed to describe the identified measures.

Main Outcome Measures

PRO or ProxRO measures in children with CL/P less than 8 years old.

Results

Of 7001 publications identified, 57 studies met the inclusion criteria. Fifty-six measures were identified, of which 43 had at least 1 published psychometric property. Thirteen studies used ad hoc measures. Most measures were not condition-specific to CL/P. ProxROs were more commonly utilized than PROs. Parent-proxy tools were used from birth to 8 years, while self-report was used in patients as young as 3 years old.

Conclusions

There is a paucity of cleft-specific ProxRO measures that assess outcomes of cleft care in patients under 8 years of age. Work to develop and validate ProxRO measures for CL/P is needed.

Keywords

outcomes pediatrics quality of life systematic review cleft lip cleft palate

Introduction

Cleft lip and/or palate (CL/P) and its sequelae have been associated with various psychosocial issues, including disruptions in psychosocial functioning, decreased quality of life, self-esteem issues, and an increased risk of psychological and emotional issues.¹

It is therefore important that healthcare providers (HCPs) and researchers assess the outcomes of treatment while identifying and responding to the unmet needs and preferences of people with CL/P and their families. One of the key methods to capture this information is through patient-reported outcome (PRO) and Caregiver Proxy-Reported Outcome (ProxRO) measures. These measures assess patient and caregiver perceptions of symptoms, functional status, and health-related quality of life (HRQoL) in a scientifically sound manner.^2,3 The primary difference is that PRO measures are self-reported and directly ask the patient about their own health and well-being, while ProxRO measures ask another person (usually a caregiver) to report on the patient's health and well-being as if they were the patient. ProxRO measures are often used when patients are unable to self-report due to cognitive impairment or young age.⁴ Importantly, the proxy-report is distinct from the caregiver or observer-report in that the proxy-report asks the respondent to answer based on what they believe the patient thinks, whereas the observer-report asks what the observer thinks.

PRO/ProxRO measures provide HCPs and researchers with a better understanding of patient experience and satisfaction, thereby allowing them to improve the quality and effectiveness of care. Incorporation of these measures has led to benefits in the clinical management of patients, evaluation of care, and research, as well as improvements in communication and shared clinical decision-making.² These outcome measures may be generic or condition-specific.⁵ The former is focused on health concepts that can be applied across many diseases or conditions, while the latter captures health outcomes unique to a particular disease or condition.⁶ Given the unique physiological and psychosocial impacts of CL/P, condition-specific PRO/ProxRO measures are necessary to capture the outcomes that matter most to people with CL/P, as well as to measure the impact of cleft-specific treatment.^5,7

Systematic reviews of the literature to identify PROs previously used in children with CL/P have found few condition-specific PRO measures, which highlighted the need for validated PRO measures.^8,9 In response to this gap, condition-specific PRO measures were developed,^10,11 including the CLEFT-Q,¹² which is designed for children and young adults with CL/P aged 8 to 29 years. The CLEFT-Q consists of 12 scales measuring satisfaction with appearance, facial function, and HRQoL. The CLEFT-Q's psychometric properties have been established, proving it to be reliable and valid in several studies to date.^12–15 As the CLEFT-Q is only validated as a PRO measure in people with CL/P over 8 years of age, it cannot be used in younger children less than 8 years of age as is.¹² The ability to measure concepts important to patients and families is therefore limited in this age group, thereby impeding the ability of HCPs to evaluate treatment outcomes. Furthermore, the reliability of self-report has been shown to be lower for children below 8 years of age,^16,17 particularly, with respect to recall past 48 h and the limited ability of children in this age group to respond beyond dichotomous response options.¹⁸

Children with CL/P less than 8 years of age reach significant cognitive, language, emotional, and social development milestones¹⁹—all of which can be impacted by CL/P and its comorbidities. While they may be unable to self-report, children with CL/P in this age group have been reported to experience neurodevelopmental and academic deficits²⁰ as well as difficulties with psychosocial adjustment.^1,21 With respect to speech alone, by the age of 3 years, children with communication disorders have been reported to experience social isolation.²²

There have not been many studies focusing on the perspectives of children with CL/P under age 8, with the majority of studies focusing on parents’ experiences.^23,24 Given a significant amount of treatment of CL/P happens early in childhood, it is important to try to include the patient perspective as early as possible. Moreover, children under the age of 8 years have been described to have lower global self-concept and experience significant stigma.^25,26 This would suggest that they are aware of the impact of having CL/P; thus, if they are undergoing treatment aimed at improving their HR-QOL, it is important to capture their perspective in the measurement of outcomes.

This study is a first step towards the multiphase adaptation of the CLEFT-Q into a caregiver ProxRO measure suitable for use in children with CL/P from birth to under 8 years of age. Development of a PRO involves (1) a literature review, (2) qualitative interviewing of the population of interest, and (3) stakeholder consultation.^27,28 This literature review identifies available measurement tools and highlights gaps in measurement in order to assess the need for a new or revised measure for young children with CL/P.

The primary objective of this scoping review is to identify all PRO and ProxRO measures that have been used in children with CL/P under the age of 8 years. The secondary objective of this scoping review is to determine the age ranges for which PRO and ProxRO measures have been used.

Methods

A preliminary search of MEDLINE, the Cochrane Database of Systematic Reviews, and JBI Evidence Synthesis was conducted, and no systematic reviews or scoping reviews on the topic were identified. This review is also a partial update to a 2012 systematic review, which identified PRO measures that assess quality of life in the CL/P literature.⁸ This scoping review was conducted in accordance with the Joanna Briggs Institute (JBI) methodology for scoping reviews²⁹ and followed the Preferred Items for Systematic Reviews and Meta-analysis—Scoping Review (PRISMA-ScR)³⁰ reporting guidelines.

Eligibility Criteria

Studies were included if a PRO or ProxRO measure was shown to have been administered to children with CL/P less than 8 years of age or their primary caregivers. All forms of CL/P were included. The study had to indicate that at least a proportion of respondents were either the child themselves or their caregiver (eg, parent, legal guardian). If the sample did not include children under the age of 8 years, or age ranges were not stated, the study was excluded. No limits on context or study design were defined to capture a diversity of perspectives and experiences on the topic of interest.

Opinion papers and gray literature were not considered for inclusion as they do not provide complete information on the use of PRO or ProxRO measures. Systematic and scoping reviews were also excluded from this review. The reference lists of all excluded systematic and scoping reviews were examined, however, for eligible studies.

Search Strategy

The search strategy was created to locate all published studies relating to our objectives with the assistance of a research librarian. The search strategy, including all identified keywords and index terms, was adapted for each included database and/or information source (Supplementary Table 1). Studies published in English were included. No date limits were applied. The initial search was completed on November 29, 2022. Databases included: Ovid MEDLINE, EMBASE, PsycINFO, the Cochrane Register and Web of Science. On July 17, 2024, the search was updated to ensure this scoping review would be current.

Study/Source of Evidence Selection

Studies were uploaded and de-duplicated in Covidence, a web-based platform for systematic/scoping reviews and evidence synthesis.³¹ Three independent reviewers (A.D., J.H., and K.J.) screened the titles and abstracts against the review's inclusion/exclusion criteria. Two reviewers (A.D., J.H.) completed full-text screening at the following stage. Reasons for exclusion during full-text review were recorded. Any disagreement between the reviewers was resolved through discussion or with an additional reviewer (K.J.).

Data Extraction

One reviewer (K.J.) extracted data from the included studies and iteratively updated a data-charting form through discussion with the study team. The data-charting form included the following study characteristics: study design, inclusion/exclusion criteria, sample characteristics (eg, age, types of CL/P), PRO or ProxRO measures used, and the constructs measured. The data-charting form can be found in Supplementary Table 2. Two independent reviewers (A.D. and J.H.) cross-checked and queried the extracted data. Any disagreements were resolved through discussion.

Synthesis of Results

The measurement tools were grouped into 3 categories: (1) PRO, (2) ProxRO, and (3) MultiRO measures. If a measure was a self-report measure, it was categorized as a PRO measure. If a caregiver answered a measure as if they were their child, the measure was categorized as a ProxRO measure. Tools where parents respond as third-party observers of their child (rather than as if they were their child) are normally considered ObsRO measures. ObsRO measures ask caregivers to observe their child for outcomes of clinical importance.^4,32 As not all studies provided copies of their questionnaires used, nor provided clear distinction between ProxRO and ObsRO measures, this study categorized all measures completed by parents under the category of ProxRO measures. Lastly, measures where caregivers and patients completed the same measure together or separately regardless of whether psychometric properties supported such use were categorized as “MultiRO” measures.

There are numerous measures (eg, PedsQL 4.0) that contain PRO and ProxRO versions of the same measure, or feature modules that assess separate constructs. As many of these measures will have differences in wording, domains, and items, they were categorized as different tools. For example, the PedsQL 4.0 Child Self-Report (Ages 5-7), the PedsQL 4.0 Parent Proxy-Report (Ages 5-7), and the PedsQL 4.0—Family Impact Module were counted as separate measures.^33,34 PROs and ProxROs should have established psychometric properties in order to ensure scientifically sound measurement. As such, we recorded whether each measure had any published psychometric properties.⁵ Measures were categorized as having some supporting validation if there was evidence of at least 1 psychometric property; otherwise, the measure was categorized as having no supporting validation. In addition, the age range of participants completing each measure was identified.

Results

After de-duplication, 8515 studies were identified through the search strategy and the citation search of excluded systematic reviews (Figure 1). Title and abstract screening resulted in the removal of 8344 studies, leaving 167 studies for full-text assessment of eligibility. During full-text review, 110 studies were excluded. The remaining 57 studies met the inclusion criteria and were included in this scoping review.

Figure 1.

PRISMA flow diagram of included and excluded studies.

The characteristics of each included study are described in Table 1, including country of origin, sample size, percentage of sample size living with CL/P, type of CL/P (ie, cleft lip and palate, cleft palate-only, cleft lip-only, cleft lip and alveolus), gender, age range of the sample, and study design.

Table 1.

Characteristics of Included Studies.

Study ID	Country	Sample size	% with cleft	Type of cleft (%)	Gender (%)	Sample age range	Study design/type
Aarabi 2018 ³⁵	Germany	396	8%	Unknown	M (47) F (53)	7 to 17 years	Cross-sectional
Ahmed 2024 ³⁶	Pakistan	80	100%	CLP (60); CP (23); CL (18)	M (55) F (45)	6 to 19 years	Cross-sectional study
Al-Hassani 2024 ³⁷	United Kingdom	659	100%	CLP (100)	M (68) F (32)	5 to 7 years	Cross-sectional study
Bhat 2019 ³⁸	India	180	33%	CLP (100)	Unknown	0 to 30 years	Cross-sectional
Bickham 2017 ³⁹	United States	168	100%	CLP (70); CP (30)	M (50) F (50)	5 to 19 years	Cross-sectional
Brand 2009 ⁴⁰	Switzerland	66	48%	CLP (100)	M (70) F (30)	6 to 16 years	Case control
Branson 2024 ⁴¹	Australia	63	100%	CLP (46); CP (38); CL (10); CLA (6)	M (54) F (46)	4 to 17 years	Cohort study
Brichacek 2021 ⁴²	Canada	37	100%	CL (100)	Unknown	0 to 6 years	Cross-sectional
Broder 1989 ²⁵	United States	58	69%	CLP (33); CP (35); CL (33)	M (50) F (50)	7 years	Cross-sectional
Broder 2014 ⁴³	United States	1200	100%	CLP (76); CP (24)	M (56) F (44)	7 to 19 years	Cohort
Broder 2017 ⁴⁴	United States	1196	100%	CLP (76); CP (24); CL (0)	M (56) F (44)	7.5 to 18.5 years	Cohort
Bruneel 2019 ⁴⁵	Belgium	30	83%	CLP (72); CP (28)	Unknown	3 to 10 years	Cross-sectional
Collett 2012 ⁴⁶	United States	217	43%	CLP (36); CP (38); CL (26)	M (48) F (52)	5 to 9 years	Case control
Cook 2016 ⁴⁷	United States	79	100%	CLP (90); CP (8); CL (3)	M (65) F (35)	4 to 10.9 years	Retrospective Chart Review
Costa 2021 ⁴⁸	UK	322	100%	CLP (41); CP (38); CL (21)	M (42) F (58)	18 months	Cross-sectional
Damiano 2007 ⁴⁹	United States	104	100%	CLP (48); CP (24); CL (28)	M (43) F (57)	2 to 12 years	Cross-sectional
DeJonge 2024 ⁵⁰	United States	102	37%	CLP (68); CP (18); CL (8); CLA (5)	Unknown	1 to 16 years	Cross-sectional study
Edwards 2022 ¹¹	United States	133	100%	CLP (74); CL (26)	M (66) F (34)	1.35 to 11.47 mo.	Development/Validation
Fadeybi 2012 ⁵¹	Nigeria	116	100%	CLP (24); CP (30); CL (46)	M (49) F (51)	0 to 64 years	Cross-sectional
Filgueira 2015 ⁵²	Brazil	31	100%	CLP (39); CP (29); CL (32)	M (71) F (29)	2 to 5 years	Cross-sectional
Habibagahi 2024 ⁵³	Iran	20	100%	Unknown	M (60) F(40)	42 mo. ± 4 mo.	Cohort study
Heike 2020 ⁵⁴	United States	492	100%	CLP (37); CL (11)	Unknown	0 to 2 years	Development/Validation
Jokovic 2003 ⁵⁵	Canada	231	NR	NR	M (45) F (55)	6 to 14 years	Development/Validation
Jones 2022 ⁵⁶	United States	25	100%	CLP (72); CL (28)	M (64) F (36)	"Infant"	Cohort
Kapp-Simon 2024 ⁵⁷	United States	123	100%	CLP (72); CLA (28)	M (34) F (66)	1.10 to 14.30 mo.	Cohort study
Khoun 2018 ⁵⁸	Cambodia	133	32%	Unknown	M (47) F (53)	2 to 8 years	Cross-sectional
Kramer 2008 ⁵⁹	Germany	147	100%	CLP (33); CP (31); CL (37)	M (59) F (41)	5 to 6.9 years	Cross-sectional
KrueckeBerg 1993 ⁶⁰	United States	52	39%	CLP (25); CP (40); CL (35)	Unknown	3 to 6.75 years	Nonrandomized experimental study
Leopoldo-Rodado 2021 ⁶¹	Spain	357	48%	CLP (37); CP (30); CL (33)	M (51) F (49)	4 to 7.92 years	Case control
Lu 2022 ⁶²	China	72	100%	CLP (100)	M (43) F (57)	4 to 20 years	Cross-sectional
Morgan 2020 ⁶³	United States	75	59%	CLP (93); CL (7)	M (49) F (51)	7 to 21 years	Cross-sectional
Murray 2010 ⁶⁴	UK	170	55%	CLP (67); CL (33)	M (62) F (38)	7 years	Cohort
Naros 2018 ⁶⁵	Germany	134	100%	CLP (49); CP (32); CL (19)	M (52) F (48)	4 to 18 years	Cross-sectional
Nguyen 2019 ⁶⁶	Vietnam, Estonia	56	100%	Unknown	Unknown	7 to 28 years	Cross-sectional survey
Nicholls 2018 ⁶⁷	Australia	359	100%	CLP (47); CP (39); CL (14)	M (51) F (49)	6 to 42 years	Cross-sectional
Nicholls 2019 ⁶⁸	Australia	359	100%	CLP (47); CP (39); CL (14)	M (51) F (49)	6 to 42 years	Cross-sectional
Pasini 2022 ⁶⁹	Italy	64	50%	CLA (100)	M (66) F (34)	6 to 15 years	Cross-sectional
Patjanasoontorn 2012 ⁷⁰	Thailand	36	100%	CLP (100)	M (50) F (50)	5 years	Cross-sectional
Prathanee 2024 ⁷¹	Thailand	15	100%	CLP (93); CL (7)	M (60) F(40)	3.83 to 10.75 years	Cohort study
Rahman 2024 ⁷²	United States	131	100%	CLP (73); CL (27)	M (66) F (34)	"Infant"	Cohort study
Ramos 2022 ⁷³	Brazil	106	100%	CLP (77); CP (12); CL (10)	M (59) F (41)	0 to 24 mo.	Nonrandomized experimental trial
Rando 2018 ⁷⁴	Brazil	121	62%	CLP (100)	Unknown	2 to 6 years	Cross-sectional
Ranganathan 2016 ⁷⁵	United States	93	100%	CLP (82); CL (18)	M (54) F (46)	5 to 20 years	Cross-sectional
Reissmann 2017 ⁷⁶	Germany	140	16%	Unknown	M (51) F (49)	7 to 17 years	Cross-sectional
Rosenberg 2022 ⁷⁷	United States	140	100%	CLP (73); CL (27)	M (4) F (96)	1.12 to 11.67 mo.	Cohort
Ruff 2017 ⁷⁸	United States	327	25%	CLP (56); CP (26); CL (9)	M (46) F (54)	2 to 5 years	Cross-sectional
Sagheri 2009 ⁷⁹	Germany	61	97%	CLP (47); CP (42); CL (10)	M (52) F (48)	4 to 7 years	Cross-sectional
Sander 2022 ⁸⁰	Germany	152	100%	Unknown	M (59) F (41)	1 to 47 years	Development/Validation
Schonmeyr 2015 ⁸¹	India	134	100%	CLP (68); CP (32)	M (53) F (47)	7 to 35 years	Cross-sectional
Seifert 2021 ⁸²	UK	412	100%	CLP (38); CP (36); CL (25)	M (64) F (36)	3 years	Cohort
Skirko 2015 ⁸³	United States	57	56%	CLP (66); CP (34)	M (44) F (56)	3 to 22 years	Cohort
Soedjana 2022 ⁸⁴	Indonesia	104	100%	CLP (74); CP (21); CLA (5)	M (57) F (43)	6 to 15 years	Cross-sectional
Thompson 2021 ⁸⁵	New Zealand	378	100%	CLP (34); CP (43); CL (22)	M (46) F (54)	5 to 12 years	Cohort
Thompson 2021 ⁸⁶	New Zealand	397	100%	CLP (35); CP (43); CL (22)	M (55) F (45)	5 to 12 years	Cohort
van Veen 2022 ⁸⁷	Netherlands	58	100%	CLP (97); CP (3)	M (74) F (26)	0 to 8 years	Cross-sectional
Warschausky 2002 ⁸⁸	United States	54	50%	CLP (74); CP (11); CL (7); CLA (7)	Unknown	5 to 18 years	Case control
Zeraatkar 2018 ⁸⁹	Iran	110	50%	CLP (100)	Unknown	2 to 5 years	Cross-sectional

Studies included participants from 22 different countries, with the United States of America^{11,25,39,43,44,46,47,49,50,54,56,57,60,63,72,75,77,78,83,88} (n = 20), Germany^{35,59,65,76,79,80} (n = 6), Brazil^52,73,74 (n = 3), and the United Kingdom^37,48,64,82 (n = 4) represented most frequently. Sample sizes ranged from 15 to 1200 participants, and the age of participants with CL/P ranged from newborns up to 64 years.

Characteristics of Measures

Fifty-six unique measures were identified. Of these, 13 were ad hoc measures that did not report any psychometric properties. The remaining 43 measures that reported at least 1 psychometric property are described in Table 2. Figure 2 displays the age ranges for which ProxRO, PRO, and MultiRO measures were given to children with CL/P and their families below the age of 8 years.

Figure 2.

Age ranges where measures were administered as proxy-report (ProxRO), patient-report (PRO), or multi-report (MultiRO) for children with CL/P between birth and 9 years of age.

Table 2.

Measures Used as Proxy-Report (ProxRO), Patient-Report (PRO), or Multi-Report (MultiRO) Outcomes.

		No. studies used as:
Name of questionnaire	Constructs assessed	ProxRO	PRO	MultiRO
Generic condition measures
Pediatric Quality of Life Inventory (PedsQL 4.0)—Proxy Report ^{46,47,49,53,85,86}	Health-related quality of life	6
Early Childhood Oral Health Impact Scale (ECOHIS) ^{50,52,73,74,89}	Impact of oral health on child and family quality of life	5
KINDL-p—Proxy Report ^59,61,65,79	Health-related quality of life	4
Child Behaviour Checklist (CBCL) ^46,64,84	Emotional and behavioral adjustment	3
Child Oral Health Impact Profile (COHIP)—Proxy Report ^44,76,78	Oral health-related quality of life	3
Velopharyngeal Insufficiency (VPI) Effects on Life Outcomes (VELO)—Proxy Report ^45,62,83	Speech-related health-related quality of life	3
Ages and Stages Questionnaire-3 (ASQ-3) ^48,57	Child development	2
Parental-Caregivers Perception Questionnaire (P-CPQ) ^55,58	Impact of oral health on child and family quality of life	2
Pediatric Quality of Life Inventory (PedsQL 4.0)—Family Impact Module ^85,86	Family functioning	2
Strengths and Difficulties Questionnaire (SDQ)—Proxy Report ^40,41	Behavior, peer relations, emotional symptoms	2
General Health Questionnaire (GHQ) ^51,70	Mental well-being	1		1
Ages and Stages Questionnaire (ASQ-SE) ⁴⁸	Child socio-emotional development	1
Child Health Questionnaire version (CHQ-PF28) ⁸⁸	Health-related quality of life	1
Children with Special Health Care Needs (CSHCN) Screener ⁴⁹	Special healthcare needs	1
Condition Quality of Life Scale (CFC-QoL)—Proxy Report ⁶³	Health-related quality of life	1
Family Impact Scale (FIS-8) ⁵⁸	Impact of oral health on child and family quality of life	1
Hospital Anxiety and Depression Scale (HADS) ⁴⁸	Anxiety and depression	1
Intelligibility in Context Scale (ICS) ⁸²	Speech intelligibility	1
Parenting Stress Inventory (PSI) ⁴⁶	Parental functioning	1
Participant in Everyday Life Communication Questionnaire (PIELCQ)—Proxy Report ⁴⁰	Communicative competencies	1
Social Competence Scale (SCS) ⁴⁶	Social competence	1
Social Skills Questionnaire ⁶⁰	Social skills	1
WHOQOL-BRIEF-THAI ⁷¹	Health-related quality of life	1
Child Oral Health Impact Profile (COHIP)—Self-Report ^{35,43,44,61,69,76}	Oral health-related quality of life		6
Pediatric Quality of Life Inventory (PedsQL 4.0)—Self-Report ^{46,47,75,85,86}	Health-related quality of life		5
KINDL—Self-Report ^59,61,65	Health-related quality of life		3
Pictorial Scale of Perceived Competence and Social Acceptance for Young Children ^60,64	Self-concept/self-esteem		1
Body-Esteem Scale for Children (BESC) ^67,68	Self-concept/self-esteem		1	1
Condition Quality of Life Scale (CFC-QoL)—Self-Report ⁶³	Health-related quality of life		1
Primary Self-Concept Inventory (PSCI) ²⁵	Self-concept/self-esteem		1
PROMIS—Anxiety ⁷⁵	Psychosocial functioning		1
PROMIS—Depression ⁷⁵	Depression symptoms		1
PROMIS—Peer Relationships ⁷⁵	Social functioning		1
PROMIS ³⁹	Health-related quality of life			1
Self-rating Depression Scale (SDS) ⁵¹	Depression			1
State-Trait Anxiety Inventory (STAI-Y1) ⁵¹	Immediate anxiety			1
State-Trait Anxiety Inventory (STAI-Y2) ⁵¹	General anxiety			1
CLP-specific measures
Infant with Clefts Observation Outcomes Instrument (iCOO) ^{11,54,56,72,77}	Infant/child cleft-related health and well-being	5
CleftChild-8 (CC-8) ⁸⁷	Quality of life, patient satisfaction	1
THAICLEFT Quality of Life Questionnaire ⁷⁰	Quality of life	1
CLEFT-Q ³⁶	Health-related quality of life (2 subscales)		1
Cleft Hearing, Appearance and Speech Questionnaire (CHASQ) ⁶⁶	Satisfaction with hearing, appearance, and speech			1
Cleft-Screen Questionnaire (CSQ) ⁸⁰	Health-related quality of life			1
		52	23	8

Of the 43 measures identified in this scoping review, the majority (n = 37) were generic rather than CL/P-specific measures. There were 6 measurement tools created specifically for children with CL/P : (1) Cleft Hearing, Appearance and Speech Questionnaire (CHASQ),⁶⁶ (2) CleftChild-8 (CC-8),⁸⁷ (3) Cleft-Screen Questionnaire (CSQ),⁸⁰ (4) Infant with Clefts Observation Outcomes Instrument (iCOO),^11,54,56,77 (5) THAICLEFT Quality of Life Questionnaire,⁷⁰ and the (6) CLEFT-Q.³⁶ One measure (the Velopharyngeal Insufficiency Effects on Life Outcomes or VELO) was specific to velopharyngeal insufficiency, which can be related to CL/P but is not necessarily associated.⁹⁰ Of all the cleft-specific measures, only the THAICLEFT study was published more than 5 years preceding this scoping review.

ProxRO Measures

Twenty-six measures were ProxRO measures. The most often used ProxRO measure was the Pediatric Quality of Life Inventory (PedsQL 4.0) parent-proxy measure in 6 studies,^{46,47,49,53,85,86} followed by the Early Childhood Oral Health Impact Scale (ECOHIS),^{50,52,73,74,89} and Infant with Clefts Observation Outcomes Instrument (iCOO)^{11,54,56,72,77}—each used in 5 studies. Most of these measures (12/26 measures) were used to assess quality of life (QoL) or health-related quality of life (HRQoL).

ProxRO measure usage ranged from birth to 8 years of age. Three of the aforementioned 6 cleft-specific measures were used as ProxRO measures—the CleftChild-8 (CC-8),⁸⁷ (newborn to 8 years old), Infants with Clefts Observation Outcomes Instrument (iCOO)^11,54,56,77 (newborn to 3 years old), and the THAICLEFT Quality of Life Questionnaire⁷⁰ (5 years of age only). The average lower age limit for ProxROs was 3.2 years.

PRO Measures

Eleven measures were PRO measures. Among these tools, the Child Oral Health Impact Profile (COHIP) was administered most (6 studies),^{35,43,44,61,69,76} followed by the PedsQL 4.0 child self-report measure (5 studies),^{46,47,75,85,86} the Kiddy/Kid/Kiddo KINDL (3 studies).^59,61,65 Again, QoL/HRQoL was the most assessed construct (5/11 measures), though self-concept/self-esteem (3/11 measures) was often investigated. PRO measures were used in patients as young as 3 years old through age 8 years. This review identified only 1 cleft-specific measure used as a PRO for children under the age of 8 years—the CLEFT-Q.³⁶ The average lower age limit for PRO measures (5.2 years) was higher than that of ProxRO measures.

MultiRO Measures

Eight of the included tools were used at least once as a MultiRO. Two measures (the General Health Questionnaire and the Body-Esteem Scale for Children) were used in 2 studies respectively as PRO measures and again in 2 separate studies as MultiRO measures. Mental health (3/8 measures), QoL/HRQoL (2/8 measures), and self-concept/self-esteem (2/8 measures) were the 2 concepts most assessed by MultiROs.

Measure Summary

Of particular interest to this scoping review are the measures that were at least partially validated for use in children with CL/P. The ages of the patients in which these measures were used is described.

CleftChild-8 (CC-8)

The CleftChild-8 (CC-8) ProxRO measure assesses domains of CL/P-specific health-related quality of life and patient satisfaction in children.⁹¹ The 43-item questionnaire assesses 4 domains on 4 subscales: (1) parental satisfaction with the functioning of the cleft team, (2) satisfaction with surgery, (3) social functioning, and (4) psychological functioning. This review identified 1 paper where the CleftChild-8 was administered as a ProxRO measure in children ranging from newborn to 8 years of age.⁸⁷

Cleft Hearing, Appearance, and Speech Questionnaire (CHASQ)

The Cleft Hearing, Appearance, and Speech Questionnaire (CHASQ) is a 15-item PRO measure that assesses patients’ satisfaction with features of appearance, speech, and hearing using 2 domains.⁹² The first domain captures patient perspectives on cleft-associated features (eg, whole appearance, lips, hearing), while the second domain is comprised of items that assess other facial features (eg, hair, eyes, cheeks).⁹² The CHASQ was administered in 1 identified paper to patients aged 7 to 28 years and/or their primary caregiver.⁶⁶

Infants With Cleft Observation Outcomes Instrument (iCOO)

The Infants with Cleft Observation Outcomes Instrument (iCOO) is an ObsRO measure,¹¹ rather than a ProxRO measure, meaning that it is designed to assess the observer's opinion of the outcome of interest. Consisting of 65 items, the iCOO assesses 9 domains: (1) behavioral and emotional health, (2) breathing, (3) comfort, (4) communication, (5) ears and hearing, (6) facial skin and mouth, (7) feeding, (8) sleep, and (9) vocalization. This review identified 5 papers where the iCOO was deployed as a ProxRO measure for infants from newborn to 3 years of age.^{11,54,56,72,77}

Cleft Screen Questionnaire (CSQ)

The Cleft Screen Questionnaire (CSQ) is a 38-item screening measure that was created to be a succint, yet comprehensive cleft-specific screening measure for HRQoL in individuals with CL/P. Consisting of 38 items, the CSQ's items group into 5 different domains of which have not yet been interpreted.⁸⁰ One paper identified in this review administered the CSQ to patients between 1 and 47 years of age as a PRO or ProxRO dependent on patient age.⁸⁰

Thai Cleft Quality of Life Questionnaire (THAICLEFT QOL)

The Thai Cleft Quality of Life Questionnaire (THAICLEFT QOL) is intended to be a ProxRO measure that captures the quality of life of children with CL/P living in a Thai sociocultural context. The number of scale items in the THAICLEFT questionnaire is unclear, with 2 studies stating that the total number of items is 20⁷⁰ or 24.⁹³ The THAICLEFT QoL was developed by extracting and modifying items from a number of generic measures (eg, KINDL, WHOQOL-BREF, and Impact on Family Scale) and notes from the 2009 Thai Cleft Lip Palate and Craniofacial Association.⁹³ The THAICLEFT measure was administered as a ProxRO in 1 paper identified in this review to primary caregivers of children 5 years of age.

CLEFT-Q

The CLEFT-Q is a condition-specific PRO measure for use in children with CL/P between 8 to 29 years of age.^12,15,94,95 This review identified 1 paper that reported using 2 CLEFT-Q subscales—Psychological Function and Social Function—among children ages 6 to 19 years.³⁶

Discussion

In this scoping review, we identified 57 primary studies where PRO and ProxRO measures were utilized among children under 8 years of age living with CL/P. We identified a total of 43 different tools that had undergone some level of psychometric testing and validation. The findings altogether suggest a widespread interest in capturing outcomes beyond conventional biomarkers and clinician-reported outcomes (ClinROs) measures in studies of CL/P.

Utilizing the HCP perspective and expertise to assess a patient's health status (eg, rating of speech intelligibility, pre- and postsurgery anthropometrics) through ClinRO measurements is a cornerstone of medicine.³² ClinRO measures, however, cannot assess patient and family perspectives on the physical and psychosocial health of a child during CL/P treatment.⁹⁵ This review highlights PRO and ProxRO measures as valuable tools to capture patient and family perspectives, as these perspectives are often undervalued in favor of ObsRO and ClinRO measures.^96–98 PRO and ProxRO measures directly assess how patients and families feel and function by eliciting patient or caregiver perspectives. In contrast, ClinRO measures and biomarkers assess the same outcomes but through the perspective and interpretation of HCPs who see their patients episodically rather than throughout their daily lives.⁹⁹ PRO and ProxRO measures also capture what is important to patients and families while avoiding observer bias, while ClinRO measures often assess outcomes that are important to HCPs.⁹⁹ Consequently, PRO and ProxRO measures can help HCPs assess patient needs, improve care quality, track progress and outcomes, and adapt interventions accordingly. These outcome measures also foster communication and shared decision-making among patients, families, and HCPs.²

Though there is a widespread interest in non-ClinRO measures with respect to CL/P, the majority of the measures included in this review were not condition-specific. As a result, many of the biophysiological, clinical, and psychosocial sequelae related to CL/P may not be captured in a clinically meaningful, scientifically sound manner. Many generic measures do not include content that is relevant to patients with CL/P, and if these measures are used to assess outcomes, changes in concepts that are not included will not be assessed.

The closest generic measures to capturing CL/P related outcomes are the ones that queried HRQoL—an outcome that is often impacted heavily by a CL/P diagnosis. The most pertinent measures used as ProxRO measures would be the Child Oral Health Impact Profile (COHIP), the Early Childhood Oral Health Impact Scale (ECOHIS), the Parental-Caregivers Perception Questionnaire (P-CPQ), and the Family Impact Scale (FIS-8). These measures can be used to capture the impact of a child's oral condition on the child, their family, or both. However, they are still limited in that their focus is primarily on the impacts of gum and tooth health and less on the impacts of CL/P on the domains of speech, eating, drinking, biophysiological development, appearance, and psychosocial growth.

We also found that there were relatively few measures that were used as PRO measures. Of the 11 PRO measures, the majority limited self-report to children aged 5 years and up. Varni et al.¹⁶ have reported that the ideal age for self-report is 8 years and older.

One limitation among the 57 identified studies was that measures were administered to age groups for which they were not validated for use. For example, a 14-item Spanish-translated version of the COHIP (COHIP-14) was deployed as a PRO measure in patients aged 4 to 7 years of age,⁶¹ although the COHIP-14 was validated in children age 12 to 21 years of age.¹⁰⁰ The CLEFT-Q was also administered as a PRO measure in patients aged 6 to 19 years of age,³⁶ when the CLEFT-Q has been validated in individuals 8 to 29 years of age.¹²

Another inherent issue is with MultiRO measures—self-report PRO measures that were administered to caregivers without modification, and where caregivers were asked to either complete on their own behalf,⁵¹ on their child's behalf,⁸⁰ or together with their child.^67,68 In these situations, cross-informant variance between self-report and proxy-report and bias threaten measure validity and interpretability.^17,80 To avoid this, patients and caregivers should be given separate but similar validated measures that assess the same constructs with parallel items.¹⁷

Below, we briefly summarize each of the 6 identified measures below in relation to the conceptual framework of Wong et al.⁹⁵ Developed through interviews with individuals living with CL/P, this conceptual framework reflects the issues that individuals with CL/P and their families (rather than HCPs) determined to be most important to measure.⁹⁵ These issues regard CL/P outcomes and are divided into 3 domains: (1) appearance outcomes (ie, face, lips, nose, nostrils, cleft scar, teeth, jaws), (2) health-related quality of life (ie, speech-related distress, psychological function, social function, school function), and (3) facial function (ie, speech function, eating, and drinking).⁹⁵

CleftChild-8 (CC-8)

Formerly known as the K(N)UZS-8,⁹¹ strengths of the CleftChild-8 (CC-8) include the engagement of parents and support groups during its development, as well as suitable internal consistency, test-retest reliability, and construct validity.^87,91 The domains of the CC-8, however, focus more on process outcomes. Half of the domains and subdomains are focused on satisfaction with the cleft team or cleft care, while key outcomes identified by Wong et al.⁹⁵ such as satisfaction with appearance, speech-related distress, or facial function, are not included. Moreover, the CC-8 has only been used once in a publication since its development in 2016,⁸⁷ and has not yet been translated from Dutch into other languages.

Cleft Hearing, Appearance, and Speech Questionnaire (CHASQ)

A full validation study on the CHASQ has yet to be published,¹⁰¹ though psychometric testing of successive CHASQ translations suggests that it shows satisfactory internal validity, construct validity, and overall psychometric properties in patients over the age of 7 years.^{10,66,92,101,102} There is, however, a lack of consistency in how the CHASQ is administered or interpreted—some studies have used a single-item version of the CHASQ while other studies have used the full 15 items.¹⁰³ There is also uncertainty in whether the CHASQ is solely a PRO measure as described by Stiernman et al.⁹² whether there is a ProxRO version, or whether caregivers are given the PRO measure and asked to answer on behalf of their child.⁶⁶ For this reason, we categorized the CHASQ as a MultiRO measure.

The “satisfaction with cleft-related features” domain aligns with some aspects of the “Appearance” domain that patients with CL/P identified as important to measure.⁹⁵ The CHASQ does not assess health-related quality of life, emotional-well-being, or other impacts of CL/P on their lives. In a comparative study between the CLEFT-Q and the CL/P, respondents found it easier and faster to complete the 1-scale CHASQ compared to the 12-scale CLEFT-Q, but still preferred the depth and detail of the CLEFT-Q scales and thought they would provide better detailed and broader information to HCPs.¹⁰⁴ Despite the relative ease and speed of completing the CHASQ, additional work needs to be done to clarify its format (ie, PRO vs ProxRO) as well as its psychometric properties as a ProxRO for children with CL/P under 8 years.

Infants With Cleft Observation Outcomes Instrument (iCOO)

The psychometric properties of the Infants with Cleft Observation outcomes Instrument (iCOO) have been established for use in newborn children with CL/P up until 3 years of age and have shown good internal consistency, construct validity, structural validity, and responsiveness.¹¹ The goal of the iCOO is to reliably record caregiver observations of CL/P-related clinical signs so that HCPs can better assess child health.⁵⁴ The iCOO was developed using robust methodology with caregiver input and feedback.⁵⁴ It is a useful tool for the age group for which it is designed, when patients themselves are unlikely to be able to self-report. The caregiver perspective is important to measure and understand in tandem with the patient perspective. Above the age of 3 years, patients may start to understand some of the concepts of interest to health care providers. For the measurement of this patient perspective at a young age (for example, being able to be understood at school or pre-school), a proxy-report measure would be ideal.

Cleft Screen Questionnaire (CSQ)

Some of the CSQ's psychometric properties have been published, showing good internal consistency in a sample of people with CL/P from 1 year to 47 years old and their parents.⁸⁰ Borumand et al.¹⁰⁵ highlight that the CSQ's development process is incomplete, as only a principal components analysis (PCA) was performed when an exploratory factor analysis should have been undertaken. Moreover, the CSQ's developers opted not to completely establish the measure's psychometric properties, as it was presumed that the CSQ's items (which had been adapted from 14 already-existing questionnaires) were already valid.^80,105 The CSQ is designed to be a self-report PRO measure, but parents were asked to respond on their child's behalf when they were too young. Of note, “young” was not defined as a specific age group in this case. As with the other MultiROs identified in this scoping review, cross-informant variance is an issue with ramifications for bias, validity, and interpretability. Without complete reporting of the measure's psychometric properties, however, it is difficult to know if the CSQ is valid as either a PRO or ProxRO measure.

Thai Cleft Quality of Life Questionnaire (THAICLEFT QOL)

The THAICLEFT QOL—similar to the CSQ—has had some psychometric testing, showing good internal consistency in a sample of parents of children with CL/P.^70,93 There was no information published on the ages of the children in the sample. The THAICLEFT QOL did not involve patients or families in the development process. The majority of questions contained in the THAICLEFT QOL focus on the quality of life, mental health, and needs of the parents as opposed to the child,⁷⁰ making this neither a PRO nor a ProxRO measure, but a more general measure of caregiver and family health. Altogether, the lack of reporting of both the measure's development and its intended use, as well as its focus on caregiver outcomes, makes this measure less useful as a ProxRO measure in children with CL/P under 8 years.

CLEFT-Q

Though its psychometric properties have been established repeatedly in individuals 8 to 29 years of age, the CLEFT-Q is not recommended for use below 8 years of age as younger children are unable to reliably self-report HRQoL.^16,17 The usage of the CLEFT-Q below its validated age range further suggests that there is a need for a validated version of the CLEFT-Q for children under 8 years of age.

Age of PRO and ProxRO Usage

Our review found that PROs were administered to children with CL/P as young as 3 years of age, despite research suggesting that the results of PROs are unreliable at this age.¹⁶ Conversely, a ProxRO version of a measure was administered alongside a PRO version in patients as old as 22 years.⁸³ This implies a lack of clarity as to what age a child should be completing a PRO as opposed to their primary caregiver completing a ProxRO. Guidelines are needed regarding why and when it is appropriate to use a ProxRO or a PRO, and at what age that children should be approached for self-report.^4,106 ProxROs should only be used when a child is otherwise unable to complete a PRO¹⁷—using a ProxRO when a child is able to self-report has implications for the reliability and validity of longitudinal research.

Limitations

Our team acknowledges the limitations of our scoping review. Our review was limited by the paucity of condition-specific PRO and ProxRO measures available for our target population. In addition, heterogeneity in the reporting of measure development and the establishment of psychometric properties was a limitation in interpreting the results of our review. Finally, as scoping reviews are exploratory and intended to capture the broader scope of the literature, critical appraisal of the study methodology and quality were not part of our review.

Conclusions

This scoping review identified 6 cleft-specific measures that could potentially be used to capture outcomes of children with CL/P under the age of 8 years. Of these 6 tools, the CleftChild-8 (CC-8) is the only ProxRO measure that covers the full age range of interest. Its focus on process-related outcomes, however, means that the CC-8 may not assess the outcomes that matter most to individuals with CL/P and their families. As a result, there are no ProxRO measures that capture concerns that matter most to children with CL/P below the age of 8 years.

To fill this gap, either a new tool must be developed or a preexisting tool must be adapted. If adaptation of an existing tool is the way forward, the process should follow guidelines similar to the development of the initial tool itself. This may include in-depth qualitative interviews with parents and HCPs, adaptation of items for proxy-report, and large-scale field-testing. The final product should be a tool that HCPs and researchers can administer to caregivers of children 3 to 8 years of age with CL/P and capture health-related quality of life, satisfaction with appearance, and facial function, as defined by the conceptual framework of Wong et al.⁹⁵ As per Varni et al.¹⁷ the adapted measure would need to mirror the existing tool, allowing for life-long, uninterrupted longitudinal assessment from birth through to adulthood.

Supplemental Material

sj-docx-1-cpc-10.1177_10556656251382058 - Supplemental material for Proxy- and Patient-Reported Outcome Measures in Cleft Lip and/or Palate Under 8 Years of Age: A Scoping Review

Supplemental material, sj-docx-1-cpc-10.1177_10556656251382058 for Proxy- and Patient-Reported Outcome Measures in Cleft Lip and/or Palate Under 8 Years of Age: A Scoping Review by Kariym Joachim, Alexandra D’Souza, Jessie Howard, Aaron M. Drucker, Christopher R. Forrest and Karen W.Y. Wong Riff in The Cleft Palate Craniofacial Journal

Supplemental Material

sj-docx-2-cpc-10.1177_10556656251382058 - Supplemental material for Proxy- and Patient-Reported Outcome Measures in Cleft Lip and/or Palate Under 8 Years of Age: A Scoping Review

Supplemental material, sj-docx-2-cpc-10.1177_10556656251382058 for Proxy- and Patient-Reported Outcome Measures in Cleft Lip and/or Palate Under 8 Years of Age: A Scoping Review by Kariym Joachim, Alexandra D’Souza, Jessie Howard, Aaron M. Drucker, Christopher R. Forrest and Karen W.Y. Wong Riff in The Cleft Palate Craniofacial Journal

Footnotes

ORCID iDs

Kariym Joachim

Christopher R. Forrest

Ethical Approval

This article does not contain any studies with human or animal participants. There are no human participants in this article and informed consent is not required.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of Conflicting Interest

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

The studies included in this review can be accessed via their respective sources as outlined in the manuscript references. Any additional datasets generated and/or analyzed within this study are available from the corresponding author upon reasonable request.

Supplemental Material

Supplemental material for this article is available online.

References

Hunt

Burden

Hepper

Johnston

. The psychosocial effects of cleft lip and palate: A systematic review. Eur J Orthod. 2005;27(3):274–285. doi:10.1093/ejo/cji004

Black

. Patient reported outcome measures could help transform healthcare. Br Med J. 2013;346(7896):f167. doi:10.1136/bmj.f167

Valderas

Kotzeva

Espallargues

, Guyatt

, Ferrans

, Halyard

, Revicki

, Symonds

, Parada

, Alonso

. The impact of measuring patient-reported outcomes in clinical practice: A systematic review of the literature. Qual Life Res. 2008;17(2):179–193. doi:10.1007/s11136-007-9295-0

Roydhouse

Cohen

Eshoj

, Corsini

, Yucel

, Rutherford

, Wac

, Berrocal

, Lanzi

, Nowinski

et al. The use of proxies and proxy-reported measures: A report of the international society for quality of life research (ISOQOL) proxy task force. Qual Life Res. 2022;31(2):317–327. doi:10.1007/s11136-021-02937-8

Wormald

JCR

Rodrigues

. Outcome measurement in plastic surgery. J Plast Reconstr Aesthet Surg. 2018;71(3):283–289. doi:10.1016/j.bjps.2017.11.015

Churruca

Pomare

Ellis

, Long

, Henderson

, Murphy

LED

, Leahy

, Braithwaite

. Patient-reported outcome measures (PROMs): A review of generic and condition-specific measures and a discussion of trends and issues. Health Expect. 2021;24(4):1015–1024. doi:10.1111/hex.13254

Ricketts

Regev

Antonyshyn

Kiss

Fialkov

. Use of the derriford appearance scale 59 to assess patient-reported outcomes in secondary cleft surgery. Plast Surg (Oakv). 2016;24(1):27–31.

Klassen

Tsangaris

Forrest

, Wong

KWY

, Pusic

, Cano

, Syed

, Dua

, Kainth

, Johnson

, et al. Quality of life of children treated for cleft lip and/or palate: A systematic review. J Plast Reconstr Aesthet Surg. 2012;65(5):547–557. doi:10.1016/j.bjps.2011.11.004

Eckstein

Akinbiyi

Silver

Taub

. Measuring quality of life in cleft lip and palate patients: Currently available patient-reported outcomes measures. Plast Reconstr Surg. 2011;128(5):518e–526e. doi:10.1097/PRS.0b013e31822b6a67

10.

Stiernman

Maulina

Zepa

, Jagomägi

, Tanaskovic

, Knežević

, Velikova

, Anastassov

, Radojićić

, Pesic

, et al. Translation and pilot study of the cleft hearing appearance and speech questionnaire (CHASQ). Eur J Plast Surg. 2019;42(6):583–592. doi:10.1007/s00238-019-01543-9

11.

Edwards

Heike

Kapp-Simon

, Jones

, Leroux

, Stueckle

, Belluci

, Rosenberg

, Albert

, Aspinall

, et al. Infant with clefts observation outcomes instrument (iCOO): A new outcome for infants and young children with orofacial clefts. Cleft Palate Craniofac J. 2022;59(10):1233–1245. doi:10.1177/10556656211040307

12.

Klassen

Riff

KWW

Longmire

, Albert

, Allen

, Aydin

, Baker

, Cano

, Chan

, Courtemanche

, et al. Psychometric findings and normative values for the CLEFT-Q based on 2434 children and young adult patients with cleft lip and/or palate from 12 countries. CMAJ. 2018;190(15):E455–E462. doi:10.1503/cmaj.170289

13.

Harrison

Sidey-Gibbons

Klassen

, Wong Riff

KWY

, Furniss

, Swan

, Rodrigues

. Deeper understanding of appearance in orofacial clefts: A structural equation model of the CLEFT-Q appearance scales. Plast Reconstr Surg Glob Open. 2021;9(9):e3806. doi:10.1097/GOX.0000000000003806

14.

Ploumen

RLM

Willemse

Jonkman

REG

Nolte

Becking

. Quality of life after orthognathic surgery in patients with cleft: An overview of available patient-reported outcome measures. Cleft Palate Craniofac J. 2023;60(4):405–412. doi:10.1177/10556656211067120

15.

Tsangaris

Wong Riff

KWY

Goodacre

, Forrest

, Dreise

, Sykes

, de Chalain

, Harman

, O'Mahony

, Pusic

, et al. Establishing content validity of the CLEFT-Q: a new patient-reported outcome instrument for cleft lip/palate. Plast Reconstr Surg Glob Open. 2017;5(4):e1305. doi:10.1097/GOX.0000000000001305

16.

Varni

Limbers

Burwinkle

. How young can children reliably and validly self-report their health-related quality of life?: An analysis of 8,591 children across age subgroups with the PedsQL 4.0 generic core scales. Health Qual Life Outcomes. 2007;5:Article 1. doi:10.1186/1477-7525-5-1

17.

Varni

Limbers

Burwinkle

. Parent proxy-report of their children’s health-related quality of life: An analysis of 13,878 parents’ reliability and validity across age subgroups using the PedsQL^TM 4.0 generic core scales. Health Qual Life Outcomes. 2007;5:Article 2. doi:10.1186/1477-7525-5-2

18.

Coombes

Bristowe

Ellis-Smith

, Aworinde

, Fraser

, Downing

, Bluebond-Langner

, Chambers

, Murtagh

FEM

, Harding

. Enhancing validity, reliability and participation in self-reported health outcome measurement for children and young people: A systematic review of recall period, response scale format, and administration modality. Qual Life Res. 2021;30(7):1803–1832. doi:10.1007/s11136-021-02814-4

19.

Dosman

Andrews

Goulden

. Evidence-based milestone ages as a framework for developmental surveillance. J Paediatr Child Health. 2012;17(10):561–568. doi:10.1093/pch/17.10.561

20.

Gallagher

Collett

. Neurodevelopmental and academic outcomes in children with orofacial clefts: A systematic review. Pediatrics. 2019;144(1):e20184027. doi:10.1542/peds.2018-4027

21.

Pinckston

Dalton

Farrar

Hotton

. The psychosocial adjustment of children born with a cleft lip and/or palate: Cross-sectional and longitudinal analyses. Cleft Palate Craniofac J. 2020;57(11):1280–1290. doi:10.1177/1055665620921669

22.

Thomas-Stonell

Oddson

Robertson

Rosenbaum

. Development of the FOCUS (focus on the outcomes of communication under six), a communication outcome measure for preschool children. Dev Med Child Neurol. 2010;52(1):47–53. doi:10.1111/j.1469-8749.2009.03410.x

23.

Sharif

Callery

Tierney

. The perspectives of children and young people living with cleft lip and palate: A review of qualitative literature. Cleft Palate Craniofac J. 2013;50(3):297–304. doi:10.1597/12-054

24.

Nelson

Glenny

Kirk

Caress

. Parents’ experiences of caring for a child with a cleft lip and/or palate: A review of the literature. Child Care Health Dev. 2012;38(1):6–20. doi:10.1111/j.1365-2214.2011.01244.x

25.

Broder

Strauss

. Self-concept of early primary school age children with visible or invisible defects. Cleft Palate J. 1989;26(2):114–117. discussion 117.

26.

Kapp-Simon

. Self-concept of primary-school-age children with cleft lip, cleft palate, or both. Cleft Palate J. 1986;23(1):24–27.

27.

Center for Drug Evaluation and Research, U.S. Food and Drug Administration. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. May 15, 2020. Accessed July 2, 2025. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-reported-outcome-measures-use-medical-product-development-support-labeling-claims

28.

Cano

Klassen

Pusic

. The science behind quality-of-life measurement: A primer for plastic surgeons. Plast Reconstr Surg. 2009;123(3):98e–106e. doi:10.1097/PRS.0b013e31819565c1

29.

Peters

Godfrey

McInerney

Munn

Tricco

Khalil

. Chapter 11: Scoping Reviews (2020 version). In: Aromataris

Munn

, eds. JBI Manual for Evidence Synthesis. JBI; 2020:407–452. Accessed March 29, 2023. https://synthesismanual.jbi.global/.

30.

Tricco

Lillie

Zarin

, O'Brien

, Colquhoun

, Levac

, Moher

, Peters

MDJ

, Horsley

, Weeks

, et al. PRISMA Extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018;169(7):467–473.doi:10.7326/M18-0850

31.

Veritas Health Innovation. Covidence systematic review software. Accessed November 29, 2022. https://www.covidence.org

32.

Powers

Patrick

Walton

, Marquis

, Cano

, Hobart

, Isaac

, Vamvakas

, Slagle

, Molsen

, et al. Clinician-Reported outcome assessments of treatment benefit: Report of the ISPOR clinical outcome assessment emerging good practices task force. Value Health. 2017;20(1):2–14. doi:10.1016/j.jval.2016.11.005

33.

Varni

Sherman

Burwinkle

Dickinson

Dixon

. The PedsQL^TM family impact module: Preliminary reliability and validity. Health Qual Life Outcomes. 2004;2:Article 55. doi:10.1186/1477-7525-2-55

34.

Varni

Seid

Kurtin

. PedsQL^TM 4.0: Reliability and validity of the pediatric quality of life Inventory^TM version 4.0 generic core scales in healthy and patient populations. Med Care. 2001;39(8):800.

35.

Aarabi

Reissmann

Sagheri

, Neuschulz

, Heydecke

, Kofahl

, Sierwald

. Oral health-related quality of life of children and adolescents with and without migration background in Germany. Qual Life Res. 2018;27(10):2619–2627. doi:10.1007/s11136-018-1903-7

36.

Ahmed

Shahid

Mumtaz

Saqulain

. Psychological and social aspects of quality of life of cleft lip and/or palate patients: Age group and gender perspective. RMJ. 2024;49(1):82–82.

37.

Al-Hassani

Fowler

Wren

Leary

Davies

. Should left- and right-sided unilateral cleft lip and palate patients be grouped together when reporting the outcomes? Orthod Craniofac Res. 2024;27(S1):49–61. doi:10.1111/ocr.12747

38.

Bhat

Shetty

Khan

Husain

Ragher

. Evaluation of patient and parent satisfaction after orthodontic treatment/orthognathic surgery for cleft lip and palate deformity. J Pharm Bioallied Sci. 2019;11(Suppl 2):S184–S187. doi:10.4103/jpbs.JPBS_289_18

39.

Bickham

Ranganathan

Wombacher

, Shapiro

, Carlozzi

, Baker

, Vercler

, Warschausky

, Buchman

, Waljee

. Speech perceptions and health-related quality of life among children with cleft lip and palate. J Craniofac Surg. 2017;28(5):1264–1268. doi:10.1097/SCS.0000000000003718

40.

Brand

Blechschmidt

Muller

, Sader

, Schwenzer-Zimmerer

, Zeilhofer

, Holsboer-Trachsler

. Psychosocial functioning and sleep patterns in children and adolescents with cleft lip and palate (CLP) compared with healthy controls. Cleft Palate Craniofac J. 2009;46(2):124–135. doi:10.1597/07-165.1

41.

Branson

Collins

McGrath

Anderson

Kilpatrick

Crowe

. The impact of COVID-19 on the well-being of victorian children born with cleft lip and/or palate. Orthod Craniofac Res. 2024;27(1):110–117. doi:10.1111/ocr.12699

42.

Brichacek

Matic

. Parental perceptions following cleft lip repair in their children. J Craniofac Surg. 2021;32(3):E321–E324. doi:10.1097/SCS.0000000000003713

43.

Broder

Wilson-Genderson

Sischo

Norman

. Examining factors associated with oral health-related quality of life for youth with cleft. J Plast Reconstr Surg. 2014;133(6):828E–834E. doi:10.1097/PRS.0000000000000221

44.

Broder

Wilson-Genderson

Sischo

. Oral health-related quality of life in youth receiving cleft-related surgery: Self-report and proxy ratings. Qual Life Res. 2017;26(4):859–867. doi:10.1007/s11136-016-1420-5

45.

Bruneel

Bettens

Van Lierde

. The relationship between health-related quality of life and speech in patients with cleft palate. Int J Pediatr Otorhinolaryngol. 2019;120:112–117. doi:10.1016/j.ijporl.2019.02.018

46.

Collett

Cloonan

Speltz

Anderka

Werler

. Psychosocial functioning in children with and without orofacial clefts and their parents. Cleft Palate Craniofac J. 2012;49(4):397–405. doi:10.1597/10-007

47.

Cook

Kerins

Heppner

. Dental impacts on health-related quality of life of children with orofacial clefts. Pediatr Dent. 2016;38(3):218–223.

48.

Costa

White

Kiff

Davies

Stock

. Parent-reported socioemotional and cognitive development in children with a cleft lip and/or palate at 18 months: Findings from a UK birth cohort. Child Care Health Dev. 2021;47(1):31–39. doi:10.1111/cch.12813

49.

Damiano

Tyler

Romitti

, Momany

, Jones

, Canady

, Karnell

, Murray

. Health-related quality of life among preadolescent children with oral clefts: The mother’s perspective. Pediatrics. 2007;120(2):E283–E290. doi:10.1542/peds.2006-2091

50.

DeJonge

Kratunova

Wang

Patel

Avenetti

Alrayyes

. Parental perception of oral health and oral health Status of pediatric patients attending an urban craniofacial center. Cleft Palate Craniofac J. 2024;61(8):1329–1335. doi:10.1177/10556656231170136

51.

Fadeyibi

Coker

Zacchariah

Fasawe

Ademiluyi

. Psychosocial effects of cleft lip and palate on Nigerians: The ikeja-lagos experience. J Plast Surg Hand Surg. 2012;46(1):13–18. doi:10.3109/2000656X.2011.643027

52.

Filgueira

Azevedo

Ramalho

Gomes

Rego

. Quality of life of patients with cleft lip and/or cleft palate: Perspective of parents/guardians. Pesqui Bras Odontopediatria Clin Integr. 2015;15(1):431–440. doi:10.4034/PBOCI.2015.151.45

53.

Habibagahi

Fallahi

Neghabian

Hoghoughi

Babanouri

. Evaluation of the effect of palatoplasty on the quality of life and speech outcomes in patients with velocardiofacial syndrome. Dent Med Probl. 2024;61(1):29–34. doi:10.17219/dmp/130094

54.

Heike

Albert

Aspinall

, Bautista

, Bellucci

, Kapp-Simon

, Mecham

, Patrick

, Rosenberg

, Siebold

, et al. Development of an outcome measure of observable signs of health and well-being in infants with orofacial clefts. Cleft Palate Craniofac J. 2020;57(11):1266–1279. doi:10.1177/1055665620922105

55.

Jokovic

Locker

Stephens

Kenny

Tompson

Guyatt

. Measuring parental perceptions of child oral health-related quality of life. J Public Health Dent. 2003;63(2):67–72. doi:10.1111/j.1752-7325.2003.tb03477.x

56.

Jones

SMW

Edwards

Leroux

, Kapp-Simon

, Patrick

, Stueckle

, Rosenberg

, Albert

, Belluci

, Aspinall

, et al. Exploration of caregiver interrater agreement and test-retest reliability on the infant cleft observer outcomes (iCOO). Cleft Palate Craniofac J. 2022;60(8):10556656221089155. doi:10.1177/10556656221089155

57.

Kapp-Simon

Albert

Edwards

, Jones

, Crilly Bellucci

, Rosenberg

, Patrick

, Heike

. Developmental risk for infants with cleft lip with or without cleft palate based on caregiver-proxy reports. Cleft Palate Craniofac J. 2024;62(5):10556656231225304. doi:10.1177/10556656231225304

58.

Khoun

Malden

Turton

. Oral health-related quality of life in young Cambodian children: A validation study with a focus on children with cleft lip and/or palate. Int J Paediatr Dent. 2018;28(3):326–334. doi:10.1111/ipd.12360

59.

Kramer

Gruber

Fialka

Sinikovic

Schliephake

. Quality of life and family functioning in children with nonsyndromic orofacial clefts at preschool ages. J Craniofac Surg. 2008;19(3):580–587. doi:10.1097/SCS.0b013e31816aaa43

60.

Krueckeberg

Kapp-Simon

Ribordy

. Social skills of preschoolers with and without craniofacial anomalies. Cleft Palate Craniofac J. 1993;30(5):475–481. doi:10.1597/1545-1569_1993_030_0475_ssopwa_2.3.co_2

61.

Leopoldo-Rodado

Pantoja-Pertegal

Belmonte-Caro

Garcia-Perla

Gonzalez-Cardero

Infante-Cossio

. Quality of life in early age Spanish children treated for cleft lip and/or palate: A case-control study approach. Clin Oral Investig. 2021;25(2):477–485. doi:10.1007/s00784-020-03394-2

62.

Yakupu

, Li

, Zhang

, Aihaiti

, Xu

. Quality of life in patients with velopharyngeal insufficiency in west China. Cleft Palate Craniofac J. 2022;59(8):1024–1029. doi:10.1177/10556656211034107

63.

Morgan

Ataseven

Din

, Blum

, Paap

, Meier

, Carrizosa

, Malcarne

, Gosman

. Health-Related quality of life in craniofacial conditions: concordance between child and parent reports. Ann Plast Surg. 2020;84(Suppl 4):S295–S299. doi:10.1097/SAP.0000000000002292

64.

Murray

Arteche

Bingley

, Hentges

, Bishop

DVM

, Dalton

, Goodacre

, Hill

. The effect of cleft lip on socio-emotional functioning in school-aged children. J Child Psychol Psychiatry. 2010;51(1):94–103. doi:10.1111/j.1469-7610.2009.02186.x

65.

Naros

Brocks

Kluba

Reinert

Krimmel

. Health-related quality of life in cleft lip and/or palate patients - A cross-sectional study from preschool age until adolescence. J Craniomaxillofac Surg. 2018;46(10):1758–1763. doi:10.1016/j.jcms.2018.07.004

66.

Nguyen

Persson

Jagomagi

. Application of a new patient-reported outcome measure in orofacial clefts: An exploratory study in two countries. Stomatologija. 2019;21(3):72–78.

67.

Nicholls

Harper

Selvey

Robinson

Hartig

Persson

. Body esteem in a Western Australian cleft lip and/or palate cohort across 3 age groups. Cleft Palate Craniofac J. 2018;55(4):487–498. doi:10.1177/1055665617730362

68.

Nicholls

Selvey

Harper

Persson

Robinson

. The psychosocial impact of cleft in a Western Australian cohort across 3 age groups. Cleft Palate Craniofac J. 2019;56(2):210–221. doi:10.1177/1055665618769660

69.

Pasini

Cagidiaco

Fambrini

Miceli

Carli

. Life quality of children affected by cleft lip palate and alveolus (CLPA). Children (Basel). 2022;9(5):757. doi:10.3390/children9050757

70.

Patjanasoontorn

Pradaubwong

Rongbutsri

Mongkholthawornchai

Chowchuen

. Tawanchai cleft center quality of life outcomes: One of studies of patients with cleft lip and palate in Thailand and the Asia pacific region. J Med Assoc Thai. 2012;95(Suppl 11):S141–S147.

71.

Prathanee

Thanawirattananit

Surit

Mitkitti

Makarabhirom

. Speech task force and quality of life after surgery in children with cleft lip and palate: Limitation of professionals. Arch Plast Surg. 2024;51(3):275–283. doi:10.1055/s-0043-1776738

72.

Rahman

Leroux

McKinney

, Kapp-Simon

, Edwards

, Jones

, Rosenberg

, Patrick

, Daniels

, Stueckle

, et al. Observations by caregivers using the infant with clefts observation outcomes instrument (iCOO): A comparison of three versus seven-day daily diaries. Cleft Palate Craniofac J. 2024;61(9):1538–1542. doi:10.1177/10556656231175290

73.

Ramos

RPL

Victorio

DJB

Ramos

Pajuelo

Abanto

. Changes in the oral health-related quality of life in infants with cleft lip and/or palate before and after surgical treatment. Cleft Palate Craniofac J. 2022;59(1):47–53. doi:10.1177/1055665621993282

74.

Rando

Jorge

Vitor

LLR

, Carrara

CFC

, Soares

, Silva

, Rios

, Machado

, Gaviao

, Oliveira

. Oral health-related quality of life of children with oral clefts and their families. J Appl Oral Sci. 2018;26:e20170106. doi:10.1590/1678-7757-2017-0106

75.

Ranganathan

Shapiro

Carlozzi

, Baker

, Vercler

, Kasten

, Warschausky

, Buchman

, Waljee

. The feasibility and validity of PROMIS: A novel measure of quality of life among children with cleft lip and palate. Plast Reconstr Surg. 2016;138(4):675E–681E. doi:10.1097/PRS.0000000000002541

76.

Reissmann

John

Sagheri

Sierwald

. Diagnostic accuracy of parents’ ratings of their child’s oral health-related quality of life. Qual Life Res. 2017;26(4):881–891. doi:10.1007/s11136-016-1427-y

77.

Rosenberg

Bellucci

Edwards

, Heike

, Leroux

, Jones

, Stueckle

, Patrick

, Albert

, Aspinall

, et al. Caregiver observations of infant well-being before and after cleft lip surgery. Cleft Palate Craniofac J. 2022;61(2):10556656221125371. doi:10.1177/10556656221125371

78.

Ruff

Sischo

Chinn

Broder

. Development and validation of the child oral health impact profile - preschool version. Community Dent Health. 2017;34(3):176–182. doi:10.1922/CDH_4110Ruff07

79.

Sagheri

Ravens-Sieberer

Braumann

von Mackensen

. An evaluation of health-related quality of life (HRQoL) in a group of 4-7 year-old children with cleft lip and palate. J Orofac Orthop. 2009;70(4):274–284. doi:10.1007/s00056-009-9906-1

80.

Sander

Dietrich

Kloss-Brandstatter

, Krause

, Zimmerer

, Neuhaus

, Bartella

, Lethaus

. The cleft-screen-questionnaire (CSQ) - A validated screening instrument for health-related quality of life in cleft patients. J Plast Reconstr Aesthet Surg. 2022;75(9):3429–3435. doi:10.1016/j.bjps.2022.04.040

81.

Schonmeyr

Wendby

Sharma

Jacobson

Restrepo

Campbell

. Speech and speech-related quality of life after late palate repair: A patient’s perspective. J Craniofac Surg. 2015;26(5):1513–1516. doi:10.1097/SCS.0000000000001811

82.

Seifert

Davies

Harding

McLeod

Wren

. Intelligibility in 3-year-olds with cleft lip and/or palate using the intelligibility in context scale: Findings from the cleft collective cohort study. Cleft Palate Craniofac J. 2021;58(9):1178–1189. doi:10.1177/1055665620985747

83.

Skirko

Weaver

Perkins

, Kinter

, Eblen

, Martina

, Sie

KCY

. Change in quality of life with velopharyngeal insufficiency surgery. Otolaryngol Head Neck Surg. 2015;153(5):857–864. doi: 10.1177/0194599815591159

84.

Soedjana

Bangun

Christine

. Evaluating psychosocial problems in school-age children with cleft lip and palate in bandung, Indonesia using CBCL/6-18. Cleft Palate Craniofac J. 2022;59(10):1246–1252. doi:10.1177/10556656211040703

85.

Thompson

JMD

Ayrey

Slykerman

Stone

Fowler

. Behavioral outcomes in children with an orofacial cleft in a national study in New Zealand. Cleft Palate Craniofac J. 2021;58(8):1032–1039. doi:10.1177/1055665620972862

86.

Thompson

JMD

Ayrey

Slykerman

Stone

Fowler

. Quality of life using general population validated questionnaires in children with cleft lip and/or palate in New Zealand. Cleft Palate Craniofac J. 2021;58(6):779–786. doi:10.1177/1055665620962371

87.

van Veen

van den Berge

Moues-Vink

. Quality of life of adopted Chinese versus nonadopted Dutch children with cleft lip and/or palate: A propensity score matched analysis. Cleft Palate Craniofac J. 2022;59(12):1502–1508. doi:10.1177/10556656211050795

88.

Warschausky

Kay

Buchman

Halberg

Berger

. Health-related quality of life in children with craniofacial anomalies. Plast Reconstr Surg. 2002;110(2):409–414. doi:10.1097/00006534-200208000-00004

89.

Zeraatkar

Ajami

Nadjmi

Golkari

. Impact of oral clefts on the oral health-related quality of life of preschool children and their parents. Niger J Clin Pract. 2018;21(9):1158–1163. doi:10.4103/njcp.njcp_426_17

90.

Skirko

Weaver

Perkins

Kinter

Sie

KCY

. Modification and evaluation of a velopharyngeal insufficiency quality-of-life instrument. Arch Otolaryngol Head Neck Surg. 2012;138(10):929–935. doi:10.1001/2013.jamaoto.122

91.

Busschers

Validatieproces Vragenlijst Kwaliteit van Leven En Patiënttevredenheid Bij Kinderen van 0-8 Jaar Met (Niet)Uitwendig Zichtbare Schisis En/of Open Neusspraak Behandeld Door Een Regionaal Schisisteam : K(N)UZS-8 . Master’s Thesis. University of Groningen. 2016. https://umcg.studenttheses.ub.rug.nl/1803/

92.

Stiernman

Klintö

Persson

Becker

. Scores of the cleft hearing, appearance and speech questionnaire (CHASQ) in Swedish participants with cleft lip and/or cleft palate and a control population. Cleft Palate Craniofac J. 2021;58(3):347–353. doi:10.1177/1055665620952296

93.

Patjanasoontorn

Pradubwong

Mongkoltawornchai

Phetcharat

Chowchuen

. Development and reliability of the THAICLEFT quality of life questionnaire for children with cleft lip/palate and families. J Med Assoc Thai. 2010;93(Suppl 4):S16–S18.

94.

Riff

KWYW

Tsangaris

Goodacre

, Forrest

, Pusic

, Cano

, Klassen

. International multiphase mixed methods study protocol to develop a cross-cultural patient-reported outcome instrument for children and young adults with cleft lip and/or palate (CLEFT-Q). BMJ Open. 2017;7(1):e015467. doi:10.1136/bmjopen-2016-015467

95.

Wong Riff

KWY

Tsangaris

Goodacre

TEE

, Forrest

, Lawson

, Pusic

, Klassen

. What matters to patients with cleft lip and/or palate: An international qualitative study informing the development of the CLEFT-Q. Cleft Palate Craniofac J. 2018;55(3):442–450. doi:10.1177/1055665617732854

96.

Heliövaara

Küseler

Skaare

, Shaw

, Mølsted

, Karsten

, Brinck

, Rizell

, Marcusson

, Sæle

, et al. Scandcleft randomised trials of primary surgery for unilateral cleft lip and palate: 6. Dental arch relationships in 5 year-olds. J Plast Surg Hand Surg. 2017;51(1):52–57. doi:10.1080/2000656X.2016.1221352

97.

Long

Hathaway

Daskalogiannakis

, Mercado

, Russell

, Cohen

, Semb

, Shaw

. The americleft study: An inter-center study of treatment outcomes for patients with unilateral cleft lip and palate part 1. Principles and study design. Cleft Palate Craniofac J. 2011;48(3):239–243. doi:10.1597/09-180.1

98.

Semb

Brattström

Mølsted

Prahl-Andersen

Shaw

. The eurocleft study: Intercenter study of treatment outcome in patients with complete cleft lip and palate. Part 1: Introduction and treatment experience. Cleft Palate Craniofac J. 2005;42(1):64–68. doi:10.1597/02-119.1.1

99.

Powers

Howard

Saretsky

, Clifford

, Hoffmann

, Llorens

, Talbot

. Patient-Reported outcome assessments as endpoints in studies in infectious diseases. Clin Infect Dis. 2016;63(Suppl 2):S52–S56. doi:10.1093/cid/ciw317

100.

Lopez

Baelum

. Spanish version of the oral health impact profile (OHIP-Sp). BMC Oral Health. 2006;6(1):11. doi:10.1186/1472-6831-6-11

101.

Miroshnychenko

Rae

Riff

, Forrest

, Goodacre

, Swan

, Slator

, Goldstein

, Thoma

, Harman

, et al. Psychometric validation of the CLEFT-Q patient reported outcome measure: A prospective study to examine cross-sectional construct validity. Cleft Palate Craniofac J. 2023;60(3):327–335. doi:10.1177/10556656211062837

102.

Forer

Cohen

Ben-Bassat

Zini

Shalish

. Orthodontic treatment of patients with clefts: Satisfaction and psychological aspects. Cleft Palate Craniofac J. 2022;60(9):10556656221093934. doi:10.1177/10556656221093934

103.

Kelly

Shearer

. Appearance and speech satisfaction and their associations with psychosocial difficulties among young people with cleft lip and/or palate. Cleft Palate Craniofac J. 2020;57(8):1008–1017. doi:10.1177/1055665620926083

104.

Stiernman

Klintö

Persson

Becker

. Comparison of corresponding scores from the cleft hearing appearance and speech questionnaire (CHASQ) and CLEFT-Q in Swedish patients with cleft lip and/or palate. Cleft Palate Craniofac J. 2021;58(6):736–745. doi:10.1177/1055665620964124

105.

Borumand

Harrison

Rodrigues

. Regarding the “cleft screening questionnaire.”. J Plast Reconstr Aesthet Surg. 2022;75(10):3877–3903. doi:10.1016/j.bjps.2022.08.022

106.

Mpundu-Kaambwa

Bulamu

Lines

, Chen

, Dalziel

, Devlin

, Ratcliffe

. A systematic review of international guidance for self-report and proxy completion of child-specific utility instruments. Value Health. 2022;25(10):1791–1804. doi:10.1016/j.jval.2022.04.1723

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.07 MB

0.05 MB