The interval scaling properties of the London Handicap Scale: an example from the adaptation of the scale for use in Turkey

Abstract

Objectives: To adapt the London Handicap Scale into the Turkish language, and to investigate the scaling properties of this version in a sample of people who have experienced a stroke.

Design: After the translation process, the internal construct validity was tested by Rasch analysis and the reliability by internal consistency and intraclass correlation coefficient. The interval scaling properties were assessed by contrasting the raw and weighted London Handicap Scale scores with the Rasch latent estimates.

Setting: An outpatient rehabilitation unit of a university hospital.

Subjects: One hundred and eighty-eight community-dwelling post-stroke patients (mean age 63 (SD 12) years, 54% male) were assessed by the Turkish version of the London Handicap Scale.

Results: After adjustment for local dependency, the data showed good fit to Rasch model expectations with a mean item fit −0.240 (SD 1.868), person fit −0.403 (SD 0.893) and chi-square interaction 8.55 (df 10, P = 0.575). The reliability was good with a Cronbach’s α and intraclass correlation coefficient of 0.845. Analysis of the scaling properties showed that either the raw London Handicap Scale score or its weighted score were non-linear with respect to the Rasch latent estimate.

Conclusions: The London Handicap Scale is a valid and reliable scale for use in stroke in Turkey. Its unweighted raw scores and weighted scores are equivalent and ordinal, but a linear transformation is possible through Rasch analysis.

Introduction

The bio-psychosocial model, as operationalized by the International Classification of Functioning, Disability and Health developed by the World Health Organization (WHO), is making a considerable impact on the way in which data concerning disability are conceptualized, collected and processed.^2,3 A recent review of 23 different health-related quality of life instruments found in 71 studies showed that a wide distribution of body function, activity and participation categories were available in these instruments.⁴ But in rehabilitation following stroke, the majority of assessments used in randomized controlled trials appear to focus on impairments or activity limitation and few on participation restriction.^5,6 This may in part be due to the lack of conceptual clarity that exists between activities and participation,⁷ something that appeared to be more clearly defined in its predecessor, the International Classification of Impairments, Disabilities and Handicaps (ICIDH).⁸

One scale, the London Handicap Scale, was specifically designed to measure the dimension concerned with roles in society, defined as handicap in the ICIDH, and potentially similar to the participation construct in the International Classification of Functioning, Disability and Health.⁹ This scale has been shown to map onto many of the current International Classification of Functioning, Disability and Health components, despite its origin in the earlier version.⁴ It has also been widely adapted into different languages.^10,11 It is an unusual scale in that when the original weighting is applied it is said to be at the interval level, although later work suggested that simply summing the items gave the same result.¹² This raises questions about the validity of the claim for interval scaling as the summation of any set of such items is invariably ordinal.¹³

The adaptation of the London Handicap Scale for use in rehabilitation setting in Turkey offers a chance to examine the scaling properties of the scale. As there is no validated measure available for the assessment of participation in the Turkish rehabilitation setting, the London Handicap Scale was chosen for this purpose. Thus the aim of this study was to adapt the scale into the Turkish language, and investigate the scaling properties in a sample of people who have experienced a stroke.

Methods

For the adaptation into the Turkish language, internationally accepted guidelines for the process of cross-cultural adaptation were utilized.¹⁴ The original London Handicap Scale was translated into Turkish by four bilingual professionals. Three professionals were rehabilitation medicine specialists, and were thus informed translators. The fourth translator was an English teacher and was thus an uninformed translator. Inconsistencies in the translations were resolved by discussions between the translators. At this stage some literal and structural modifications were necessary to be understandable in the Turkish setting. For example; for the first item ‘getting around’ this activity was stressed as ‘getting around/going from one place to another’. For the second item ‘looking after yourself,’ ‘… looking after money’, ‘… be left alone safely’ and ‘… need constant attention’ were translated as ‘… managing the money issues’, ‘… stay alone safely’ and ‘… need constant care’, respectively.

The accepted Turkish version of London Handicap Scale at the synthesis of translation stage was back-translated by a bilingual translator (native English speaker) who was blind to the original version. Then at the expert committee stage, all five translators involved in the adaptation process reviewed and discussed the translations and the pre-final version before field-testing was considered without any necessary modification. Field-testing for face validity was undertaken in a group of 20 patients receiving neurological or musculoskeletal rehabilitation with variable ages and educational levels. At this stage, some items necessitated further modifications to be more understandable. For example, in the item ‘getting around’, the statement ‘you are confined to a bed or a chair’ was modified to ‘you can’t get out of the bed or stand up from a chair’. In the item ‘work and leisure’, the statement ‘… sports’ and ‘… take part in any activities’ were changed to ‘… sports activities’ and ‘… perform any activities’, respectively.

Community-dwelling stroke patients, who had been hospitalized for stroke rehabilitation during the last four years at the Department of Physical Medicine and Rehabilitation of the Medical Faculty, Ankara University were asked to participate by telephone call. Then the patients who agreed to participate were invited to the outpatient clinic for evaluation. To be included in this study, all the patients were interviewed and assessed by physical medicine and rehabilitation specialists. Exclusion criteria were: (1) Age younger than 18 years, (2) failure to obtain informed consent for reasons including conscious disturbances and severe cognitive impairment; (3) patients with relevant comorbidities which might affect their disability (uncompensated heart failure, amputee, etc.); (4) serious communication problems (moderate to severe aphasia) based on clinical judgement. Written informed consent was obtained from all patients (or their close relatives) and the study was approved by the Ethical Committee of Ankara University, Medical Faculty.

Assessment included the administration of the Turkish versions of the London Handicap Scale, the WHO Disability Assessment Schedule II and the Functional Independence Measure.^15–17

The London Handicap Scale is a widely used measure of handicap based on the ICIDH model of the World Health Organization.⁸ It includes six items: mobility, physical independence, occupation, social integration, orientation and economic self-sufficiency. In the original London Handicap Scale, each item is classified on a six-level scale (1: no disadvantage, 6: extreme disadvantage). A weighted total handicap score is produced with scores ranging from 0 (maximum disadvantage) to 100 (no disadvantage) by using a matrix of scale weights for each item and each classification level.^9,15

The reliability of the London Handicap Scale was initially tested by the intraclass correlation coefficient (ICC 3,1), and by internal consistency.^18,19 Subsequently reliability was further tested by the Person Separation Index from the Rasch analysis (see below).

The internal construct validity of the London Handicap Scale was assessed by Rasch analysis.²⁰ The Rasch model is consistent with additive conjoint measurement, a theory which defines how interval scale measurement can be derived from ordinal questionnaires.^21,22 This is of crucial importance given the claim that the London Handicap Scale was interval in nature.⁹ Consequently, the ‘interval’ scale estimate derived from the weighted scores can be contrasted against that derived from the Rasch model. Various aspects of measurement are addressed in the process of Rasch analysis, including local dependency, appropriate response category structure of the items, differential item functioning and unidimensionality. Full details of these processes are given in greater detail elsewhere.^23,24 Briefly, the scale will be expected to satisfy the assumptions of local dependence, including strict unidimensionality, to be free of differential item functioning, and the items to conform to model expectations, indicated by a range of fit statistics where, for example, chi-square values are non-significant, and individual item residuals range between ± 2.5. Expected levels of good fit are given at the base of the fit tables presented below. Where necessary, breaches of local independence assumptions are accommodated by creating testlets – summary scores from the items that are locally dependent, which are then treated as one new larger variable.²⁵ When this is undertaken, the item residual summary standard deviation may inflate when testlets are of different lengths. An estimate of the internal consistency reliability of the scale is also available, based on the Person Separation Index where the estimates on the logit scale for each person are used to calculate reliability, and are equivalent to Cronbach’s alpha.¹⁸

The external construct validity of the London Handicap Scale was assessed through convergent validity with the Functional Independence Measure and the WHO Disability Assessment Scale II (WHODAS-II). The degree of association with the Functional Independence Measure and the WHODAS-II was analysed by Spearman’s correlation coefficient.

The interval scaling properties of the London Handicap Scale was assessed by contrasting the raw and weighted London Handicap Scale scores with the interval scale Rasch person estimates.

For the Rasch analysis, it is reported that a sample size of 150 patients will, for most purposes, estimate item difficulty, with α of 0.01, to within ± 0.5 logits.²⁶ This sample size is also sufficient to test for differential item functioning where, at α of 0.01 a difference of 0.7 standard deviations within the residuals can be detected for any two groups with β of 0.20. Bonferroni corrections are applied to both fit and differential item functioning statistics due to the number of tests undertaken.²⁷ A value of 0.05 is used throughout.

Statistical analysis was undertaken with SPSS version 11.5 (SPSS Inc., Chicago, IL, USA), and Rasch analysis with RUMM2020.²⁸

Results

Two hundred and sixty-nine community-dwelling stroke patients were called by telephone. Thirty-five patients could not be reached, 26 patients had died and 20 patients did not want to participate for various reasons such as comorbidities or communication problems. Therefore 188 community-dwelling stroke patients eligible to participate were recruited for the study. The mean age was 63.1 (SD 12) years, 53.7% (n = 101) were male and 1.6% of the patients were working. The median time since stroke was 27 months (minimum–maximum: 3–240 months) and the median time since discharge was 12.5 months (minimum–maximum: 1–82 months). 39% (n = 74) of the patients were at the primary level of education (5 years of education), 23% (n = 44) at the middle level (8–11 years of education) and 11% (n = 20) at university level of education. Twnenty-seven per cent (n = 50) of the group were illiterate or had not completed primary level of education. Seven out of eight (83.5%, n = 154) patients had an ischaemic stroke, and just over half (51.6%, n = 97) were left hemiplegic. The mean London Handicap Scale score of the 188 patients was 54.3 (SD 19.8) (range 0–100).

Cronbach’s α, and ICC (95% confidence interval) values were 0.845 and 0.845(0.809–0.877) for the weighted score.

Data from the six items were checked for appropriateness of the rating scale version of the Rasch model.²⁹ The likelihood ratio test indicated that the assumptions were not met and therefore the unrestricted or partial credit version of the Rasch model was used.³⁰ Only two of the six items were found to have ordered categories, that is, the transition points between adjacent categories represented an increasing level of handicap. Consequently four of the items had two categories collapsed together, meaning that each was reduced to five categories. Fit to the model at this stage was poor (Table 1, Analysis 1), and the six items displayed multidimensionality.

Table 1

Rasch analysis: Summary fit statistics

Analysis	Item residual		Person residual		Chi-square interaction		PSI	Unidimensionality (%) (CI when % >5.0)
	Mean	SD	Mean	SD	Value (df)	P-value
1)	−0.342	2.247	−0.381	1.037	42.96 (12)	<0.001	0.876	9.29 (6.1–12.4)
2)	−0.526	2.849	−0.410	0.873	8.36 (6)	0.214	0.797	0.55 (0–3.7)
3)	−0.240	1.868	−0.403	0.893	8.55 (10)	0.575	0.811	Not available
Satisfactory fit		<1.4		<1.4		>0.05	>0.7	<5.0 or lower CI (<5.0)

PSI, Person Separation Index; SD, standard deviation; CI, confidence interval.

Local dependency was then observed in the data. Thus the items ‘getting around’, ‘looking after yourself’ and ‘work and leisure’ were grouped together (testlet 1), as were the items ‘getting along with people’ and ‘awareness of your surroundings’ (testlet 2). The economic self sufficiency item was left by itself (testlet 3). This resulted in a much improved fit to the model (Analysis 2), given the caveat of the inflated item residual standard deviation. However, the reliability had fallen substantially, reflecting the influence of local dependency. This had also influenced the multidimensionality test, which now showed a strictly unidimensional scale.

Differential item functioning by age was evident for the first and third testlet, and by education for the first testlet. Adjusting for age by allowing item difficulty to vary by age group (splitting for differential item functioning) solved the problem, and the educational bias in testlet 1 was no longer present. Fit to the model further improved with a mean item fit −0.240 (SD 1.868), person fit −0.403 (SD 0.893) and chi-square interaction 8.55 (df 10, P = 0.575) (Analysis 3).

The scale appeared well targeted to the sample with a mean person fit of −0.573, suggesting that this sample were just slightly more disadvantaged than the average of the scale (which has a logit value of zero) (Figure 1). Reliability of the resulting final version was 0.81, consistent with group use.

Figure 1

Targetting of London Handicap Scale.

Regarding the external construct validity, the correlations (Spearman’s R) of the London Handicap Scale with the WHODAS-II, Functional Independence Measure motor and cognitive scales were 0.92, 0.79 and 0.68, respectively. These correlations were as expected and confirm the convergent validity of the scale.

The interval scale latent estimate from the Rasch analysis was exported to contrast with both the raw and the weighted London Handicap Scale scores. Figure 2 clearly demonstrates that either the raw score or its weighted score were non-linear with respect to the Rasch latent estimate. The raw London Handicap Scale score and the weighted score appear equivalent, consistent with the earlier work.¹²

Figure 2

Comparisons of London Handicap Scale raw and weighted scores with each other and with Rasch latent estimates.

Discussion

The Turkish adaptation of the London Handicap Scale resulted in good reliability, internal and external construct validity, which supports its use as a measure of participation in community-living stroke patients in Turkey. Reliability and external validity results are in concordance with previous reports.^9,11,12 This is only the second paper to subject the London Handicap Scale to Rasch analysis,³¹ and the first to specifically examine the scaling properties of the instrument. Having adjusted for local dependency, the set of items showed good fit to the Rasch model, confirming that the raw score is a sufficient statistic for estimating participation or, as the original paper suggested,⁹ the extent of disadvantage experienced by a person.

The study raises a number of issues. Foremost is that of interval scaling derived from conjoint analysis, which was the original method used to derive the weighted London Handicap Scale score. The earlier work by Jenkinson et al. ¹² had laid the grounds for concern that the weighted score may have been ordinal, given they found concordance between the simple summed score and the weighted score. The summed score from any set of items will always be at the ordinal level and thus this implies that the weighted level may also be so. However, correlation-based evidence, which they presented, does not confirm this. Given no missing data, there is always a one-to-one relationship between a raw score and its interval scale transformation in Rasch analysis, and thus the correlation, which is a measure of co-variation, is always very high.³² Consequently it was important to evaluate the weighted London Handicap Scale score against the Rasch latent estimate derived from the instrument, and this confirmed that the weighted score is indeed ordinal. As the type of approach used to develop the London Handicap Scale is also similar to those used to develop health utility scales, this raises concerns about the true interval nature of those scales.

Another issue is that of the definition of, and the measurement of participation itself. The London Handicap Scale originates in the earlier framework of the ICIDH and thus can be questioned as a measure of participation in the manner of the International Classification of Functioning, Disability and Health. However, the interpretation of participation, and its relationship with activity limitation, remains an area of concern with continuing attempts to redefine and clarify the distinction.⁷ Thus in the presence of such debate, a widely used scale which addresses the restriction in participation, albeit from a somewhat older perspective, remains of value. However, an increasing number of participation scales are being developed, some of which are making use of modern psychometric approaches in their development.^33,34 It remains to be seen if the London Handicap Scale has concordance with these new measures, given that it shares many of the components of the International Classification of Functioning, Disability and Health and is still being adapted for use in various countries.^4,11

There are a number of limitations to this study. This is only one relatively small sample and thus the results would benefit from replication in a much larger sample. As a consequence the sample was also too small to provide a raw-score interval scale translation with a sufficient degree of precision. Only one diagnostic group was included, yet the London Handicap Scale is a generic scale, and thus the findings would also benefit from replication across other diagnostic groups. Likewise, this study is concerned with the Turkish adaptation of the scale and thus the question arises as to whether or not different language versions will replicate the findings. Given the widespread use of the scale since its development, secondary analysis of existing data sets should be encouraged

In conclusion, the London Handicap Scale is now available for use in Turkey and can be obtained from the authors. The scale has demonstrated concordance with modern psychometric standards. The interval scaling claims of the scale have been shown to be erroneous, at least in this sample of stroke patients, but an interval scale transformation is available through the process of Rasch analysis. Replication of these findings from other samples across countries and diagnoses would be a distinct advantage.

Clinical messages

The Turkish version of the London Handicap Scale is reliable and valid for use in stroke.

The raw or weighted London Handicap Scale score is ordinal in nature. Therefore non-parametric statistics are appropriate.

References

Kutlay S, Elhan AH, Oztuna D, Yanik B, Kucukdeveci AA, Tennant A. Scaling properties of the London Handicap Scale in stroke. Clin Rehabil 2010 24: 276.

World Health Organization. International classification of functioning, disability and health (ICF). Geneva: World Health Organization, 2001.

Jeisma

. Use of the International Classification of Functioning, Disability and health: a literature survey. J Rehabil Med 2009; 41: 1–12.

Geyh

Cieza

Kollerits

Grimby

. Content comparison of health related quality of life measures used in stroke based on the international classification of functioning, disability and health (ICF): a systematic review. Qual Life Res 2007; 16: 833–51.

Haigh

Tennant

Biering-Sørensen

. The use of outcome measures in Physical Medicine and Rehabilitation within Europe. J Rehabil Med 2001; 33: 273–78.

Salter

Foley

Jutai

Teasell

. Assessment of participation outcomes in randomised controlled trials of stroke rehabilitation interventions. Int J Rehabil Res 2007; 30: 339–42.

Badley

. Enhancing the conceptual clarity of the activity and participation components of the International Classification of Functioning, Disability, and Health. Soc Sci Med 2008; 66: 2335–45.

World Health Organization. International Classification of Impairment, Disability and Handicap. Geneva: World Health Organization, 1980.

Harwood

Rogers

Dickinson

Ebrahim

. Measuring handicap: the London handicap scale, a new outcome measure for chronic disease. Qual Health Care 1994; 3: 11–16.

10.

Groothuis-Oudshoorn

Chorus

Taeke van Beekum

Detmat

van den Hout

. Modelling and estimation of valuations for the Dutch London Handicap Scale. J Health Econ 2006; 25: 1119–38.

11.

Westergren

Hagell

. Initial validation of the Swedish version of the London Handicap Scale. Qual Life Res 2006; 15: 1251–56.

12.

Jenkinson

Mant

Carter

Wade

Winner

. The London Handicap Scale: a re-evaluation of its validity using standard scoring and simple summation. J Neurol Neurosurg Psychiatry 2000; 68: 365–67.

13.

Stucki

Daltroy

Katz

Johannesson

Liang

. Interpretation of change scores in ordinal scales and health status measures: the whole may not egual the sum of the parts. J Clin Epidemiol 1996; 49: 711–17.

14.

Beaton

Bombardier

Guillemin

Ferraz

. Guidelines for the process of cross cultural adaptation of self report measures. Spine 2000; 25: 3186–91.

15.

Harwood

Ebrahim

. Manual of the London Handicap Scale. Nottingham: University of Nottingham, 1995.

16.

Ulug

Ertugrul

Gogus

Kabakcı

. Yetiyitimi degerlendirme cizelgesinin (WHODAS II) sizofreni hastalarında gecerlilik ve güvenilirligi. Turk Psikiyatr Derg 2001; 12: 121–30.

17.

Kücükdeveci

Yavuzer

Elhan

Sonel

Tennant

. Adaptation of the Functional Independence Measure for use in Turkey. Clin Rehabil 2001; 15: 311–19.

18.

Cronbach

. Coefficient alpha and the internal structure of tests. Psychometrika 1951; 16: 297–334.

19.

Streiner

Norman

. Health measurement scales. Oxford: Oxford University Press, 1995.

20.

Rasch

. Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press, 1960.

21.

Luce

Tukey

. Simultaneous conjoint measurement: a new type of fundamental measurement. J Math Psychol 1964; 1: 1–27.

22.

Perline

Wright

Wainer

. The Rasch model as additive conjoint measurement. Appl Psychol Measure 1979; 3: 237–56.

23.

Pallant

Tennant

. An introduction to the Rasch measurement model: an example using the hospital anxiety and depression scale (HADS). Br J Clin Psychol 2007; 46: 1–18.

24.

Tennant

Conaghan

. The Rasch Measurement Model in Rheumatology: What is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis Rheum 2007; 57: 1358–62.

25.

Wainer

Kiely

. Item clusters and computer adaptive testing: A case for testlets. J Educ Meas 1987; 24: 185–210.

26.

Linacre

. Sample size and item calibration stability. Rasch Meas Trans 1994; 7: 28–28.

27.

Bland

Altman

. Multiple significance tests: the Bonferroni method. BMJ 1995; 310: 170–170.

28.

Andrich

Lyne

Sheridan

Luo

. RUMM 2020. Perth: RUMM Laboratory Duncraig, 2003.

29.

Andrich

. Rating formulation for ordered response categories. Psychometrika 1978; 43: 561–73.

30.

Masters

. A Rasch model for partial credit scoring. Psychometrika 1982; 47: 149–74.

31.

Dubuc

Haley

Kooyoomjian

Jette

. Function and disability in late life: comparison of the Late-Life Function and Disability Instrument to the Short-Form-36 and the London Handicap Scale. Disabil Rehabil 2004; 26: 362–70.

32.

Linacre

. Do correlations prove scores linear? Rasch Meas Trans 1998; 12: 605–606.

33.

Pallant

Misajon

Bennett

Manderson

. Measuring the impact and distress of health problems from the individual’s perspective: development of the Perceived Impact of Problem Profile (PIPP). Health Qual Life Outcomes 2006; 29: 36–36.

34.

Noonan

Kopec

Noreau

Singer

Dvorak

. A review of participation instruments based on the International Classification of Functioning, Disability and Health. Disabil Rehabil 2009; 8: 1–19.