Abstract
This article presents the computed adaptive testing of the World Health Organization Disability Assessment Schedule 2.0 (CAT–WHODAS 2.0) as an efficient solution to provide reliable, valid, and sex-unbiased assessments of disability among people with dementia.
Dementia is a major cause of disability that typically begins with cognitive decline in memory function and eventually affects higher level cognition, like executive function (Lisko et al., 2021; Smits et al., 2015). People with dementia may initially experience limitations in social participation, followed by difficulties in routine daily activities (Giebel et al., 2015). Ultimately, they may struggle to live independently, which could lead to a decreased quality of life (Giebel et al., 2015). Although cognitive decline may not be effectively controlled (Perneczky, 2019), its impact on daily living can be managed by assistance efforts, such as environmental modifications and compensatory strategies (Maki et al., 2018; Tomaszewski Farias et al., 2018). A reliable and valid measure for the assessment of disability levels across multiple functioning domains would be beneficial for efficient identification of the need for assistance.
The World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0) is widely used to assess levels of disability (Federici et al., 2017; Üstün, 2010). Its multidimensional structure provides informative profiles regarding disability in six domains (Chiu et al., 2014; Üstün, 2010). Moreover, it uses generic items, so comparisons can be made across people with various health conditions such as dementia and stroke (Castro et al., 2019). The wide applicability has led to its widespread use as a critical outcome measure in medical and rehabilitative interventions (Potcovaru et al., 2024).
However, the lengthy assessment severely limits the utility of the WHODAS 2.0, which takes approximately 20 min to administer (Üstün, 2010). Because most people with dementia have impaired attention spans (Grant & Chamberlain, 2023), their willingness to complete a lengthy assessment is reduced. The utility may be further constrained for people with cognitive impairments because they may need additional clarifications to ensure accurate responses (Huang et al., 2017). Therefore, an efficient measure with good reliability and validity can help overcome this utility challenge (Kruyen, 2012).
Computerized adaptive testing (CAT) offers a promising solution for the tradeoff between reliability and efficiency in traditional assessments (Wang & Chen, 2004). The CAT can increase assessment efficiency while maintaining sufficient reliability compared with the full-length version, because it uses only the most informative items, which are neither too easy nor too difficult for examinees (Lee et al., 2018, 2021; Lin et al., 2019). Moreover, construction of the CAT can be based on the Rasch model, offering advanced testing properties that are helpful for users, such as interval Rasch scores and an individualized reliability index (Hays et al., 2000). Therefore, developing a measure such as the CAT can improve both the quality and efficiency of the assessment.
In this study, we aimed to develop a CAT version of the WHODAS 2.0 (CAT–WHODAS 2.0) for people with dementia. We expected the CAT–WHODAS 2.0 to address the aforementioned utility challenge.
Method
Participants
Data were extracted from the Taiwan Databank of Persons with Disabilities, a nationwide database that registers people seeking disability certification. The data were collected from 2012 to 2022 through interviews conducted by trained interviewers who passed the examinations authorized by the Ministry of Health and Welfare. The interviewers selected suitable respondents (i.e., the patients themselves or proxies) to ensure the validity of the data, given confounding factors such as cognitive and hearing impairments. The access and utilization of the data were approved by Ministry of Health and Welfare and the institutional review board of the National Taiwan University Hospital.
Data were included if the examinees were diagnosed as having dementia and were applying for certification of disability for the first time. We excluded data reported by proxies or that had any missing response on the WHODAS 2.0.
Measures
The WHODAS 2.0 is a generic measure used to assess levels of disability. It comprises 36 items divided into the following six domains: Cognition, Mobility, Self-Care, Getting Along (e.g., maintaining a friendship), Life Activities (e.g., taking care of household responsibilities), and Participation (Üstün, 2010). Each item is rated on a 5-point Likert-type scale ranging from no difficulty at all (0) to extremely difficult (4). A higher score indicates a more significant disability. The WHODAS 2.0 exhibits good reliability and validity in people with disabilities (Chiu et al., 2014; Yen et al., 2014).
Procedure
We initially extracted the participants’ responses on the WHODAS 2.0 from the database. Then, we examined the model fits of the CAT–WHODAS 2.0 items, deleted items that showed unsatisfactory model fits, and used the remaining items to form the item bank. We examined the unidimensionality of each domain and the consistency of item difficulties between men and women using residual-based principal component analysis (PCA) and differential item functioning (DIF) analysis, respectively. Finally, we simulated the reliability and efficiency of the CAT–WHODAS 2.0 with 16 candidate sets of stopping rules and determined the best set of stopping rules to achieve both reliable and efficient assessments. The following section provides further details.
Statistical Analyses
We used the one-parameter Rasch model because it offers interval scores with consistent meanings across examinees, which cannot be achieved by models with more item parameters (Stemler & Naples, 2021). The partial credit model was used to allow each item to have unique step difficulties (Andersen, 2012). A multidimensional model was used because the WHODAS 2.0 assesses six domains simultaneously (Üstün, 2010).
We used infit mean square (MNSQ) and outfit MNSQ to evaluate whether the examinees’ responses aligned with the Rasch model’s expectations (Wang & Chen, 2004). Infit MNSQ focuses on unexpected responses for items with difficulty close to examinees’ levels of disability, whereas outfit MNSQ treats all responses equally. Items with an infit or outfit MNSQ exceeding 1.4 were considered misfitting and were deleted iteratively (Wright & Linacre, 1994). The Rasch analysis was conducted using ConQuest, Version 2.0 (Wu et al., 2007). We also calculated the variances explained by the Rasch model, and values exceeding 50% were considered sufficient for scientific investigations (Linacre, 2008).
We examined the unidimensionality of each domain further, using residual-based PCA (Linacre, 1998). The eigenvalue was calculated to examine whether any common factor existed underlying the residuals. A common factor was considered to exist if the largest eigenvalue exceeded 3.0 (Costello & Osborne, 2019).
We assessed the influence of the examinee’s sex on the scoring by using DIF analysis with DIF values and Z scores. The DIF values are the differences in item difficulties estimated for men and women, reflecting the magnitude of influence of examinees’ sex on item difficulties (Zieky, 2003). We calculated the Z scores by dividing the DIF values by their corresponding standard error to rule out differences attributed to measurement errors (Lee et al., 2022). Items were determined to exhibit DIF if their DIF values exceeded ±.38 and the Z scores were outside the range of ±1.96 (Lee et al., 2022). Uniform DIF was targeted because we adopted the Rasch model.
The candidate sets of stopping rules were proposed on the basis of two commonly used approaches, both individually and collectively: the minimal required reliability (MRR) and limited reliability increase (LRI; Lee et al., 2018, 2021). A total of 16 sets of stopping rules were proposed, including three for the MRR alone (i.e., MRR ≥.90, ≥.80, and ≥.70), three for the LRI alone (i.e., LRI <.001, <.005, and <.010), nine for combinations of both stopping rules (e.g., MRR ≥.90 or LRI <.001), and one without any stopping rules to represent the performance of the item bank.
The best set of stopping rules was selected to achieve high reliability and efficiency through simulations. The simulations included the selection of the most informative item at each step, estimation of examinees’ disability levels on the basis of their responses, and judgment on whether to stop the assessments depending on the set of stopping rules. The first item was selected by setting the initial Rasch scores as means (i.e., zeros) of the item difficulties for all domains. We used the Bayesian maximum a posteriori estimation with a Newton-Raphson method for ability estimation (Wang & Chen, 2004). At least one item would be selected for each domain unless the total number of items for an administration was less than the number of WHODAS 2.0 domains.
We examined the concurrent validity of the CAT–WHODAS 2.0 using the Pearson correlation coefficient (r). The correlations were calculated between the domain scores of the CAT–WHODAS 2.0 and the item bank, and between the CAT–WHODAS 2.0 and the original WHODAS 2.0. High rs (>.7) were considered to indicate good concurrent validity (Akoglu, 2018).
We calculated the two-way random absolute agreement intraclass correlation coefficient (ICC) to examine the absolute difference and/or agreement between scores (Koo & Li, 2016). The ICC was calculated only for the CAT–WHODAS 2.0 and the item bank because of they are scaled the same.
Results
Table 1 shows the characteristics of the participants. Data from a total of 3,124 participants were analyzed. The participants’ mean age was 74.2 yr. Most participants were certified as having moderate disability levels (36.6%), followed by mild (28.4%) and severe (22.4%) disability levels.
Participant Characteristics
Note. N = 3,124.
Table 2 shows the model fits, the item and step difficulties, and the DIF of sex for the items of each domain. In total, 27 of 36 items exhibited good model fit (infit MNSQ = 0.80–1.34; outfit MNSQ = 0.58–1.35; those of the misfit items are shown in Table A.1 in the Supplemental Material, available online with this article at https://research.aota.org/ajot) and were retained for further analysis. The multidimensional Rasch model explained 84.8% of the variances. None of the retained items showed significant DIF of sex (range of DIF values = −0.07 to 0.04). However, the Z scores (−6.36 to −2.36, or 2.00–3.64) of six items exceeded the acceptable range. The largest eigenvalue of the residual-based PCA was 2.6.
The Model Fits, Item and Step Difficulties, and DIF of Sex of the Item Bank
Note. τ1 to τ4 represent the (centralized) step difficulties. D = domain; DIF = differential item functioning.
The DIF value indicates the differences in item difficulties estimated for men and women. DIF items were determined if DIF values exceeded ±0.38 and Z scores exceeded ±1.96 simultaneously.
Table 3 exhibits the reliability and efficiency of the CAT–WHODAS 2.0 and the item bank. The LRI-alone sets of stopping rules provided high reliability comparable with the item bank (range = .93–.95 vs. .93–.96). However, the assessments were lengthy, requiring more than 19.1 items, on average. The “MRR ≥.90” rule seemed promising, offering reliabilities that were slightly lower than those of the item bank (range = .90–.91 vs. .93–.96). However, we determined the combined set (MRR ≥0.90 or LRI <0.010) as the best set of stopping rules, which used fewer items (8.1 vs. 9.4) than the “MRR ≥.90” rule to achieve same reliabilities (.90–.91). In summary, the CAT–WHODAS 2.0 required about nine items to provide high reliabilities ranging from .90 to .91. The mean numbers of items for each domain ranged from 0.8 (the Getting Along domain) to 1.8 (the Participation domain). Extremely high correlations and agreement were found between the domain scores on the CAT–WHODAS 2.0 and those of the item bank (rs and ICCs = .96–.99).
Comparisons of the Reliability and Efficiency Between the WHODAS 2.0 and the Item Bank
Note. CAT–WHODAS 2.0 = computerized adaptive testing of the World Health Organization Disability Assessment Schedule 2.0; COG = Cognition domain; GA = Getting Along domain; ICC = interclass correlation coefficient; LA = Life Activities domain; MOB = Mobility domain; PART = Participation domain; SC = Self-Care domain.
% ≥.90 indicates the percentages of the participants who obtained individual Rasch reliabilities higher than .90. Conceptually, a larger percentage represents a more reliable measure.
r with the raw score indicates the correlations between the CAT–WHODAS 2.0 domain scores and the summed scores in each domain.
Discussion
The good model fit for the 27 items indicates that the participants’ responses matched the expectations of the Rasch model. The good model fit supports the unidimensionality of these items in each domain, because the Rasch model assumes that items in a domain assess the same construct (Hays et al., 2000). Moreover, the Rasch model explained about 85% of the variance, indicating that these items can effectively capture the assessment information. Furthermore, the largest eigenvalue within the residuals was below the threshold for a common factor, suggesting that no common factors existed within the residuals. On the basis of these findings, the items within each domain appear unidimensional and can provide separate domain scores for the disability levels of the six functioning domains.
Our analysis showed small DIF values for all CAT–WHODAS 2.0 items, despite some Z scores being large. The findings support the similarity in item difficulty for men and women, enabling unbiased comparisons of disability levels between sexes for persons with dementia. Using sex-unbiased items helps users avoid misinterpreting score differences as sex advantages (Chiu et al., 2014; Üstün, 2010; Yen et al., 2014); higher scores may be caused by better functioning and/or relatively easier items that cannot be clarified if the DIF related to sex is unexamined. Sex-biased items are common in measures that assess life perspectives and social interactions (Chiang et al., 2020; Lee et al., 2021), possibly because men and women have different life experiences shaped by social and/or cultural expectations. For example, women are often expected to have better social skills to get along with others (Löffler & Greitemeyer, 2023). These sex-specific experiences might have altered the item difficulties, leading to sex-biased assessments and misinterpretations. Although the evidence is not yet sufficient to determine whether the causes of DIF are the result of nature or nurture, our findings support that the CAT–WHODAS 2.0 items were not affected by examinees’ sex. Thus, the scores can be comparable for people across sexes.
The CAT–WHODAS 2.0 showed improved utility by using about nine items to achieve high Rasch person reliabilities exceeding .90. Furthermore, it exhibited good concurrent validity and agreement to the item bank, with high correlation and ICC values exceeding .96. These findings support that the CAT–WHODAS 2.0 is an efficient, reliable, valid, and sex-unbiased measure for assessing the levels of disability in six domains in people with dementia. Thus, the CAT–WHODAS 2.0 can serve as a promising alternative for users in both clinical and research settings to reduce assessment burden for both administrators and participants.
The nine misfit items may be attributed to their content being affected by multiple factors (Byrne, 2013; Lee et al., 2023). For instance, the four items about the limited access to work and school might be influenced by real-life opportunities to participate in these activities and by respondents’ function or disability in life activities. Thus, the participants’ responses to these items might have deviated from model expectations, because the Rasch model assumes that items within a domain assess a single factor (Hays et al., 2000). Clarifying the item descriptions or refining the latent trait that the items aim to assess may resolve these issues and improve their appropriateness for assessing disability levels (Schreiber et al., 2006).
Several advantages are noted for the CAT–WHODAS 2.0. First, it provides interval scores that reflect the severity of disability, which benefits clinicians by giving them a clearer understanding of patients’ needs. In addition, interval scores are a prerequisite for parametric statistical analysis, which offers greater power for researchers (Mircioiu & Atkinson, 2017). Second, the CAT–WHODAS 2.0 can predict responses to items not directly administered (Wang & Chen, 2004). This advantage helps identify examinees’ specific problems, prioritize intervention targets, and determine appropriate intervention goals and programs tailored to each person. Third, the Rasch scores are not affected by the examinees’ sex, thereby optimizing the interpretability of the CAT–WHODAS 2.0 scores. With these advantages, the CAT–WHODAS 2.0 can be useful for users in clinical and research settings.
The following limitations are noted. First, the findings were analyzed on the basis of interview data with potential confounding factors addressed by the interviewers. Thus, the results may differ from those collected through self- or proxy reporting, and the influence of confounding factors on our results remains uncertain. Second, the parameters were estimated in people with dementia. Therefore, the findings cannot be generalized to people with or without the other diagnoses, because the item difficulties and Rasch person reliabilities may be inconsistent across different populations. Third, some crucial psychometric properties have yet to be examined, such as test–retest reliability and responsiveness to changes (improvements or deterioration). Thus, the CAT–WHODAS 2.0 scores in repeated assessments should be interpreted cautiously. Fourth, we did not examine the DIF related to other confounding factors, such as marital and socioeconomic status, which might bias the CAT–WHODAS 2.0 scores. Future validations are warranted for the unexamined psychometric properties and the DIF of other confounding factors.
Implications for Occupational Therapy Practice
The results of this study have the following implications for occupational therapy education: ▪ The CAT–WHODAS 2.0 can be a useful tool to provide efficient, reliable, and valid assessments of disability across multiple domains of functioning among people with dementia. ▪ The CAT–WHODAS 2.0 may be used for routine assessment to improve the interpretability of assessment results. It offers interval Rasch scores that directly represent the magnitudes of disability and are suitable for most parametric statistical methods.
Conclusion
Our findings suggest that the CAT–WHODAS 2.0 can provide efficient, reliable, valid, and sex-unbiased assessments for disability levels across six key domains for people with dementia. Accordingly, the CAT–WHODAS 2.0 may be a promising alternative for users in both clinical and research settings, improving the utility and quality of assessments while maximizing their efficiency.
Supplemental Material
Supplementary material for Development of a Computerized Adaptive Testing System of the World Health Organization Disability Assessment Schedule 2.0 (CAT–WHODAS 2.0) for People With Dementia
Supplementary material, sj-pdf-1-aot-10.5014_ajot.2025.050965.pdf for Development of a Computerized Adaptive Testing System of the World Health Organization Disability Assessment Schedule 2.0 (CAT–WHODAS 2.0) for People With Dementia by Shih-Chieh Lee, Yi-Ching Wang, Gong-Hong Lin, Hsin-Yu Chiang, Chih-Wen Twu and Ching-Lin Hsieh in The American Journal of Occupational Therapy
Footnotes
Acknowledgments
The manuscript has been read and approved by all of the authors. The manuscript has not been previously published, nor is it currently under consideration for publication elsewhere. None of the authors have any conflicts of interest to declare. This work was supported by Taiwan National Science and Technology Council (112-2811-B-002 -093 and 112-2314-B-002 -138 -MY3).
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
