Abstract
Schizophrenia is characterized by impairments in a wide range of cognitive domains, including speed of processing, attention, memory, executive function, and social cognition (Nuechterlein et al., 2004). In both cross-sectional and longitudinal analyses, these cognitive deficits, either individually or in combination, have been shown to have a negative effect on multiple aspects of functional outcome, such as work performance and maintenance, instrumental activities of daily living (IADLs), skill acquisition, and social problem solving (Green, 1996, 2006; Green, Kern, & Heaton, 2004; Lipkovich et al., 2009). Thus, it is important for clinicians to screen people with schizophrenia for cognitive problems and to select an appropriate intervention accordingly.
The Allen Cognitive Level Screen (ACLS; Allen, 1990, 1996) has been used extensively by occupational therapists across cultures and settings as a quick screening tool to derive a view of cognitive function in people with psychiatric or geriatric disabilities (Lindstedt, Ivarsson, & Söderlund, 2006; McAnanama, Rogosin-Rose, Scott, Joffe, & Kelner, 1999; Yeung & Chan, 2006). The ACLS requires the person to perform three leather-stitching tasks of increasing difficulty and yields a single score that can be used to determine what type of environment or social support is indicated to facilitate the person’s ability to function (Allen, Earhart, & Blue, 1992). Various versions of the ACLS have demonstrated high interrater reliability (rs = .91–.98; Howell, 1993; Penny, Musser, & North, 1995) and satisfactory test–retest reliability (r = .75; Newman, 1987). Performance on the ACLS has been related to the ability to carry out basic activities of daily living (ADLs; Keller & Hayes, 1998; Moore, 1978); social competence (Penny et al., 1995); many elements of cognition, including processing speed (David & Riley, 1990); short-term verbal memory (Velligan et al., 1998); perceptual organization (Velligan et al., 1998); executive function (Secrest, Wood, & Tapp, 2000); and fluid abilities reflected in the performance subtests of the Wechsler Adult Intelligence Scale–Revised (Mayer, 1988; Wechsler, 1981). The ACLS has not been significantly correlated with age, sex, and education (Moore, 1978; Penny et al., 1995). Apart from those variables, the ACLS is also a useful predictor of community functioning in people with psychiatric disabilities (Henry, Moore, Quinlivan, & Triggs, 1998; Velligan et al., 1998).
The ACLS scores are mapped to Allen’s (1985) Model of Cognitive Disabilities, which contains a six-level cognitive scale ranging from severe disability (Level 1) to normal ability (Level 6) as well as corresponding functional capacities or limitations. A decimal system was later added to the original six levels to form modes of performance within the cognitive levels to increase sensitivity in measuring change (Allen et al., 1992).
This taxonomy of cognitive disability is based on extensive clinical observation and information processing theory. In brief, people functioning at Level 1 (automatic actions) and Level 2 (postural actions) attend primarily to their internal self and thus need total care. At Level 3 (manual actions), people demonstrate perseveration, generalized disorientation, and unreliable episodic memory. Attention shifts from the internal self to the external environment but is restricted to tactile cues. At Level 4 (goal-directed actions), people’s motor actions are spontaneous and goal directed, but they rely on visual cues. Tasks that require learning cannot be done independently. Attention is restricted to one cue at a time, and episodic memory remains disorganized. People at Level 5 (exploratory actions) can attend to related cues and learn new tasks through inductive reasoning. Deficits in orientation and memory are much less severe than at Level 6. Level 6 (planned actions) was designed to describe the absence of disability. At this level, symbolic cues are used to formulate plans, and deductive reasoning is applied to aid in the selection of alternative courses of action. Attention is directed toward anticipating errors and planning modes of action to prevent mistakes.
Allen (1985) delineated the cognitive abilities at each level in behavioral terms. However, the quantitative pattern of cognitive functioning that characterizes each level is little understood. By clarifying which cognitive abilities are impaired or preserved, clinicians may be better able to provide targeted intervention for people functioning at different levels. Another unresolved issue pertains to the validity of Allen’s cognitive hierarchy model. Although Herzig (1978) found that cognitive level attained a higher predictive value as a dichotomous variable (one consisting of Levels 2–4, the other consisting of Levels 5 and 6) than as a continuous variable on community adjustment, the hierarchical relationships among Allen Cognitive Levels have not been validated against objective measures of cognitive function. A valid hierarchy not only would serve as a scale of the amount of cognitive assistance needed but also would be important in planning, therapeutic care, and policymaking.
We sought to explore the aforementioned issues with people diagnosed with schizophrenia classified as Levels 4 and 5. We chose these levels for two reasons. First, people at Levels 4 and 5 are presumed to follow test directions better than those at lower levels, thereby ensuring valid test results. Second, the difference in functioning between Levels 4 and 5 appears to be greater than that between Levels 3 and 4 or between Levels 5 and 6 (Allen, 1985; Herzig, 1978). Thus, we undertook this study to quantify the profile of cognitive impairments in people with schizophrenia rated at Levels 4 and 5 and to compare performance on tests of cognition between two levels. The cognitive functions of interest included processing speed, memory, and executive function because those functions are not only the most commonly impaired in schizophrenia but also the most consistent predictors of functional outcome (Green, Kern, Braff, & Mintz, 2000; Ojeda, Peña, Sánchez, Elizagárate, & Ezcurra, 2008).
Method
Research Design
This cross-sectional correlational study was approved by the Kaohsiung Medical University hospital’s institutional review board. All participants gave written informed consent before their inclusion in the study.
Participants
Seventy-six participants were recruited from an outpatient clinic of the Department of Psychiatry at the Kaohsiung Medical University Hospital in Taiwan. They met Diagnostic and Statistical Manual of Mental Disorders (DSM–IV; American Psychiatric Association, 1994) criteria for schizophrenia or schizoaffective disorder, which was confirmed by attending psychiatrists using the Structured Clinical Interview for DSM–IV (SCID; First, Spitzer, Gibbon, & Williams, 1996). Participants had ACLS scores between 4.0 and 5.8 and were between 18 and 65 yr old. Participants were excluded if they showed evidence of current substance abuse, mental retardation, or history of neurological illness. Participants with significant changes in clinical state or in drug treatment during the preceding 3 mo were also excluded. All participants were receiving antipsychotic medications at the time of the study. Daily doses of antipsychotics were converted to approximate daily chlorpromazine equivalents using published guidelines (Atkins, Burgess, Bottomley, & Riccio, 1997; Woods, 2003). Participants were divided into two groups according to their ACLS scores: participants who scored 4.0–4.8 (n = 35) and those who scored 5.0–5.8 (n = 41).
Instruments
The ACLS is an individualized criterion-referenced tool designed to assess the cognitive level of people with psychiatric illness or cognitive impairment. It requires the person to replicate three different stitches with leather-lacing materials. Participants are told to watch and copy what the examiner does. The examiner makes several mistakes on purpose and asks them whether they can point out the mistake and then whether they can fix the mistake. For the last stitch, participants are asked whether they can figure out how to make the stitch from a completed model. The ACLS yields a global score based on error frequency and type, ability to recognize and correct errors, and the level of assistance demanded. Scores range from 3.0 to 5.8, with higher scores indicating higher cognitive levels. Information obtained from the test includes the person’s ability to follow directions, solve problems, and learn new information in novel situations.
To ensure accuracy in administration and scoring, the ACLS was translated into Chinese by P.-C. Tsai, a master’s-level occupational therapist specializing in psychiatric rehabilitation. The first-draft translation was then reviewed by W.-L. Su, who has extensive experience in teaching Allen’s Model of Cognitive Disabilities and administering the ACLS. After minor corrections to wording, the final draft was used in the data collection.
To determine interrater reliability of the Chinese-translated ACLS, 3 people with schizophrenia were evaluated by 1 rater (P.-C. Tsai) on an individual basis, whereas the other rater (W.-L. Su) observed testing sessions. The two raters were blinded to each other’s scores. The interrater reliability, as measured by an intraclass correlation coefficient, was .98. Concurrent validity was examined by assessing the relationship among the ACLS, global cognitive function, and executive function in 44 people with schizophrenia (28 men and 16 women; mean [M] age = 40.27 yr, standard deviation [SD] = 9.46; M education = 13 yr, SD = 2.92), independent of the current study sample. The Chinese version was found to correlate with perseverative errors (r = −.32, p = .04) on the Wisconsin Card Sorting Test (WCST; Heaton, Chelune, & Talley, 1993) and total Mini-Mental State Examination score (r = .25, p = .03; Folstein, Folstein, & McHugh, 1975). The psychometric properties of this version are similar to those of the original English version (Allen, 1996) and other Chinese or Cantonese versions (Chan, Ray, & Trudeau, 2001; Wu, Lin, Liu, & Chiu, 2007).
Chu’s Attention and Processing Speed Test (CAPST; Chu, 1997) is routinely used by occupational therapists in Taiwan to assess processing speed in patients with psychiatric disabilities. It consists of one test sheet that contains 100 items arranged in 33 rows with 3 items per row. The first item is located separately above the rows and is preceded by a printed test title. All items are arranged in numerical sequence from left to right and top to bottom. Each item begins with a two-digit number (00–99) and is followed by a string of 14 symbols and letters, some of which are asterisks. The response sheet consists of seven columns containing, in all, 200 small blank squares, each paired with a randomly assigned two-digit number. The task is to fill in the empty spaces with the number of asterisks in each corresponding item on the test sheet as quickly as possible. Eight practice trials were administered before beginning the task, as specified in the manual. The total score is the number of squares filled in correctly in 10 min. Test–retest reliability is high (r = .84). The CAPST is correlated moderately with work performance (r = .44) and ADLs (r = .62) in patients with schizophrenia (Chu, 1997).
The Chinese version of the Wechsler Memory Scale–Third Edition (Wechsler, 1997) contains 10 subtests (Logical Memory I and II, Faces I and II, Verbal Paired Associates I and II, Family Pictures I and II, Letter–Number Sequencing, and Spatial Span). These subtests were developed to provide summary measures thought to reflect specific aspects of episodic memory, including verbal immediate and delayed, visual immediate and delayed, and working memory. Specifically, verbal memory indexes consist of the Logical Memory and Verbal Paired Associates subtests, and the visual memory indexes consist of the Faces and Family Pictures subtests. The working memory index consists of one auditory (Letter–Number Sequencing) and one visual (Spatial Span) task.
The computerized version of the WCST (Tien et al., 1996) was administered using instructions translated into Chinese. The computerized card-sorting task was found to be as effective at eliciting knowledge as a manual card sort (Tien et al., 1996). Participants did not need previous training with computers for the on-screen administration of the test. The participants paid careful attention to the screen and responded by pressing one of four keyboard keys (coded 1, 2, 3, and 4 on the top of the keyboard) corresponding to the sequence of the four principal stimulus cards. The test required participants to match cards by color, shape, and number to four stimulus cards that appeared from left to right in the lower part of the screen. Participants had to determine the correct category according to the computer-generated verbal feedback, “right” or “wrong.” Once 10 consecutive cards were categorized correctly, the sorting principle changed without warning or comment, and all sorts made according to the previous strategy received negative feedback. The test results were scored by a computer software program. Perseverative errors and conceptual-level responses were included in the analysis. When a participant persisted in responding to a stimulus characteristic that was incorrect, the response was scored as perseverative. Conceptual-level responses reflected insight into the correct sorting principles.
The Positive and Negative Syndrome Scale (PANSS; Kay, Fiszbein, & Opler, 1987) assessed the presence or absence and severity of positive symptoms (7 items), negative symptoms (7 items), and general psychopathology (16 items) of schizophrenia. The psychotic symptoms were rated on the basis of objective criteria according to a scale ranging from 1 (no evidence) to 7 (extreme). On the PANSS, a higher score signifies a worse psychopathological profile.
Procedures and Data Collection
Participants’ demographic and clinical data were collected first, followed by administration of the ACLS and other cognitive tests. All cognitive measures were administered according to standard procedures in a single session lasting approximately 3 hr. Breaks were given, when appropriate, to minimize the effects of fatigue on performance. PANSS ratings were completed in a separate session in the same week as the cognitive assessment.
All cognitive measures were administered by the trained examiner (P.-C. Tsai), an occupational therapist with >10 yr of clinical experience with this population. To reduce experimenter bias, the participants were first administered the tests of memory, processing speed, and executive function and then the ACLS. PANSS ratings were completed by a board-certified psychiatrist (T.-C. Tang) who was blinded to participants’ ACLS scores.
Data Analysis
We performed statistical analyses using SPSS Version 15.0 for Windows (SPSS, Inc., Chicago). To facilitate comparisons between the cognitive variables, all standard and raw scores were converted into z scores (M = 0, SD = 1) using normative data reported for healthy samples. Specifically, each individual CAPST raw score was transformed into z score to denote the relative performance in speed of processing using normative data (M = 107.40, SD = 25.70) reported in the manual (Chu, 1997).
For Wechsler Memory Scale–Third Edition indexes and WCST perseverative errors and conceptual-level responses, each standard score was subtracted from 100 (test M) and divided by 15 (test SD) to generate a z score. To examine group differences in cognitive performance, we conducted a one-way multivariate analysis of covariance (MANCOVA) with negative symptom score and gender as covariates and eight cognitive test scores as the dependent variables. Negative symptoms and gender were treated as covariates because these two variables have shown varying degrees of association with cognition in schizophrenia (Brébion, Gorman, Amador, Malaspina, & Sharif, 2002; Goldstein et al., 1998).
To control for Type I error in these analyses, the α level for each test score was set by dividing the .05 level by the eight comparisons conducted (0.05/8 = 0.006; Abdi, 2007). Last, we carried out a discriminant function analysis with a direct entry to identify the cognitive measures that best discriminated between the Level 4 and 5 Level groups. We included in the discriminant analysis only cognitive test scores that showed statistically significant differences between the two groups during MANCOVA.
Results
Of the 102 participants who met inclusion and exclusion criteria, 26 did not complete the entire cognitive assessment for various reasons. They were therefore deleted from the analyses, leaving a sample of 76 participants. These 76 participants were classified into two groups according to ACLS scores: Level 4 (n = 35) and Level 5 (n = 41). Demographic and clinical features did not differ significantly between the groups, with the exception of gender distribution and negative symptoms (Table 1). The Level 5 group had more women, and the Level 4 group fared significantly worse on negative symptoms.
Clinical and Demographic Characteristics of the Sample
Note. Statistical analysis was performed with the t test for continuous variables and with the χ2 test for categorical variables. ACLS = Allen Cognitive Level Screen; M = mean; PANSS = Positive and Negative Syndrome Scale; SD = standard deviation.
Before conducting the MANCOVA analysis, we examined the interaction between the covariates and the grouping variable to assess the assumption of homogeneity of regression slopes. The result yielded no significant Group × Negative Symptoms (Wilks’ λ = .70, F[16, 128] = 1.53, p = .10, 
Cognitive Test Results in Allen Cognitive Level 4 and 5 Groups
Note. Values are means adjusted for gender and negative symptoms. M = mean; SD = standard deviation.
The univariate F tests were significant at p ≤ .006.
As shown in Figure 1, the Level 5 group fell in the range between the normative mean and 1 SD below the mean across all cognitive measures, and the Level 4 group scored between 1 and 2 SD below the normative mean on the same measures.
Cognitive test performance of participants at Levels 4 and 5, standardized according to the normative data.
Discriminant analysis yielded a statistically significant function (Wilks’ λ = .67, χ2 [8, N = 76] = 29.41, p < .0001). Cognitive test predictors as a whole were able to correctly classify 77.6% of the participants as either Level 4 or Level 5. The standardized canonical discriminant function coefficients reflect the relative importance of the independent variables in differentiating between the two groups of interest. The larger the coefficient is, the more the particular variable discriminates between the groups. In this discriminant model, delayed verbal recall was the best discriminator between the groups (standardized discriminant coefficients = 1.26), followed by immediate verbal recall (−0.77), processing speed (0.44), and working memory (0.26). The canonical correlation coefficient of .58 indicates that 33.5% of the variance in decision on cognitive level was accounted for by these cognitive measures. Group membership was correctly predicted for 71.4% of participants at Level 4 and 82.9% of participants at Level 5.
Discussion
To our knowledge, this study is the first to describe in detail and compare the cognitive features of people with schizophrenia at Allen Cognitive Levels 4 and 5. Two main findings emerged from this analysis: (1) Both groups scored below the normative mean across cognitive domains, with participants at Level 5 performing significantly better than those at Level 4 on processing speed, verbal memory, and working memory, and (2) the cognitive variables were able to predict an overall correct classification rate of 78%.
Our results not only lend support for Allen’s (1985) clinical impressions of the qualitative differences between these two levels but also extend previous work (Herzig, 1978) on the validity of cognitive hierarchy by showing that on a global level, the cognitive function of participants at Level 4 was significantly inferior to that of participants at Level 5: Group means were <1 SD apart. In particular, the Level 5 group demonstrated a mild cognitive deficit, as evidenced by the slightly below-average performance across all domains of cognitive function except processing speed. Future studies are necessary to extend these findings by including people with schizophrenia at Levels 3 and 6 to provide further support for the hierarchical nature of Allen’s Cognitive Disabilities Model.
The cognitive difference between the two groups highlights the importance of verbal memory, working memory, and processing speed in achieving Level 5 performance criteria. That is, the person at Level 5 is better able to adopt efficient encoding strategies that facilitate the retrieval of verbal directions provided along with visual demonstration. Likewise, higher ACLS scores require both working memory, which involves the ability to remember multistep stitch sequences so as to anticipate and guide performance, and an information processing speed that entails the efficiency to execute all relevant mental operations (e.g., encoding and retrieval, response selection and execution) within a specified time limit (Bachman et al., 2010; Salthouse, 1996). However, given that visual memory, ability to inhibit inappropriate responses, and problem solving were also implicated as the cognitive processes underlying the ACL (Secrest et al., 2000; Velligan et al., 1998), we were surprised to find no significant differences between the two samples in those cognitive abilities. Nevertheless, the Level 5 group tended to be more efficient in encoding and retrieving visually presented stimuli, to make fewer perseverative errors, and to show higher conceptual-level responses than the Level 4 group, as evidenced by their higher mean scores on these measures. The lack of significant difference may be ascribed to the underrepresentation of lower functioning people at Levels 4.0–4.4, which may, in turn, artificially inflate the cognitive performance of the Level 4 group. Replicating this study with an equal number of participants in each performance mode within Levels 4 and 5 is warranted to illuminate the role of visual memory and executive function in ACL.
Tests of processing speed, verbal episodic memory, and working memory were able to account for 34% of the variance in the determination of cognitive levels. The proportion of variance explained might be higher if other cognitive domains associated with ACL scores, such as attention and perceptual organization, were entered as the potential discriminators (Velligan et al., 1998). This possibility would be interesting to explore in future research.
These findings should be interpreted within the context of the study’s limitations. Our sample might not represent all people with schizophrenia at Levels 4 and 5 because of unequal distribution of participants across the 10 modes of performance embedded in the two levels. For instance, no participants were in Mode 4.0, but more than half of the Level 4 participants were in Mode 4.8 (Table 1). In other words, our Level 4 group may function at the higher end of this level’s ability continuum. Our Level 5 group, however, may function at the lower to middle end of the ability continuum of this level, because only 14% of the participants with schizophrenia at Level 5 were classified into Modes 5.6 and 5.8. To establish empirical support for the mode-related differences in treatment methods and extent of cognitive assistance (Allen et al., 1992), validation of the hierarchical arrangement of performance modes in people with psychiatric disabilities awaits further investigation.
Regardless of these limitations, our results shed light on the magnitude and pattern of performance on measures of specific cognitive abilities associated with Levels 4 and 5. Clinicians may incorporate these findings into the formulation of intervention strategies that aim to either enhance cognitive functioning through specific training exercises or bypass these deficits through the use of compensatory approaches. Taking compensatory intervention as an example, allowing sufficient time for the Level 4 person with impaired processing speed to comprehend the instructions given during therapeutic activities is plausible, whereas verbal memory deficit can be compensated for with the use of written materials or frequent cues. Last, the results allow occupational therapists to interpret ACLS scores in measurable and quantifiable terms when communicating with other professionals.
In conclusion, people at Allen Cognitive Level 5 significantly outperformed those at Level 4 on tests of processing speed, verbal memory, and working memory. An in-depth understanding of their patterns of cognitive impairment may facilitate the modification of task environment by capitalizing on their strengths and minimizing their weaknesses. More efforts should be made to provide a comprehensive profile of cognitive function for these two levels and identify which cognitive deficits adversely affect their community functioning.