Dynamic Lowenstein Occupational Therapy Cognitive Assessment: Evaluation of Potential to Change in Cognitive Performance

Dynamic Assessment

Conventional standardized cognitive tests are static in nature; they examine individual performance in the here and now for the purpose of identifying and quantifying cognitive deficits (Toglia, 2005). Dynamic assessment is an interactive procedure that systematically and objectively measures the degree of change that occurs in response to cues, strategies, feedback, or task conditions that are introduced during testing. In contrast to static assessment, dynamic assessment focuses on individual variations and changes rather than on comparison with normative or typical performance. The goal is to measure how and to what extent performance can improve with guidance (Haywood & Lidz, 2007).

In recent years, we have seen an increased awareness within the health professions of the potential for dynamic assessment techniques to provide professionals with the opportunity to estimate the person’s potential for learning or receptiveness to instruction (Feuerstein & Feuerstein, 1991). Dynamic assessment is based on Vygotsky’s (1978) concept of the zone of proximal development, which refers to the discrepancy between what a child can do independently and what he or she can do with the help and guidance of others (Grigorenko & Sternberg, 1998; Sternberg & Grigorenko, 2002). This concept is somewhat similar to that independently developed by Feuerstein, Rand, Hoffman, and Miller (1980) of the mediated learning experience, in which adults serve as catalysts for learning by modifying the person’s internal arousal and the specific task demands to allow for improved cognitive performance (Feuerstein, 1979; Feuerstein et al., 1980; Hadas-Lidor & Weiss, 2005). In dynamic assessment, mediation is defined as the process whereby the tester or evaluator provides cues that may facilitate the person’s ability to perform the task.

Toglia (1994, 2005) introduced the use of a structured, graded system of cues to the assessment of cognitive and perceptual deficits among adults with cognitive impairments. Following in the footsteps of earlier dynamic cognitive theorists, she believed that the examiner could learn much about underlying information-processing strategies by observing a client’s responses to such cues. In this way, dynamic assessment becomes naturally linked to intervention and can be used as a baseline for choosing and designing an intervention program (Lidz, 1992; Lidz & Elliot, 2000; Toglia, 1994). Toglia and Cermak (2009) demonstrated that in an intervention based on dynamic assessment, clients with unilateral neglect showed significant differences—more left-sided search and reduced neglect—compared with a control group performing an object search task.

In dynamic assessments such as the DLOTCA, a person’s initial incorrect response is not taken as the final product of the evaluation task. Instead, the examiner uses a systematic approach to modify the task, through prompts or other forms of mediation, to understand the type of information that is essential so that the person can best complete the task. This feature, which is unique to dynamic assessments, also enables the examiner to collect information that can be helpful in developing effective remediation strategies (Lidz, 1992). Indeed, the use of such feedback during and after testing demonstrates how dynamic assessment ascribes to the principle basic to the learning-test concept—namely, the assumption that learning can take place even within the actual testing process (Guthke & Beckmann, 2000). Other researchers have applied a process similar to ours in changing the LOTCA to a dynamic assessment to well-known cognitive assessments. For example, Uprichard, Kupshik, Pine, and Fletcher (2009) tested 77 people with acquired brain injury using the Wisconsin Card Sorting Test (Heaton, 1981) using the regular static administration and using a dynamic procedure. The dynamic process scores showed better prediction of community integration than did the static scores. This adaptation of the Wisconsin Card Sorting Test suggests the same underlying assumptions that dynamic measures will be ecologically more valid than static testing.

Original Psychometric Properties of the LOTCA

The LOTCA battery, on which the DLOTCA is based, was originally designed by a team of clinicians from the Lowenstein Rehabilitation Hospital, Ranaana, Israel, in 1974 (Katz et al., 1989; Itzkovich et al., 2000). It was derived from an accumulation of clinical experience as well as from neuropsychological and developmental theories and evaluation procedures (Averbuch & Katz, 2005). The LOTCA battery provides a cognitive profile along six cognitive areas: orientation, visual and spatial perception, praxis, visuomotor organization, and thinking operations. The LOTCA has been standardized, and its reliability and validity in various populations have been researched since 1989 (Averbuch & Katz, 2005, 2011). It was further studied in various countries to explore its universal validity as a culture-free evaluation (Annes, Katz, & Cermak, 1996; Cermak et al., 1995; Jang, Chern, & Lin, 2009; Jang, Hwang, & Li, 1997; Katz, Champagne, & Cermak, 1997; Lieberman et al., 1996). Its ecological validity was also studied in terms of relationships with function that were found to have significant correlations with performance of activities of daily living and instrumental activities of daily living in a stroke population with and without unilateral spatial neglect (Katz, Hartman-Maeir, Ring, & Soroker, 1999, 2000).

The goals of this study were (1) to study the DLOTCA’s psychometric properties, interrater reliability, internal consistency, and construct validity; (2) to determine the level of change from before to after mediation on performance of the DLOTCA subtests, indicating the potential for learning and change; and (3) to examine the most frequent level of mediation used in the different domains for further planning of appropriate methods of intervention.

Method

Results

The mean age of the 83 hospitalized clients following first stroke was 57.7 (SD = 8.33), and that of the 45 healthy control participants was 62.67 (SD = 9.22). Mean years of education was lower for the healthy control participants, but the difference was not statistically significant.

Psychometric Properties of the DLOTCA

Interrater Reliability.

We calculated interrater reliability for three pairs of raters for 10 participants in the different testing locations. Results showed high Spearman correlation coefficients between all pairs of raters for the domains (rs = .90–.98).

Internal Consistency Reliability.

The α coefficient level for each domain was found to be moderately high (Spatial Perception, α = .665; Praxis, α = .602; Visuomotor Construction, α = .813; Thinking Operations, α = .737; and Verbal Mathematical Questions, α = .647), except for the Visual Perception domain (α = .313). The shape identification subtest was found to involve naming difficulties for the second set of the four more difficult items that are highly correlated with years of education. We therefore decided to remove this subtest from the final version (Katz, Livni, et al., 2011).

Construct Validity: Comparison Between Groups.

Using Mann–Whitney U tests, we calculated means and standard deviations for each group’s performance on all subtests before mediation. We found significant differences on most of the subtests before mediation, p < .05, on which healthy control participants performed better than stroke clients.

Means, standard deviations, and quartiles are presented in Table 1 for performance before mediation and were compared with two-sample t tests between healthy control participants and stroke clients on the DLOTCA domains. Significant differences can be seen on the domains of Orientation, Visual and Spatial Perception, and Praxis at p < .05 before mediation and in the Visuomotor Construction and Thinking Operations domains; the difference was not significant because the healthy control participants also had difficulties, probably as a result of the correlation with years of education (presented later). However, time was significant at p < .01, showing that the stroke clients needed much more time to accomplish the tasks (see Table 1). The subgroups who needed mediation improved in both groups, and therefore we found no significant differences after mediation except for Thinking Operations; whereas before mediation, there was no difference, after mediation a significant difference was found (t = 5.441, df = 126, p < .01; see Table 1).

OPEN IN VIEWER

Table 1.

Mean Performance of Healthy Control Participants and Clients After Stroke on DLOTCA Domains Before Mediation and Time by Group With Two-Sample t-Test Analyses

Domain Before Mediation	Healthy Control Participants						Clients After Stroke						t
	n	M	SD	Quartile (%)			n	M	SD	Quartile (%)
				25	50	75				25	50	75
Orientation (0–2)	45	1.98	0.05	2.00	2.00	2.00	83	1.91	0.23	1.88	2.00	2.00	2.707**
Visual Perception (1–4)	45	3.96	0.10	3.75	3.75	4.00	83	3.83	.240	3.50	3.75	4.00	2.059*
Spatial Perception (0–1)	45	0.95	0.08	0.92	1.00	1.00	83	0.91	0.13	0.83	1.00	1.00	2.210*
Praxis (0–2)	45	1.74	0.23	1.67	1.83	1.92	83	1.83	0.17	1.75	1.92	2.00	−2.578*
Visuomotor Construction (1–5)	45	4.21	0.76	3.57	4.43	4.86	83	3.96	0.84	3.57	4.14	4.57	1.614
Visuomotor Construction (time)	45	57.24	26.48	31.29	55.86	78.00	81	77.26	31.35	60.29	74.14	87.79	−3.622**
Thinking Operations (1–5)	45	3.85	0.71	3.33	4.00	4.50	83	3.78	0.80	3.17	3.83	4.50	0.515
Thinking Operations (1–5) after mediation	44	4.58	0.36	4.33	4.60	5.00	80	4.02	0.80	3.70	4.00	4.50	5.441**
Thinking Operations (time)	45	68.44	21.00	53.75	65.00	78.33	81	81.81	31.78	63.66	76.50	91.42	−2.530**
Verbal Mathematical Questions (0–1)	45	0.66	0.36	0.37	0.75	1.00	82	0.70	0.27	0.50	0.75	1.00	−0.550

*

p < .05. ** p < .01.

The quartile data provide the scoring for the 25%, 50%, and 75% quartiles. The 25% score means that a lower score should be considered as low performance that indicates possible deficit and should be further tested. Regarding Thinking Operations, it can be seen in the quartile columns that both groups improved, but the healthy control group received scores of 4.33, 4.60, and 5.00 for the 25%, 50%, and 75% quartiles, respectively, whereas the stroke client group received lower scores (3.70, 4.00, and 4.50; see Table 1).

In addition, we performed Pearson correlation analysis on the DLOTCA domain scores before mediation to study the relationship of years of education with performance because the healthy control group had a wide range of years of education. For the healthy control group before mediation, we found moderate to high significant correlations ranging from .295 (p < .05) to .699 (p < .001); however, in the stroke client group, the significant correlations were mild to moderate, ranging from .203 (p < .05) to .449 (p < .01), probably as a result of the acquired brain injury that overshadowed the difference in education.

Performance in the Stroke Client Group Before and After Mediation.

Comparing the scores of the stroke client group before and after mediation for those who needed mediation revealed significantly high differences on all domains at ps < .01 and .001 (Table 2). Moreover, effect sizes were moderate to high in all domains, ranging from 0.35 for Thinking operations to 2.11 for visual Perception (Figure 1).

OPEN IN VIEWER

Table 2.

DLOTCA Domain Means Before and After Mediation With Paired t Tests Within the Stroke Client Group

Domain and Mediation	M	SD	Mean Difference	Paired t Test
Visual Perception (n = 24)			0.361	9.865***
Before	3.60	0.18
After	3.97	0.17
Spatial Perception (n = 15)			0.170	9.735***
Before	0.81	0.13
After	0.98	0.09
Praxis (n = 39)			0.111	2.676**
Before	1.76	0.16
After	1.87	0.32
Visuomotor Construction (n = 36)			0.520	7.861***
Before	3.83	0.80
After	4.35	0.68
Thinking Operations (n = 44)			0.278	3.025**
Before	3.74	0.78
After	4.02	0.80
Verbal Mathematical Questions (n = 25)			0.199	3.806***
Before	0.58	0.22
After	0.78	0.36

Note. n = only those who needed mediation. M = mean; SD = standard deviation.

**

p < .01. *** p < .001.

OPEN IN VIEWER

Figure 1.

Effect sizes for the DLOTCA domains in the stroke client group.

Frequency of Level of Mediation for Each Domain.

The percentages of participants who needed mediation at each level of each domain are presented in Figure 2. As can be seen, stroke clients needed more mediation in the Visuomotor Construction and Thinking Operations domains. They needed higher levels of mediation in specific feedback (Level 3, specific feedback), structured category (Level 4, partial intervention), and reduced amount (Level 5, copying or subtracting amount; see Figure 2).

OPEN IN VIEWER

Figure 2.

Frequency of levels of mediation in the DLOTCA domains in the stroke client group.

Discussion

Our findings support the use of the DLOTCA. We found high interrater reliability and good internal consistency reliability for almost all domains. The results partially support construct validity because the test significantly differentiated between clients after stroke and healthy control participants in the basic domains (Orientation, Visual and Spatial Perception, Praxis), but the higher cognitive domains (Visuomotor Construction and Thinking Operations) showed only a trend in favor of the healthy control participants.

Moreover, we found significant differences in the stroke client group before and after mediation, suggesting potential change and providing an indication of the level of mediation from which the person would benefit in intervention. These findings strengthen the original LOTCA’s psychometric properties (Averbuch & Katz, 2005) and further substantiate the dynamic procedure, which is a fundamental change and addition to the evaluation. Thus, the assessment provides the level of cognitive performance along the different domains as well as the potential for change for each individual and the starting point for intervention.

The levels of mediation used by the stroke clients suggest that for the domains of Perception and Praxis, Levels 1 and 2 (general intervention and general feedback) were enough for most participants to reach best performance. In the Visuomotor Construction and Thinking Operations domains, Levels 3 and 4 (specific feedback and partial intervention) were frequently needed, and in some cases, Level 5 (reduced amount) was needed, especially in the verbal mathematical questions (see Figure 2). These results support the hierarchy of cognitive skills that were the basis for the LOTCA development (Katz et al., 1989) and further support the notion that the higher the cognitive demands are, the more mediation is required to change and learn.

These findings regarding the DLOTCA assessment are in line with those of previous studies and support the premise of a new method of evaluation that leads directly to intervention (Feuerstein et al., 1980; Hadas-Lidor & Weiss, 2005; Lidz, 1992; Lidz & Elliot, 2000; Toglia, 2005; Toglia & Cermak, 2009; Sternberg & Grigorenko, 2002; Uprichard et al., 2009). Only a few dynamic assessments are available in occupational therapy (Katz et al., 2007; Toglia, 1994), and some, such as the Executive Function Performance Test (Baum, Morrison, Hahn, & Edwards, 2003; Baum et al., 2008) and the Cognitive Performance Test (Burns, 2006; Levy & Burns, 2005), have built in a cueing system for scoring. All of them assess cognitive abilities. This new approach to evaluation is of utmost importance because it goes beyond the basic evaluation of abilities to understanding the learning potential and possible change that may occur if a person is to be approached in the right way and at the best possible level. Dynamic assessment, combined with functional evaluation, will provide occupational therapists with the best data for planning interventions and helping people actualize their desired goals.

Implications for Occupational Therapy Practice

The DLOTCA profile provides guidelines at three stages:

In addition, response time, or length of performance time, describes whether the person is able to perform the task over a longer time. The intervention approach will differ on the basis of response time.

Limitations and Future Research

We assessed interrater reliability on only 10 participants, and the number could be larger. The sample size is small for test standards, so the results have to be regarded as first indications. Healthy control participants had, on average, fewer years of education, a variable that has been found to be significantly correlated with most of the domains. Therefore, this group is not an ideal control group. The client population included only people after stroke; thus, the data cannot be generalized to other neurological groups.

Future research should use a larger sample size; collect data from additional groups, such as people with traumatic brain injury, Parkinson’s disease, multiple sclerosis, and schizophrenia; translate and adapt the DLOTCA for use in other countries and languages; and collect data on and compare the DLOTCA across cultures. Beginning work is already underway in Brazil and Chile; namely, the DLOTCA has been translated into Portuguese and Spanish, and data collection has begun. The DLOTCA has also been translated into Arabic, and data have been collected on clients with stroke and healthy control participants among the Arab population in Israel (Abdelqader, 2010). Results have shown findings similar to the ones reported here. The next major step in validation should include determining the DLOTCA’s predictive or ecological validity for daily functioning and participation in the community.