Validation of the Spanish version of the Oxford Cognitive Screen (S-OCS): psychometric properties of a short cognitive stroke-specific screening tool

Abstract

Objective:

The aim of this study is to examine the psychometric properties of the Spanish version of the Oxford Cognitive Screen in a subacute stroke population.

Participants and setting:

Fifty-seven subacute stroke outpatients and 54 healthy individuals were recruited in Alicante province, Spain.

Main measure:

The Oxford Cognitive Screen.

Other measures:

The Montreal Cognitive Assessment, the Barcelona test, and the Barthel Index.

Design:

A validation study was conducted to analyze the inter-rater, intra-rater, test–retest, and internal consistency of the Spanish version of the Oxford Cognitive Screen. Concurrent validity was assessed using the Montreal Cognitive Assessment and the Barcelona test, and divergent validity using the Barthel index. Discriminant indices such as the sensitivity, specificity, predictive values, and optimal cut-offs were also estimated.

Results:

The subtests of the Spanish version of the Oxford Cognitive Screen showed excellent estimates for the inter-reliability (intraclass correlation coefficient (ICC) = 0.790 to 1.000; r_s = 0.693 to 1.000), and acceptable-good for intra-reliability (ICC = 0.181 to 0.990) and test–retest reliability (r_s = 0.173 to 0.971). Internal consistency was also excellent (standardized Cronbach’s α = 0.907). Spearman correlations for the concurrent validity were low-strong (r_s = −0.193 to 0.95) and low-moderate (r_s = −0.091 to 0.443) for divergent validity. The optimal cut-offs estimated for the subtests of the Spanish version of the Oxford Cognitive Screen showed good-high specificity (66.7%–100%) and positive predictive value (67.9%–100%), and low-good sensitivity (14.8%–83.3%) and moderate-good negative predictive value (53.5%–76.6%). Discriminant power as measured by the area under the curve indicated acceptable-good values (0.397 to 0.894).

Conclusion:

Our findings support that the Spanish version of the Oxford Cognitive Screen is a reliable and valid tool for screening cognitive impairments in subacute stroke patients.

Keywords

Stroke psychometric properties validation assessment Oxford Cognitive Screen

Introduction

Cognitive impairment is a very common symptom in the acute stage after stroke (i.e. within seven days post-stroke)¹ with prevalence rates ranging up to 70%–80%;^2,3 however, it is often neglected or poorly treated unlike other neurological disorders such as sensory or motor dysfunctions.^4,5 Importantly, the identification of cognitive impairment may provide valuable information to clinicians for an early cognitive recovery and a prevention of long-term adverse outcomes concerning disability, independence, and quality of life.⁵ Moreover, although some improvement or stabilization of cognition are expected in stroke survivors after the acute and early subacute phases (i.e. within seven days and three months post-stroke)¹, cognitive impairment remains considerably prevalent even in later phases. In fact, several recent studies have indicated that cognitive deficits are also frequent symptoms in stroke patients in a late subacute phase (i.e. within three to six months post-stroke)¹ with apparently successful clinical recovery and no functional disability,⁶ as well as in patients in chronic stage⁷ (i.e. after six months post-stroke),¹ with rates ranging from 38% to 73% of the cases.⁸

The detection of and subsequent monitoring of cognitive losses in stroke survivors are usually disregarded because these patients are remaining stable and are being able to preserve some degree of autonomy, despite poor cognitive recuperation.⁹ In this sense, since the evolution of the cognitive dysfunction after stroke may be influenced by several factors involving different adverse consequences, regular cognitive assessments over longer periods of time should be clearly warranted.¹⁰

The mostly used cognitive screening tests do not assess common manifestations of stroke such as aphasia, visual loss, spatial neglect, apraxia, and reading or writing problems. To the best of our knowledge, the Oxford Cognitive Screen is a tool originally created to measure deficits that normally occur after stroke and targeted at identifying these problems. It includes clinical reporting at the level of different cognitive domains, thereby allowing the affected domains to be addressed by a specific treatment,¹¹ suggesting a clear advantage over other screening tests. Therefore, for this study, we aimed to examine the psychometric properties of the Spanish version of the Oxford Cognitive Screen (S-OCS) for use in subacute stroke outpatients.

Methods

This study was approved by the ethics committee of the University San Juan Hospital (reference no. 15/317), and all participants provided written informed consent. This validation study is part of a previous study aimed to translate and adapt the original Oxford Cognitive Screen into Spanish and carried out between January 2015 and May 2016.

The Oxford Cognitive Screen

The Oxford Cognitive Screen is a short neuropsychological battery specifically designed to measure common impairments affecting stroke survivors, while attempting to avoid the confounding effects mediated by such conditions as aphasia or neglect. This test assesses five general cognitive domains, including language, attention and executive function, numeracy, praxis, and memory, and consists of 10 tasks or subtests: picture naming, semantics, orientation, visual field, sentence reading, number writing and calculation, hearts test, gestural imitation, verbal recall and verbal recognition, and executive tasks. Moreover, in order to control the overall influence of the upper limb motor deficits fairly frequent after stroke, all the items can be performed using only one hand. This battery can be easily administered at bedside within 15 minutes, providing a general outline as “visual snapshot” of the patient’s cognitive profile (Supplemental Figure S1), which allows health professionals to readily make a domain-level interpretation.

The Oxford Cognitive Screen consists of a set of several materials: (1) a test booklet containing different tasks, which can be reused for all participants; (2) a participant booklet including tasks for the participant to complete; (3) an examiner’s sheet to note the responses of each participant; and (4) an examiner’s manual including the instructions on test administration and a reference guide to score.

To culturally adapt and translate the Oxford Cognitive Screen into Spanish, we obtained the necessary permission from the author of the original scale, the late Glyn W. Humphreys. The cross-cultural adaptation was made following a thorough forward-translation/back-translation methodology.¹² Two native English translators carried out independently the forward translations of the test. The resulting two versions were reviewed and compared with the English original version by a committee made up of the research team and the authors of the original test. After introducing few semantic and idiomatic changes, and conceptual adaptations, a new version of the test was finally accepted. A back-translation of this version into English was made by other two individual translators blinded to the original version of the test. Semantic and grammatical equivalence was checked by comparing the new English version to the original version. An expert committee (two physiatrists, one neurologist, one medical doctor, one speech therapist, and one occupational therapist) reviewed thoroughly the process of translation and adaptation of the test and agreed a first pre-final S-OCS version with few changes. Briefly, the instructions for the examiner were slightly modified by including a short explanation about the correct order of the materials and the procedure for scoring. In addition, a more extended explanation of each item was given in order to standardize the administration of the test. Discriminant capacity of the words used in the sentence-reading task to identify neglect or surface dyslexia was also discussed with the authors of the original test (e.g. a proposed word “areoplano” (aeroplane) was changed to “galeón” (galleon)).

This pre-final version was pilot-tested in a sample of 25 healthy participants with similar sociodemographic characteristics to the target population to detect possible difficulties in the comprehension of the items. The content of all the items was carefully reviewed by the research team and discussed with the participants to ensure the items reflected an adequate level of cultural understanding. Moreover, and before approving the definitive version for further psychometric testing, this version of the test was completed by 25 stroke outpatients in subacute phase (i.e. within two weeks and six months post-stroke) to verify that the Spanish version could be understood and that the items measured what they were intended to measure.

The S-OCS will be available online at http://www.ocs-test.org, or contact Dr María García-Manzanares (mardealacant@gmail.com).

Participants

A convenience consecutive sample of stroke outpatients were recruited from three different hospitals in Alicante province, Spain: the acquired brain injury unit at the San Vicente Hospital and the rehabilitation unit at University San Juan Hospital and Hospital La Pedrera. Patients enrolled should be within a period between two weeks and six months after stroke onset and be able to have sustained attention for at least 15 minutes. The stroke was diagnosed by neurologists according to the Official Guidelines for the diagnostic and management of stroke by the Spanish Neurology Association.¹³ Clinical information about main diagnosis was obtained from medical reports.

Control group was made up of neurologically healthy participants enlisted among relatives of stroke patients and elderly people from “Aulas de la Experiencia” (Third Age education program) at Miguel Hernandez University. All participants were recruited according to the following inclusion criteria: aged between 18 and 88 years, native Spanish speakers, and without medical conditions such as neurological disease history (e.g. previous stroke, transient ischemic attack, or epilepsy), alcohol or drug abuse, and serious psychiatric illness (e.g. major depressive disorder or schizophrenia). Information on sociodemographic features of all participants was also collected at the time of the neurocognitive assessment: age, gender, marital status, education level, and employment status.

Neuropsychological tests used for the validation

In addition to the S-OCS, the stroke outpatients completed other neuropsychological tests during the initial assessment for the validation purposes. Based on the former validation studies,^11,14 the scores of each subtest of the S-OCS were validated against the results obtained from other chosen subtests assessing the same neuropsychological function. For the concurrent validity, we used the validated Spanish versions of the Montreal Cognitive Assessment¹⁵ and the Barcelona test.¹⁶ Moreover, functional independence of activities of daily living as measured by the Barthel index, adapted and validated in Spanish population,¹⁷ was used to assess the divergent validity.

Procedure

Data collection was carried out by four research assistants trained in the administration of the measures. The S-OCS and all the other neuropsychological tests were administered in a one-on-one session by only one research assistant. The S-OCS was completed by the stroke outpatients and healthy participants during the initial assessment. On the same occasion, only the stroke outpatients completed the other neuropsychological measures to assess concurrent and divergent validity. Moreover, at this first evaluation, two research assistants interviewed a subsample of these participants simultaneously and scored independently to determine inter-rater reliability. Stroke outpatients were asked to complete the S-OCS again over a period between 4 and 16 days after the first assessment to evaluate the test–retest reliability and intra-rater reliability.

Statistical analysis

Statistical analyses were conducted using R statistical software 3.3.2. (R Foundation for Statistical Computing, Vienna, Austria; http://www.r-project.org). The applied statistical tests were bilateral and signification was established at 0.05.

Descriptive analyses were performed using frequencies and percentages for the categorical variables. The distribution of the continuous variables was checked: they were presented as mean and standard deviation and compared with parametric tests if normally distributed; they were presented as median and interquartile range (IQR) and compared with non-parametric tests if not normally distributed. Categorical variables were compared with Fisher’s exact test. Test–retest, intra-rater, and inter-rater reliability were estimated by Spearman and intraclass correlation coefficients. A good intraclass correlation coefficient was deemed a value ⩾0.75¹⁸ and a coefficient ⩾0.50 for a strong Spearman correlation.¹⁹ Furthermore, the internal consistency of S-OCS scores was analyzed by the standardized Cronbach’s alpha. A criterion of 0.70 or higher was considered as a measure of good internal consistency.²⁰ Concurrent and divergent validity measures were also performed using Spearman correlation coefficients.

A paired sample matched for gender, age, and educational level was used to analyze differences in S-OCS scores between the stroke and the healthy participants; and within the stroke group, between right- and left-hemispheric stroke patients. The nearest neighbor matching method implemented in MatchIt package²¹ of the R statistical program was applied to make the best matching selection. We also used OptimalCutpoints package²² of R statistical software to calculate the sensitivity, specificity, predictive values, area under the curve, and optimal cut-offs using the best-known and widely used Youden index.²³ An area under the curve value of 0.70 or higher would indicate a good discriminant power²⁰ of the S-OCS subtests to detect cognitive deficits in the stroke outpatients. Additional stratified analyses were made to explore the influence of gender, age, and educational level on the discriminatory capacity of the test to distinguish between stroke and healthy participants, and between right- and left-hemispheric stroke among the outpatients.

Results

A total sample of 111 participants (57 stroke outpatients and 54 healthy individuals) recruited between November 2016 and September 2017 participated in this study. General characteristics of the study participants are summarized in Table 1. Due to the skewed distribution of the continuous variables in our sample, they were presented as median and IQR, and comparisons were made using the non-parametric Mann–Whitney U-test. Stroke outpatients were older, consisted of mostly men, and had a lower educational level compared to the healthy group. Clinical characteristics of the stroke outpatients indicated that they were largely in an early subacute phase of stroke recovery, with a median time from the clinical diagnosis of stroke to the first study assessment of 63 days, had a right-hemispheric brain lesion, had no previous sensory or motor disorders, had left-sided hemiparesis, and had ischemic stroke, of which more than one-third was affected by partial anterior cerebral infarction.

Table 1.

Demographic and clinical characteristics of the study participants (n = 111).

	Stroke group (n = 57)	Control group (n = 54)	P-value^a
Age, median (IQR)	74 (62–81)	63 (58–71)	0.003
⩽65 years	40.4 (23)	61.1 (33)	0.037
>65 years	59.6 (34)	38.9 (21)
Gender, % (n)			0.001
Male	73.7 (42)	42.6 (23)
Female	26.3 (15)	57.4 (31)
Education level, % (n)			0.011
Uneducated or primary	75.4 (43)	51.9 (28)
Secondary or higher	24.6 (14)	48.1 (26)
Marital status, % (n)			0.434
Married or living with partner	57.9 (33)	66.7 (36)
Without partner	24 (42.1)	18 (33.3)
Working status, % (n)			0.842
Working	33.3 (19)	31.5 (17)
Not working	66.7 (38)	68.5 (37)
Time (days) from clinical diagnosis of stroke, median (IQR)	63 (46–82)	–	–
Phases of stroke recovery,^b % (n)
Early subacute stroke (⩽90 days)	46 (80.7)	–	–
Late subacute stroke (>90 days)	11 (19.3)	–	–
Brain lesion location, % (n)
Right hemisphere	50.9 (29)	–	–
Left hemisphere	42.1 (24)	–	–
Bilateral	7.0 (4)	–	–
Previous sensory or motor disorders, % (n)
No	78.9 (45)	–	–
Yes	12.1 (21)	–	–
Hemiparesis, % (n)
Right side	40.3 (23)	–	–
Left side	49.1 (28)	–	–
Bilateral	1.8 (1)	–	–
No	8.8 (5)
Type of stroke, % (n)
Ischemic stroke	68.4 (39)	–	–
TACI	17.5 (10)	–	–
PACI	38.6 (22)	–	–
LACI	3.5 (2)	–	–
POCI	8.8 (5)	–	–
Haemorrhagic stroke	31.6 (18)	–	–
Intracerebral hemorrhage	29.8 (17)	–	–
Subarachnoid hemorrhage	1.8 (1)	–	–

P-value according to Mann–Whitney U-test for continuous variables and Fisher’s exact test for categorical variables.

Post-stroke phases have been established according to the framework of the critical time-points post stroke provided by Bernhardt et al.¹

IQR, interquartile range; TACI, total anterior cerebral infarction; PACI, partial anterior cerebral infarction; LACI, lacunar infarction; POCI, posterior circulation infarction.

Table 2 displays the results of the reliability analyses of the S-OCS subtests. Inter-rater reliability was computed in a subsample of 26 stroke outpatients using values independently scored by two research assistants at the first assessment. Intra-rater reliability and test–retest reliability were estimated in a subsample of 27 stroke outpatients using the scores obtained at the first and second assessments. The mean time interval between the test and retest assessment was 8.5 days (SD 4.2). The estimates showed an excellent inter-rater reliability of the S-OCS subtests (i.e. coefficients ⩾0.90), suggesting an optimal grade of agreement among the raters. Intra-rater agreement as measured by intraclass correlation coefficient was mostly within values ⩾0.75, which indicated a good reliability. Spearman correlation coefficients of the test–retest reliability showed acceptable or moderately good estimates, except for picture naming, visual field task, and hearts subtests (i.e. space and object asymmetry items). Standardized Cronbach’s alpha was 0.907, which suggests that S-OCS subtests are internally consistent. The estimation of alpha value remained almost the same when an item was removed from the test.

Table 2.

Reliability of the subtests of S-OCS in stroke outpatients.

Cognitive domain	S-OCS subtest	Reliability measures
		Inter-rater reliability^a		Intra-rater reliability^b	Test–retest reliability^b
		ICC	r_s	ICC	r_s
Language	Picture naming	0.990***	0.999***	0.330	0.279
	Semantics	0.790***	0.693***	0.880***	0.500**
	Sentence reading	1.000***	0.939***	0.990***	0.841***
Memory	Orientation (free response)	0.980***	0.974***	0.780***	0.608***
	Orientation (free response and multiple choice)	0.980***	0.926***	0.680**	0.635***
	Verbal memory (recall)	1.000***	1.000***	0.640**	0.534**
	Verbal memory (recall and recognition)	1.000***	1.000***	0.760***	0.401*
	Episodic memory	0.930***	0.935***	0.840***	0.625***
Attention	Visual field task	0.950***	0.913***	0.570*	0.340
	Hearts (total)	1.000***	1.000***	0.990***	0.971***
	Hearts (space asymmetry)	1.000***	1.000***	0.630**	0.362
	Hearts (object asymmetry)	1.000***	1.000***	0.181	0.173
	Executive tasks	1.000***	1.000***	0.660**	0.469*
Number	Number writing	1.000***	1.000***	0.920***	0.848***
Number	Calculation	0.980***	0.928***	0.890***	0.697***
Praxis	Gestural imitation (right hand)	0.990***	1.000***	0.890***	0.717***
Praxis	Gestural imitation (left hand)	1.000***	1.000***	0.960***	0.917***

Inter-rater reliability was computed in a subsample of 26 stroke outpatients at the first assessment.

Intra-rater reliability and test–retest reliability were estimated in a subsample of 27 stroke outpatients using the scores obtained at the first (i.e. initial evaluation) and second (i.e. within a period of 4–16 days after the first evaluation) assessments.

S-OCS: Spanish version of the Oxford Cognitive Screen; ICC, intraclass correlation coefficient; r_s, Spearman coefficient.

***

P < 0.001; **P < 0.01; *P ⩽ 0.05.

Table 3 presents the results of Spearman correlation coefficients estimated for the concurrent and divergent validity measures. Both the validity measures were computed in a sample of 56 stroke outpatients with complete information because a high proportion of the items measured on this occasion were missing in one participant. Almost all of the S-OCS subtests were positively and significantly correlated with the respective Montreal Cognitive Assessment and Barcelona test items, except for the space asymmetry and executive task subtests. Overall, the S-OCS showed strong (⩾0.50) or moderate (0.30–0.49) correlations, except for the aforementioned subtests. In contrast, the results from the Barthel index were mostly low and not statistically significant, except for total hearts and imitation items which were statistically significant and moderately correlated.

Table 3.

Validity measures: correlations between the performance on the subtests of S-OCS and the other neuropsychological tests in stroke outpatients (n = 56).

Cognitive domain	S-OCS subtest	Concurrent validity			Divergent validity (BI)
Cognitive domain	S-OCS subtest	Validation measures	r_s	P-value	r_s	P-value
Language	Picture naming	Picture naming (MoCA)	0.621	<0.001	0.182	0.175
	Semantics	Verbal comprehension (Barcelona test)	0.417	0.001	0.207	0.123
	Sentence reading	Sentence reading (Barcelona test)	0.735	<0.001	0.170	0.207
Memory	Orientation (free response)	Orientation (MoCA)	0.588	<0.001	0.235	0.079
	Orientation (free response and multiple choice)	Orientation (MoCA)	0.587	<0.001	0.082	0.544
	Verbal memory (recall)	Delayed recall (MoCA)	0.575	<0.001	0.102	0.448
	Verbal memory (recall and recognition)	Delayed recall (MoCA)	0.466	<0.001	0.263	0.048
	Episodic memory	Delayed recall (MoCA)	0.409	0.002	0.123	0.361
Attention	Visual field task	–	–	–	0.222	0.097
	Hearts (total)	Visuographic attention task (Barcelona test)	0.818	<0.001	0.443	<0.001
	Hearts (space asymmetry)	Visuographic attention task (Barcelona test)	0.209	0.123	−0.091	0.499
	Heart (object asymmetry)	–	–	–	−0.253	0.057
	Executive tasks	Executive task (MoCA)	−0.193	0.153	−0.026	0.849
Number	Number writing	Writing (Barcelona test)	0.908	<0.001	0.268	0.044
Number	Calculation	Attention (MoCA)	0.791	<0.001	0.222	0.096
Praxis	Imitation (right hand)	Gestural imitation (Barcelona test)	0.822	<0.001	0.403	0.002
Praxis	Imitation (left hand)	Gestural imitation (Barcelona test)	0.895	<0.001	0.318	0.016

S-OCS: Spanish version of the Oxford Cognitive Screen; BI, Barthel index; MoCA, Montreal Cognitive Assessment; r_s, Spearman coefficient.

The results of the between-group (i.e. stroke outpatient vs. control) and within-group (i.e. right- vs. left-hemispheric stroke) comparisons are displayed in Tables 4 and 5, respectively. Between-group differences among the stroke and healthy participants were analyzed in a paired sample matched for gender, age, and educational level (n = 108, 54 pairs). Similarly, within-group differences among right- and left-hemispheric stroke outpatients were estimated in a paired sample matched for the same covariates (n = 48, 24 pairs). Statistically significant differences between the stroke outpatients (lower scores) and the control group (higher scores) were observed in almost all the S-OCS scores. The performance of the right- and left-hemispheric stroke outpatients was also statistically significantly different for visual field task, number writing, space/object asymmetry, and left-hand imitation subtests.

Table 4.

Comparison of the S-OCS scores between stroke and control group in a paired sample.^a

S-OCS subtest	Score	Score range	Stroke group (n = 54)		Control group (n = 54)		Median difference	P-value^b
S-OCS subtest	Score	Score range	Median	IQR	Median	IQR	Median difference	P-value^b
Picture naming	Total	0 to 4	3.5	2 to 4	4	4 to 4	−0.5	<0.001
Semantics	Total	0 to 3	3	3 to 3	3	3 to 3	0	0.003
Orientation	Free response	0 to 4	3	2 to 4	4	4 to 4	−1	<0.001
Orientation	Total	0 to 4	4	3 to 4	4	4 to 4	0	<0.001
Visual field task	Total	0 to 4	4	2 to 4	4	4 to 4	0	<0.001
Sentence reading	Total	0 to 15	14	3.5 to 15	15	15 to 15	−1	<0.001
Number writing	Total	0 to 3	2	0 to 3	3	3 to 3	−1	<0.001
Calculation	Total	0 to 4	3	2 to 4	4	4 to 4	−1	<0.001
Hearts	Total	0 to 50	17	3.5 to 43	46.5	45 to 48	−29.5	<0.001
	Space asymmetry	−2 to 2	0	−1 to 5	0	−1 to 0	0	0.061
	Object asymmetry	0 to 1	0	0 to 2	0	0 to 0	0	0.031
Gestural imitation	Right hand	0 to 12	6	0 to 10	12	11 to 12	−6	<0.001
Gestural imitation	Left hand	0 to 12	0	0 to 9	12	11 to 12	−12	<0.001
Verbal memory	Recall	0 to 4	0.5	0 to 2	2.5	1 to 4	−2	<0.001
Verbal memory	Recall and recognition	0 to 4	3	1 to 4	4	3 to 4	−1	<0.001
Episodic memory	Recognition	0 to 4	3	2 to 4	4	4 to 4	−1	<0.001
Executive tasks	Executive	−1 to 12	0	0 to 3.7	−1	−1 to 4	−1	0.006

Participants were matched for gender, age, and educational level.

P-value according to Mann–Whitney U-test.

S-OCS: Spanish version of the Oxford Cognitive Screen; IQR, interquartile range.

Table 5.

Comparison of the S-OCS scores between left- and right-hemispheric stroke paired groups.^a

S-OCS subtest	Score	Score range	RH stroke (n = 24)		LH stroke (n = 24)		Median difference	P-value^b
S-OCS subtest	Score	Score range	Median	IQR	Median	IQR	Median difference	P-value^b
Picture naming	Total	0 to 4	3	2 to 4	4	2 to 4	−1	0.576
Semantics	Total	0 to 3	3	3 to 3	3	3 to 3	0	1
Orientation	Free response	0 to 4	4	2 to 4	4	1.7 to 4	0	0.381
Orientation	Total	0 to 4	4	3 to 4	4	3.7 to 4	0	0.652
Visual field task	Total	0 to 4	3	1.5 to 4	4	4 to 4	−1	0.013
Sentence reading	Total	0 to 15	14	4.5 to 15	14	7.5 to 15	0	0.879
Number writing	Total	0 to 3	2.5	0.7 to 3	0	0 to 2.2	2.5	0.020
Calculation	Total	0 to 4	3	2 to 4	3.5	2 to 4	−0.5	0.758
Hearts	Total	0 to 50	13.5	4.5 to 29.2	31.5	5.5 to 43.7	−18	0.189
	Space asymmetry	−2 to 2	7	0 to 13.5	−0.5	−4.2 to 0	6.5	<0.001
	Object asymmetry	0 to 1	2	0 to 3	0	−0.2 to 0	2	<0.001
Gestural imitation	Right hand	0 to 12	7	3.7 to 8.5	1.5	0 to 11	5.5	0.217
Gestural imitation	Left hand	0 to 12	0	0 to 0.2	9	4.7 to 11	−9	<0.001
Verbal memory	Recall	0 to 4	0	0 to 2	0	0 to 2	0	0.847
Verbal memory	Recall an recognition	0 to 4	3	2 to 4	3	1 to 4	0	0.758
Episodic memory	Recognition	0 to 4	3	2 to 4	3	2 to 3.2	0	0.823
Executive tasks	Executive	−1 to 12	0.5	0 to 3.2	0	−1 to 2	0.5	0.290

Hemispheric stroke groups were matched for gender, age, and educational level.

P-value according to Mann–Whitney U-test.

S-OCS: Spanish version of the Oxford Cognitive Screen; RH, right hemispheric; LH, left hemispheric; IQR, interquartile range.

Table 6 shows the sensitivity, specificity, predictive values, and optimal cut-offs of the S-OCS to distinguish stroke outpatients from healthy individuals. Overall, the results indicated that the S-OCS is an instrument with high specificity, presenting, therefore, a low probability of false positives. Although the test showed low sensitivities for the semantics and space/object asymmetry tasks, the responsiveness of the test as measured by the area under the curve was adequate, considering most of the values were around 0.70 or higher. Except the estimates for the aforesaid subtests, the calculated cut-offs may be suitable to differentiate clearly between stroke outpatients and healthy people. The influence of gender, age, educational level, and the lesion location among stroke outpatients evaluated in further analyses is displayed in Supplemental Tables 1S–4S, respectively. Briefly, the discriminant power of the subtests of the S-OCS was generally higher in women and those older than 65 years. The sentence reading, episodic memory, total hearts, and calculation subtests showed better estimates in those participants with low educational level. Among stroke outpatients, visual field task, total hearts, and executive tasks indicated a greater discriminant capacity in right-hemispheric stroke; however, episodic memory and number writing were better in left-hemispheric stroke. The S-OCS subtests targeted at the detection of neglect performance and apraxia (i.e. space/object asymmetry and gestural imitation subtests, respectively) obtained clearly higher discriminant estimates depending on the affected side of the stroke patients.

Table 6.

Discriminant indices of the S-OCS subtests in a paired sample matched for gender, age, and educational level (n = 108, 54 pairs).

S-OCS subtest	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	AUC	Optimal cut-off
Picture naming	31.5	98.1	94.4	58.9	0.671	2
Semantics	14.8	100	100	54.0	0.574	2
Sentence reading	53.7	88.9	82.9	65.7	0.735	14
Orientation (free response)	51.9	98.1	96.6	67.1	0.754	3
Orientation (free response and multiple choice)	31.5	100	100	59.3	0.657	3
Verbal memory (recall)	50.0	96.3	93.1	65.8	0.753	0
Verbal memory (recall and recognition)	70.4	66.7	67.9	69.2	0.743	3
Episodic memory	68.5	77.8	75.5	71.2	0.768	3
Visual field task	33.3	100	100	60.0	0.667	2
Hearst (total)	83.3	83.3	83.3	83.3	0.880	43
Hearts (space asymmetry)
Right neglect	20.4	98.1	91.7	55.2	0.397	−4
Left neglect	42.6	90.7	82.1	61.2	0.603	2
Heart (object asymmetry)
Right neglect	14.8	98.1	88.9	53.5	0.407	−1
Left neglect	33.3	98.1	94.7	59.6	0.593	1
Executive tasks	75.9	70.4	71.9	74.5	0.647	0
Number writing	61.1	96.3	94.3	71.2	0.796	2
Calculation	51.8	85.2	77.8	63.9	0.710	3
Gestural imitation (right hand)	72.2	90.7	88.6	76.6	0.850	9
Gestural imitation (left hand)	72.2	94.4	92.9	77.3	0.894	8

PPV, positive predictive value; NPV, negative predictive value; AUC, area under the curve.

Discussion

The findings of this study support the hypothesis that the S-OCS is a reliable and valid instrument for assessing cognitive impairments in subacute and chronic stroke outpatients. The estimates of inter-/intra-rater reliability, test–retest reliability, and internal consistency were good, which makes the Spanish version of test suitable to use it as a uniform battery. The correlation coefficients obtained from the concurrent and divergent validations provide the S-OCS with an accurate construct validity for testing cognitive functions. Moreover, the optimal cut-offs calculated correctly discriminated stroke outpatients from the control group, which makes it a useful screening tool for Spanish population.

Currently, there is no instrument for Spanish speakers that has been particularly created to assess cognitive impairment in stroke patients being sufficiently sensitive to detect common problems after stroke (e.g. aphasia, visual loss, spatial neglect, and apraxia). This study verifies the good performance of the S-OCS and confirms the results obtained from the original English version¹¹ and from the validation of the Hong Kong version for Cantonese speakers.¹⁴ As in these prior versions, for an effective clinical application of the S-OCS, the cut-off scores for each subtest were estimated. However, unlike the both previous validation studies of the Oxford Cognitive Screen,^11,14 our analyses provide a more accurate approach using an extensive set of adequate measures to examine the discriminant capacity of the test thoroughly. Specifically, we calculated the optimal cut-offs of the S-OCS subtests based on the Youden index²³ by maximizing the sensitivity and specificity across several cut-off points. Moreover, the area under the curve values from the receiver operating characteristic (ROC) analysis were estimated, which are considered as global measures of diagnostic accuracy;²⁴ therefore good indicators of the inherent discriminant ability of a test to suitably distinguish disease from health.

Similar to that reported by Kong et al.,¹⁴ we also matched our sample for gender, age, and educational level and performed additional stratified analyses of the discriminant indices in order to control the potential confounding.²⁵ Although between-group paired comparisons indicated clear differences among the scores of stroke and control participants, we observed that an overall best discriminatory capacity of the S-OCS subtests was more evident in women, those older than 65 years, and those with low educational level, suggesting, in this case, different cut-off points in certain cognitive functions. In terms of clinical practice, it should be noted that the optimal cut-offs of the S-OCS subtests estimated in this study may allow an extensive assessment, furnishing health professionals with a more detailed cognitive profile of individuals. Moreover, the different performance of the visual field, number writing, space/object asymmetry, and left-hand imitation subtests observed in paired right- and left-hemispheric stroke outpatients confirms the accuracy of the test to detect specific impairments affecting stroke survivors according to brain lesion location. In this regard, the cut-offs by the laterality of the brain lesion could provide valuable help for clinicians to make more comprehensive examinations of particular areas or to adequately design therapeutic interventions and treatments favoring adaptation processes of stroke patients.

The Montreal Cognitive Assessment is a commonly used short cognitive screening tool that has demonstrated a good ability for the detection of cognitive impairments in stroke survivors.^8,26 However, it has shown limitations on the capacity for distinguishing these deficits in right hemisphere stroke patients as well as on the identification of domain-specific cognitive impairment, suggesting an overall poorer test accuracy.^27,28 In contrast, the Oxford Cognitive Screen may clearly identify stroke-specific deficits using a domain-specific screening system to ascertain impairments at the domain level that helps to restrain the confounding effects from the stroke lateralization.²⁹ In this respect, it is worth mentioning that the identification of domain-specific cognitive impairments is considered a clinically crucial factor influencing the length of the stay, long-term rehabilitation needs, and functional outcomes,^28,30 which reinforces the need of monitoring cognitive conditions of stroke survivors. Thus, our results adequately support the potential application of the S-OCS at regular time-points following stroke as a suitable and accurate tool for detecting cognitive impairments.

Nevertheless, our study has several shortcomings. The relatively small size of our sample limits the potential representativeness of the data, thereby reducing the generalization of our findings to all Spanish-speaking stroke patients. Moreover, although we accounted for sociodemographic (age, sex, and educational level) and clinical (stroke location) factors closely related to cognitive decline⁹ and performed matching and stratified analysis in order to control potential confounding, a wider national sample ensuring the diversity of participants is warranted to confirm our findings. Regarding the discriminant power of S-OCS to detect cognitive impairment, it can be argued that semantic task and space/object asymmetry measuring right neglect presented very low sensitivities (⩽20%). However, it should be noted that the Oxford Cognitive Screen is an aphasia and neglect-friendly test targeted at maximizing patient inclusion and minimizing confounding effects of cognitive deficits such as language or attentional problems;¹¹ therefore, the level of difficulty of the tasks is very low trying to check the preserved basic cognitive abilities. In addition, although the asymmetry measures are derived from the visual attention hearts test, the low sensitivity obtained may be partly attributed to the fact that they examine separate aspects within this domain (i.e. egocentric or allocentric neglect).¹¹

The S-OCS is a tool for assessing cognitive impairments in subacute stroke survivors. This screening instrument facilitates the identification of domain-specific cognitive impairments, and its application may significantly contribute to increase the rehabilitation effectiveness of the therapeutic interventions targeted at the specific affected domains of the stroke survivors. Thus, it should be highly recommended to implement it within clinical practice as a routine part of stroke rehabilitation.

Clinical messages

The Spanish version of the Oxford Cognitive Screen is a reliable, valid, and accurate tool to assess common cognitive impairments in Spanish outpatients in a subacute stroke phase.

The small size of sample (n = 111) limits the generalization of our findings to all Spanish-speaking stroke survivors.

Supplemental Material

Suplementary_Material – Supplemental material for Validation of the Spanish version of the Oxford Cognitive Screen (S-OCS): psychometric properties of a short cognitive stroke-specific screening tool

Supplemental material, Suplementary_Material for Validation of the Spanish version of the Oxford Cognitive Screen (S-OCS): psychometric properties of a short cognitive stroke-specific screening tool by Desirée Valera-Gran, Sofía López-Roig, Miriam Hurtado-Pomares, Paula Peral-Gómez, María García-Manzanares, Marta Sunyer Catlla, Gemma Más Sesé, Eva María Navarrete-Muñoz and Alicia Sánchez-Pérez in Clinical Rehabilitation

Footnotes

Acknowledgements

The authors would like to thank all participants for their involvement. A.S.-P., S.L.-R, and M.G.-M. conceived the study, designed the S-OCS, and contributed to the conceptual approach; D.V.-G. conducted the data analysis and prepared the manuscript; A.S.-P., M.H.-P., P.P.-G., E.M.N.-M., and D.V.-G. contributed to the data interpretation and manuscript preparation; and A.S.-P., M.S.C., G.M.S., and D.V.-G. supervised the data collection. All the authors were responsible for critical revision of the manuscript for important intellectual content.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Desirée Valera-Gran

Supplemental material

Supplemental material for this article is available online.

References

Bernhardt

Hayward

Kwakkel

et al . Agreed definitions and a shared vision for new standards in stroke recovery research: the Stroke Recovery and Rehabilitation Roundtable taskforce. Int J Stroke 2017; 12: 444–450.

Moran

Fletcher

Feltham

et al . Fatigue, psychological and cognitive impairment following transient ischaemic attack and minor stroke: a systematic review. Eur J Neurol 2014; 21(10): 1258–1267.

Sun

J-H

Tan

J-T.

Post-stroke cognitive impairment: epidemiology, mechanisms and management. Ann Transl Med 2014; 2(8): 80.

Nys

GMS

van Zandvoort

de Kort

et al . Cognitive disorders in acute stroke: prevalence and clinical determinants. Cerebrovasc Dis 2007; 23(5–6): 408–416.

Kalaria

Akinyemi

Ihara

Stroke injury, cognitive impairment and vascular dementia. Biochim Biophys Acta 2016; 1862(5): 915–925.

Jokinen

Melkas

Ylikoski

et al . Post-stroke cognitive impairment is common even after successful clinical recovery. Eur J Neurol 2015; 22(9): 1288–1294.

Nakling

Aarsland

Næss

et al . Cognitive deficits in chronic stroke patients: neuropsychological assessment, depression, and self-reports. Dement Geriatr Cogn Dis Extra 2017; 7(2): 283–296.

Burton

Tyson

SF.

Screening for cognitive impairment after stroke: a systematic review of psychometric properties and clinical utility. J Rehabil Med 2015; 47(3): 193–203.

Tang

Amiesimaka

Harrison

et al . Longitudinal effect of stroke on cognition: a systematic review. J Am Heart Assoc 2018; 7(2): e006443.

10.

McDonnell

Bryan

Smith

et al . Assessing cognitive impairment following stroke. J Clin Exp Neuropsychol 2011; 33(9): 945–953.

11.

Demeyere

Riddoch

Slavkova

et al . The Oxford Cognitive Screen (OCS): validation of a stroke-specific short cognitive screening tool. Psychol Assess 2015; 27(3): 883–894.

12.

Wild

Grove

Martin

et al . Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health 2005; 8(2): 94–104.

13.

Diez-Tejedor

Guía oficial para el diagnóstico y tratamiento de ictus. Spanish Neurology Society, http://www.dep4.san.gva.es/contenidos/urg/archivos/guias/2004/Guia%20SEN%20ictus.pdf (accessed 10 January 2018).

14.

Kong

Lam

et al . The Hong Kong version of the Oxford Cognitive Screen (HK-OCS): validation study for Cantonese-speaking chronic stroke survivors. Neuropsychol Dev Cogn B Aging Neuropsychol Cogn 2016; 23(5): 530–548.

15.

Lozano

Hernández-Ferrándiz

Turró-Garriga

et al . Validación del Montreal Cognitive Assessment (MoCA): test de cribado para el deterioro cognitivo leve. Datos Preliminares. Alzheimer Real Invest Demenc 2009; 43: 4–11.

16.

Peña-Casanova

Programa integrado de exploración neuropsicológica test—Barcelona-revisado. 2nd ed. Barcelona: Masson, 2005.

17.

Cid-Ruzafa

Damián-Moreno

Valoración de la discapacidad física: el índice de Barthel. Rev Esp Salud Pública 1997; 71: 127–137.

18.

Koo

MY.

A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15(2): 155–163.

19.

Cohen

Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, MI: Lawrence Erlbaum, 1988.

20.

Terwee

Bot

de Boer

et al . Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007; 60(1): 34–42.

21.

Imai

King

et al . MatchIt: nonparametric preprocessing for parametric causal inference. J Stat Softw 2011; 42: 1–28.

22.

López-Ratón

Rodríguez-Álvarez

Cadarso-Suárez

OptimalCutpoints: an R package for selecting optimal cutpoints in diagnostic tests. J Stat Softw 2014; 61: 1–36.

23.

Youden

WJ.

Index for rating diagnostic tests. Cancer 1950; 3(1): 32–35.

24.

Eusebi

Diagnostic accuracy measures. Cerebrovasc Dis 2013; 36: 267–272.

25.

Hajian-Tilaki

Receiver operating characteristic (ROC) curve analysis for medical diagnostic test evaluation. Caspian J Intern Med 2013; 4(2): 627–635.

26.

Lees

Selvarajah

Fenton

et al . Test accuracy of cognitive screening tests for diagnosis of dementia and multidomain cognitive impairment in stroke. Stroke 2014; 45(10): 3008–3018.

27.

Chan

Khan

Oliver

et al . Underestimation of cognitive impairments by the Montreal Cognitive Assessment (MoCA) in an acute stroke unit population. J Neurol Sci 2014; 343(1–2): 176–179.

28.

Chan

Altendorff

Healy

et al . The test accuracy of the Montreal Cognitive Assessment (MoCA) by stroke lateralisation. J Neurol Sci 2017; 373: 100–104.

29.

Demeyere

Riddoch

Slavkova

et al . Domain-specific versus generalized cognitive screening in acute stroke. J Neurol 2016; 263(2): 306–315.

30.

Middleton

Lam

Fahmi

et al . Frequency of domain-specific cognitive impairment in sub-acute and chronic stroke. NeuroRehabilitation 2014; 34(2): 305–312.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.50 MB