Reliability of behavioral tests in the middle cerebral artery occlusion model of rat

Abstract

German

Spanish

French

Stroke is one of the leading causes of death and disability worldwide, and its incidence is increasing. To overcome impairment from stroke, translational research for developing new therapeutic technologies has been conducted and middle cerebral artery occlusion (MCAo) in rat is the representative model. Since recovery from neurological impairment in contralateral limbs caused by brain damage is the major goal of treatment, behavioral tests that assess the relevant function are used. To determine therapeutic effect, obtaining reliable results of behavioral assessment is a prerequisite. However, studies on the reliability of behavioral tests in the MCAo rat model and necessity of prior training have not yet been reported. In this study, the authors investigate relative and absolute inter-rater reliabilities of modified neurological severity score (mNSS), cylinder test, and grid-walking test before training and repeated training every week until the reliability of results reached a satisfactory level. The training included repeated learning of the scoring system and decreasing disagreements among the raters. For MCAo modeling, adult male Sprague-Dawley rats were subjected to 90 min of transient MCAo. Six raters conducted behavioral tests via observation of video-recording on sham-operated and MCAo model rats at 3 or 7 days after the intervention. An independent experimenter randomly numbered each video clip to blind the experiment. The results of reliabilities were unacceptable before training and improved to a satisfactory level after 6 weeks of training in all of the tests. In conclusion, mNSS, cylinder test, and grid-walking test on the MCAo rat model are reliable evaluation methods after conducting appropriate training.

Keywords

MCAo behavioral test reliability rodents stroke

Introduction

Completed stroke means continued neurologically impaired status from cerebrovascular events including ischemia and hemorrhage, and it is the second leading cause of death.^1,2 As damage to brain tissue remains difficult to repair, much therapeutic effort has been made with fundamentals from translational research using animal models.³

To date, the experimental focal transient ischemic stroke model by middle cerebral artery occlusion (MCAo) in rat is the model most commonly used for mimicking human stroke.^4,5 Using rat has advantages for cost and ethical reasons. Moreover, the cerebrovascular physiology and anatomy of rat is similar to that of humans.⁶

One of the major sequelae of stroke in humans is the paralysis of contralateral extremities.^7,8 The MCAo rat model also reveals paralytic features, and thus an important target for therapy is usually motor and/or sensory recovery in use of the limbs.⁹ There are several behavioral tests used in the MCAo rat model to evaluate neurological impairment in the contralateral limbs, such as the modified neurological severity score (mNSS), cylinder test, grid-walking test, accelerated rotarod, and grip strength test.^9–15

As MCAo rat model analysis is the most important reference for approving the effect of a specific therapy, responses in behavioral tests have great significance.¹⁶ Therefore, acquiring reliability of behavioral tests is necessary.¹⁷ Although reliability evaluation is widely used for brain-injured human subjects, including in stroke,^18-20 it has not yet been reported for the behavior tests in the MCAo rat model.^21,22

“Reliability” means consistency and stability of scores on an assessment tool, and it is classified into relative and absolute reliabilities.²³ Inter-rater reliability is an estimate of how consistently different raters use the test on the same subject.¹⁹ Although “relative reliability” is the most commonly used tool in reliability studies, measurement errors between repeated measurements by the other rater can be quantified with “absolute reliability.”^24,25 Relative reliability expresses agreement in order of scores in a group and not the absolute value of the score,²⁶ and the intraclass correlation coefficient (ICC) is widely used to quantify the relative reliability.²⁶ Even when the ICC is high enough, it does not ensure agreement in absolute values of scores.²⁴ The absolute reliability is derived by calculating the difference among the scores from the raters; the most well-known indicators are standard error measurement (SEM) and smallest real difference (SRD).²⁰ The SEM represents standard deviation (SD) of measurement errors and the SRD signifies sensitivity to detect change in the scale.^24,27 Therefore, securing successful values of ICC, SEM, and SRD is important to guarantee reliable results of a test.

Clinical studies usually offer a training program before conducting a test in order to secure reliability.^18,19 However, the necessity of training programs for behavioral tests in the MCAo rat model has not yet been reported, although guidelines have been provided.^28,29

In this study, the authors investigated the relative and absolute reliabilities of behavioral tests in the MCAo rat model using mNSS, cylinder test, and grid-walking test. The study also aimed to verify whether there is a necessity to train the raters, by conducting reliability evaluation prior to and during the training until the indices of the reliabilities were satisfactory.

Materials and methods

Animals

All experimental protocols were approved by the Institutional Animal Care and Use Committee of the study institute. Adult male Sprague-Dawley rats were purchased from Charles River Laboratories (Seoul, Korea). The rats were housed in a temperature-controlled room (22℃ ± 2°), with constant humidity (50% ± 10%), and a constant supply with purified water and food ad libitum on 12-hour night–day cycle. The Sprague-Dawley rats usually weighed 250–300 g (8 weeks old) at the time of surgery. The rats were adapted to the environment for at least 1–2 days before surgery. Rats were fed feed and water freely until the evening before surgery. Feeds were stopped from the night before surgery until the day of surgery, but water was continuously supplied.

Modeling transient middle cerebral artery occlusion

Transient MCAo was induced by an intraluminal vascular occlusion modified in our laboratory.³⁰ Rats were initially anesthetized with 3.0% isoflurane in 70% N₂O and 30% O₂ (v/v). Under an operating microscope, a midline neck incision was made with scissors between the manubrium and the jaw. The right common carotid artery (CCA) was separated and isolated from the vagus nerve and temporarily ligated during the whole period of occlusion. The right external carotid artery (ECA) and the right internal carotid artery (ICA) were also isolated. The right ECA was ligated and cauterized to make an ECA stump. The hypothalamic artery was also ligated. A small hole was made with micro scissors in the ECA stump. A nylon filament suture with a silicon-coated tip was inserted through the hole in the ECA. To steadily hold the filament and effectively occlude the MCA, the ICA was ligated temporarily. The ECA stump over the hole was temporarily ligated with a silk suture in order to stop bleeding from the hole. After 90 min of occlusion, the rats were re-anesthetized and the suture was gently removed from the MCA to re-perfuse the MCA territory. The hole in ECA stump was permanently ligated and the temporary ligation in the CCA was removed to allow regaining of blood flow to the brain. The midline neck incision was closed and injection of 3 ml of intraperitoneal saline was provided to prevent dehydration. As a control group, sham-operated rats were subjected to the same procedure without MCAo. The rats used for behavioral tests were randomly selected from sham-operated and MCAo rats at 3 days and 7 days after surgery.

Behavioral tests

Modified neurological severity score

On the mNSS, neurological functions including motor and sensory systems as well as reflexes and balance are graded on a numeric scale from 0 to 18.³¹ One experimenter performed the mNSS and another experimenter video-recorded it to evaluate the reliability of multiple raters. The scoring method follows the description in Table 1 by observation of abnormal asymmetric responses in the limbs of the impaired side.³¹ The higher score indicates more severe neurological impairment from the unilateral cerebral lesion.³² The original mNSS consists of sub-tests including motor tests (raising rat by tail, walking on floor), sensory tests (placing test, proprioceptive test), beam balance test, reflex absence and abnormal movements (Figure 1). However, in this study, reflex absence and abnormal movements were excluded because they are irrelevant to damage in the MCA area.¹⁰ Therefore, in this study, zero points stands for normal and 14 was the score of the most severe neurological defect (Table 1).^29,33-35 Although not mentioned in previous studies,^31,33 1 point was given when there was no reaction more than three times out of five attempts, in order to increase the accuracy in the sensory tests in this research.

Figure 1.

Modified Neurological Severity Score (mNSS). (a) Raising rat by tail; (b) Beam balance test; (c) Placing test; and (d) Proprioceptive test.

Table 1.

Modified neurological severity score (mNSS).

Tests	Points
Raising rat by tail (normal = 0; maximum = 3)
Flexion of forelimb	1
Flexion of hindlimb	1
Head moved > 10° to vertical axis within 30 s	1
Walking rat on floor (normal = 0; maximum = 3)
Normal walk	0
Inability to walk straight	1
Circling toward the paretic side	2
Falls down to paretic side	3
Beam balance test (normal = 0; maximum = 6)
Balances with steady posture (>60 s)	0
Grasps side of the beam	1
Hugs beam and 1 limb falls down from beam	2
Hugs beam and 2 limbs falls down from beam, or spins on beam (>60 s)	3
Attempts to balance on beam but falls off (>40 s)	4
Attempts to balance on beam but falls off (>20 s)	5
Falls off, no attempt to balance or hang on the beam (<20 s)	6
Sensory tests (normal = 0; maximum = 2)
Placing test (visual and tactile test, try 5 times and have not moved more than 3 times)	1
Proprioceptive test (deep sensation, pushing the paw against the table edge to stimulate limb muscles, try 5 times and have not moved more than 3 times)	1
Maximum Points	14

Cylinder test

On the cylinder test, asymmetric use of forepaw was analyzed by video-recording of the rat in a transparent cylinder (20 cm in diameter and 30 cm in height).³⁶ A mirror was placed under the cylinder at a 45° angle to video-record (Figure 2). The rat was placed by an experimenter in the cylinder, and the vertical exploratory behavior was observed. Movement of the rat getting up and placing its forepaws on the wall 20 times in total was used for rating. For the analysis, the video was played at 0.3 times slower than real time. During a rear, the first forepaw to place the wall was recorded as right and left. When one forepaw was placed on the floor and other forepaw was on the wall, it was not counted. In cases of both forepaws touching the wall at the same moment or a forepaw touching the wall followed by the other forepaw without interval, the rater recorded “bilateral.” Exploring the wall after initial touching was not recorded. In order to count the next contact, both forepaws should have touched the floor after the initial placing on the wall. The number of contacts consisting of ipsilateral (intact side), contralateral (impaired), or bilateral contacts were counted. An asymmetry score was made by the calculation formula as $(ipsilateral + bilateral / 2) / (ipsilateral + contralateral + bilateral) .$

The cylinder test is advantageous in that it is objective, easy, and sensitive to chronic deficits which cannot be detected by other methods.¹⁰ However, the Sprague-Dawley rats used in this study show inactivity when placed in a cylinder. Therefore, in this study, we used ways to encourage movement in the rats that did not affect the asymmetry score. These methods include picking up and replacing the rat into the cylinder, placing food in the top of the cylinder, and putting another rat in the cylinder for a while.³⁷

Figure 2.

Cylinder Test. Spontaneous forelimb use of rat being assessed.

Grid-walking test

In the grid-walking test, an elevated metal square grid (40 × 40 cm², with each grid cell 3 × 3 cm²; height: 41 cm) was used.³⁸ A video camera for assessment of forepaw slip was located under the grid-walking apparatus with an angle between 20° and 40° (Figure 3). The rat was placed on the center of the apparatus by an experimenter, and allowed to explore for 5 min. The behaviors of the rat on the grid were recorded with video camera and the motions of both forepaws were analyzed by raters. For the analysis, the video was played at 0.3 times slower than real time. The slip of forepaw was scored when (a) the forepaw entirely missed a rung at the time of stepping on the rung, (b) the forepaw fell down between the rungs, or (c) the forepaw slipped off during weight bearing.³⁸ The case in which a rat placed its forepaws on the rung without slip was considered as a step. The analysis was performed until the summation of steps and slips of both forepaws was 100. A percentage of affected-side forepaw slip among the summation of steps and slips of both forepaws was calculated.

Figure 3.

Grid-walking test. Spontaneous motor deficits and limb movements of rat being assessed.

Assessment of behavioral tests

Six raters participated in this study. The experience of the raters ranged from 1 month to 5 years. After reviewing the literature on behavioral tests, the initial guidelines were made by the most experienced rater.^{10,13,31,32,38} In the first week, video-recordings for five sham-operated rats and 20 MCAo rats were assessed without grouping. The results were analyzed to find out disagreement among raters. After that, five new MCAo rats were weekly analyzed individually, and weekly meetings were held thereon from the second week to the sixth week. The causes of error and the disagreements among the raters were identified. Although the raters followed the published protocols, consensus had to be reached for some parts of the rating processes, as described in the results section. After achieving a satisfactory level of reliability, the last assessment for reliability was conducted on 20 sham-operated rats and 40 MCAo rats without grouping again for confirmation. For blinding of the experiment, the videos of sham-operated rats and MCAo rats were randomly numbered by an independent provider who was not part of the rater team at the 6 weeks’ training session and the last reliability assessment. After random numbering, the six raters assessed the behavioral tests. Through the random numbering, the six raters did not know whether the video was a sham-operated or MCAo rat during in the scoring process.

Statistical analysis

The ICC value with 95% confidence interval was used to evaluate the relative reliability of each test, and a value of at least 0.90 for ICC was targeted to acquire a satisfactory level of relative reliability.³⁹ The SEM and SRD values were calculated to analyze absolute reliability for each test. The calculation formula for SEM was: $SEM = SD \times \sqrt (1 - r)$ .⁴⁰ In this study, ICC values were used as r values.⁴¹ The smaller SEM means that the reliability of the measurement is higher. ⁴¹ In the case of SEM values being less than 10% of the total average score, it was suggested that such SEM values are acceptable.²³ The SRD was calculated by the formula $nbsSRD = 1.96 \times \sqrt 2 \times SEM$ .⁴² The target value of acceptable SRD was less than 10% of the highest score.²⁶ All statistical analyses were performed using SPSS version 21.0 software (SPSS Inc, Chicago, Illinois).

Results

Difficulties that required consensus during the training

Published criteria for mNSS were sufficient, therefore no further discussion was needed.^31,33 However, in the case of the cylinder test, disagreement among the raters occurred. The following conditions were observed: (1) a part of the rat’s forepaw was touched on the wall and (2) rats stood up at an angle not perpendicular to the ground.

After discussion, a score was given only when (1) the forepaw completely touched the wall, and (2) the rat stood up perpendicular to the ground. A similar debate occurred in the grid-walking test. Unexpected behaviors of rats were observed, such as (1) climbing up and down the wall around the grid apparatus and (2) grabbing the edge of the grid. These were solved by step-counting in the case of climbing down from the wall, and not as step-counting in climbing up the wall and grabbing the outer edge.

Reliability values at initial training session

The ICC, SEM, and SRD values of the mNSS, cylinder test, and grid-walking test in 20 MCAo rats and five sham-operated rats at the initial training session are presented in Table 2. The ICC for all measures showed unacceptable relative reliability (ICCs < 0.8, p < 0.001). As for absolute reliability, the SEM values were greater than 10% of the mean value of mNSS and cylinder test. The SRD were also greater than 10% of highest score of mNSS, cylinder test, and grid-walking test.

Table 2.

Reliability for mNSS, cylinder test, and grid-walking test in ischemic stroke model and sham-operated rats at initial training session (six raters).

	Mean ± SD	Range of scores	(ICC:95% CI)	p	SEM	SRD
mNSS	7.408 ± 2.961	0–14	0.775 (0.577–0.899)	<0.001	1.404	3.893
Cylinder test	30.694 ± 21.253	0–85	0.789 (0.604–0.905)	<0.001	9.763	27.061
Grid-walking test	12.982 ±1.958	10–23	0.749 (0.529–0.887)	<0.001	0.981	2.719

SD: standard deviation; ICC (95% CI): Intraclass Correlation Coefficient (95% Confidence Interval); SEM: Standard Error Measurement; SRD: Smallest Real Difference; mNSS: modified neurological severity score. Number of ischemic stroke rats used n = 20, and sham-operated rats used n = 5.

Changes of reliability values during training

According to the plan and weekly results, the training continued to the sixth week. While the indicator of relative reliability, ICCs, for total mNSS, cylinder test, and grid-walking test reached the target level from the fourth week of training (ICCs > 0.9, p < 0.001), the indicators of absolute reliability, SEM, and SRD reached the target from the third to fifth week of training (Figure 4).

Figure 4.

Changes in reliabilities of modified neurological severity score (mNSS), cylinder test, and grid-walking test during training. (a) ICC value, (b) SEM/Average, and (c) SRD/Measured highest score value. ICC (95% CI), Intraclass Correlation Coefficient (95% Confidence Interval); SEM, Standard Error Measurement; SRD, Smallest Real Difference. n = 5 MCAo rats were used each time.

Regarding the results of sub-tests of mNSS, ICCs became greater than 0.9 in “raising the rat by the tail,” “sensory tests,” and “beam balance test” from the fourth week of training, and in “walking on the floor” from the sixth week of training (p < 0.001). In most sub-tests, SEM reached the target from the fifth week of training and SRD reached the target from the fourth week of training. Only “walking on the floor” reached the target at the sixth week of training for SEM, while it took 5 weeks for SRD (Figure 5).

Figure 5.

Changes in reliabilities of sub-tests of modified Neurological Severity Score during training. (a) ICC value, (b) SEM/Average, and (c) SRD/Measured highest score value. ICC (95% CI), Intraclass Correlation Coefficient (95% Confidence Interval); SEM, Standard Error Measurement; SRD, Smallest Real Difference. n = 5 MCAo rats were used each time.

Reliability of mNSS test after 6 weeks’ training

The ICC, SEM, and SRD values of the mNSS in 20 sham-operated rats and 40 MCAo rats after training are presented in Table 3. ICC showed excellent relative reliability in the sham-operated rats and the MCAo rats for total and all sub-tests (ICC ≥ 0.9, p < 0.001). As for absolute reliability, SEMs were less than 10% of means, and SRDs were less than 10% of highest scores in the sham-operated rats and the MCAo rats for total and all sub-tests of the mNSS.

Table 3.

Reliability for mNSS in sham-operated and MCAo rats after 6 weeks’ training (six raters).

		Mean ± SD	Range of scores	ICC(95% CI)	p	SEM	SRD
Raising rat by tail test	Sham-operated rats	0.358 ± 0.158	0 - 2	0.951 (0.908–0.978)	<0.001	0.035	0.097
Raising rat by tail test	MCAo rat	1.454 ± 0.238	0 - 3	0.933 (0.895–0.961)	<0.001	0.061	0.170
Walking on floor test	Sham-operated rats	0.042 ± 0.014	0 - 1	0.960 (0.812–0.955)	<0.001	0.003	0.008
Walking on floor test	MCAo rat	0.525 ± 0.219	0 - 3	0.944 (0.912–0.967)	<0.001	0.052	0.144
Beam balance test	Sham-operated rats	0.433 ± 0.189	0 - 3	0.961 (0.927–0.982)	<0.001	0.037	0.103
Beam balance test	MCAo rat	2.658 ± 0.274	0 - 6	0.991 (0.987–0.995)	<0.001	0.026	0.072
Sensory tests	Sham-operated rats	0.342 ± 0.014	0 - 2	0.996 (0.992–0.998)	<0.001	0.001	0.002
Sensory tests	MCAo rat	0.992 ± 0.746	0 - 3	0.992 (0.987–0.995)	<0.001	0.067	0.185
Total score	Sham-operated rats	1.175 ± 0.317	0 - 4	0.960 (0.925–0.982)	<0.001	0.063	0.176
Total score	MCAo rat	5.629 ± 0.590	0 - 12	0.991 (0.986–0.995)	<0.001	0.056	0.155

Reliability of cylinder test and grid-walking test after 6 weeks’ training

The ICC, SEM, and SRD values of the cylinder test and the grid-walking test in 20 sham-operated rats and 40 MCAo rats after training are shown in Table 4. The values of ICC showed excellent relative reliability in the sham-operated rats and the MCAo rats for the cylinder test and grid-walking test (ICC ≥ 0.9, p < 0.001). Regarding absolute reliability, the values of SEM were less than 10% of means, and the values of SRD were less than 10% of highest scores in the sham-operated rats and the MCAo rats.

Table 4.

Reliability for cylinder test and grid-walking test in sham-operated and MCAo rats after 6 weeks’ training (six raters).

		Mean ± SD	Range of scores	ICC(95% CI)	p	SEM	SRD
Cylinder test	Sham-operated rats	36.805 ± 4.271	0–82.5	0.950 (0.950–0.978)	<0.001	0.955	2.647
Cylinder test	MCAo rats	28.673 ± 2.584	0–65	0.994 (0.988–0.998)	<0.001	0.200	0.555
Grid-walking test	Sham-operated rats	4.808 ± 4.533	0–16	0.989 (0.978–0.995)	<0.001	0.475	1.318
Grid-walking test	MCAo rats	8.556 ± 8.556	1–54	0.99 (0.989–0.997)	<0.001	0.716	1.984

SD: standard deviation; ICC (95% CI): Intraclass Correlation Coefficient (95% Confidence Interval); SEM: Standard Error Measurement; SRD: Smallest Real Difference. Number of ischemic stroke rats used n = 20, and sham-operated rats used n = 40.

Discussion

In this study, relative and absolute inter-rater reliabilities of behavioral tests were evaluated for the MCAo rat model, and changes in the reliabilities during the training were also described. To our knowledge, this is the first study that shows reliability evaluation of behavioral tests in rat with brain injury. As assessment tools, the mNSS, cylinder test, grid-walking test, accelerated rotarod, and grip strength test were chosen after reviewing previous studies. As preliminary results of the rotarod and grip strength test were poor, these were excluded. The tests used in this study are the most representative tools for evaluating neurological impairment status of the rat stroke model.^9-15 The mNSS is the one of the most frequently used comprehensive scales for the MCAo rat model.⁴³ The cylinder test evaluates unilateral motor impairment from stroke by observing spontaneous forelimb use in rats,⁴⁴ and the grid-walking test is a relatively simple and objective tool to assess the accuracy of limb movement after stroke.¹⁰

The behavioral tests have been used to evaluate efficacy of a possible therapy for stroke.^31,45 Based on the evidence from animal experiments, including behavioral tests, clinical trials are underway.^16,46 Thus, in a pre-clinical study using MCAo rat model, it would be important to ensure reliability of the behavioral tests.

According to our results, pre-evaluation training for raters in the behavioral test is necessary. The indices of relative and absolute reliabilities of behavioral tests prior to the training were unacceptable. However, reliability of the tests improved by conducting training. Ideally, a good scale does not need additional criteria from the published version. Although the authors made efforts to follow the published protocol, we found difficulties in reaching agreement. In the cylinder test, area of forepaw touching and angle of trunk at rearing were the main problems of disagreement, and in the grid-walking test unexpected motion at the edge of the grid needed criteria for scoring.

It took 4–6 weeks to obtain the target level of reliability in each test. In terms of relative reliability, we set the target value of ICCs as 0.90. Although ICC of at least 0.70 was suggested for group comparisons,⁴⁷ more stringent criteria of ICC ≥ 0.90 was selected because this criterion is recommended for methods to make clinical decisions.³⁹ In our study, the SEM and SRD were used additionally to evaluate the absolute reliability.²⁰ To obtain the target level of absolute reliability, 3–5 weeks of training were required for total mNSS and most of its sub-tests, cylinder test, and grid-waking test, and 6 weeks for a sub-test of the mNSS, “walking on the floor.” Final assessment, which measured 40 MCAo model rats and 20 sham-operated rats, showed fulfillment of reliability level for all behavioral tests.

In clinical studies, training such as workshops and teaching of the evaluation tool are offered before evaluation. An evaluation tool for motor behavior in children with cerebral palsy was accompanied by a 30-hour workshop before confirming reliability.¹⁹ For evaluation of Parkinson’s disease, a teaching videotape of the Unified Parkinson’s Disease Rating Scale was reviewed by researchers before inter-rater reliability evaluation.¹⁸ However, in existing reports, there is no mention about the necessity of training for raters before behavioral tests in animal models of disease. This study demonstrates the importance of the training process for behavioral tests in animal brain injury models.

The most widely known methods to improve the quality of randomized controlled studies are randomization and blinding. In this study, rater blindness was secured through random number assignment for each subject rat. The raters did not know whether rats were sham-operated or MCAo rats, to avoid any possible bias.

This study has some limitations. As some of the raters were novices to the rat experiments, the reliability at initial evaluation could have been lower than that in well-established laboratories. Additional definition in the cylinder test and grid-walking test might not be appropriate for the original intention of the tests. In addition, the mNSS used in this study cannot be universal for all experiments using the rat stroke model.

Conclusion

This study demonstrates the relative and absolute reliabilities of the mNSS, cylinder test, and grid-walking test in the MCAo rat model for the first time, using ICC, SEM, and SRD values obtained from six raters. According to the significant changes in the reliabilities through training, it is enough to conclude that pre-evaluation training is necessary to acquire a satisfactory level of reliability.

Footnotes

Acknowledgments

All experimental protocols were approved by the Institutional Animal Care and Use Committee of the study institute.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Industrial Strategic Technology Development Program (10051152) funded by the Ministry of Trade, Industry & Energy (MOTIE, Korea).

References

Organization WH. Global Health Estimates: Deaths by Cause, Age, Sex and Country, 2000–2012, Geneva: WHO, 2014, pp. 9–9.

Andersen

Olsen

Dehlendorff

, et al. Hemorrhagic and ischemic strokes compared: Stroke severity, mortality, and risk factors. Stroke 2009; 40: 2068–2072.

Bai

Yang

, et al. Anti‐inflammatory and neuroprotective effects of triptolide via the NF‐κB signaling pathway in a rat MCAO model. Anatomical Record 2016; 299: 256–266.

Durukan

Tatlisumak

. Acute ischemic stroke: Overview of major experimental rodent models, pathophysiology, and therapy of focal cerebral ischemia. Pharmacol Biochem Behav 2007; 87: 179–197.

Bederson

Pitts

Tsuji

, et al. Rat middle cerebral artery occlusion: Evaluation of the model and development of a neurologic examination. Stroke 1986; 17: 472–476.

Mhairi

. New models of focal cerebral ischaemia. Br J Clin Pharmacol 1992; 34: 302–308.

Oliveira

CBD

Medeiros

IRTD

Frota

NAF

, et al. Balance control in hemiparetic stroke patients: Main tools for evaluation. J Rehabil Res Dev 2008; 45: 1215–1226.

Whitall

Waller

Silver

, et al. Repetitive bilateral arm training with rhythmic auditory cueing improves motor function in chronic hemiparetic stroke. Stroke 2000; 31: 2390–2395.

Barth

Jones

Schallert

. Functional subdivisions of the rat somatic sensorimotor cortex. Behav Brain Res 1990; 39: 73–95.

10.

Hunter

Hatcher

Virley

, et al. Functional assessments in mice and rats after focal stroke. Neuropharmacology 2000; 39: 806–816.

11.

Chen

Wang

, et al. Treatment of stroke in rat with intracarotid administration of marrow stromal cells. Neurology 2001; 56: 1666–1672.

12.

Carmichael

. Rodent models of focal stroke: Size, mechanism, and purpose. NeuroRx 2005; 2: 396–409.

13.

Gharbawie

Whishaw

. The topography of three-dimensional exploration: A new quantification of vertical and horizontal exploration, postural support, and exploratory bouts in the cylinder test. Behav Brain Res 2004; 151: 125–135.

14.

Rogers

Campbell

Stretton

, et al. Correlation between motor impairment and infarct volume after permanent and transient middle cerebral artery occlusion in the rat. Stroke 1997; 28: 2060–2066.

15.

Allred

Jones

. Maladaptive effects of learning with the less-affected forelimb after focal cortical infarcts in rats. Exp Neurol 2008; 210: 172–181.

16.

Jiang

Zhu

, et al. Feasibility of delivering mesenchymal stem cells via catheter to the proximal end of the lesion artery in patients with stroke in the territory of the middle cerebral artery. Cell Transplant 2013; 22: 2291–2298.

17.

Miller

Garner

Mench

. The test–retest reliability of four behavioural tests of fearfulness for quail: A critical evaluation. Appl Anim Behav Sci 2005; 92: 113–127.

18.

Richards

Marder

Cote

, et al. Interrater reliability of the Unified Parkinson’s Disease Rating Scale motor examination. Move Disord 1994; 9: 89–91.

19.

Kim

. Reliability and responsiveness of the gross motor function measure-88 in children with cerebral palsy. Phys Ther 2013; 93: 393–400.

20.

Flansbjer

U-B

Holmbäck

Downham

, et al.

What change in isokinetic knee muscle strength can be detected in men and women with hemiparesis after stroke?

Clin Rehabil 2005; 19: 514–522.

21.

Basso

Beattie

Bresnahan

. A sensitive and reliable locomotor rating scale for open field testing in rats. J Neurotrauma 1995; 12: 1–21.

22.

Salomé

Viltart

Darnaudéry

, et al. Reliability of high and low anxiety-related behaviour: Influence of laboratory environment and multifactorial analysis. Behav Brain Res 2002; 136: 227–237.

23.

Y-M

Lin

J-H

Hsiao

S-F

, et al. The relative and absolute reliability of leg muscle strength testing by a handheld dynamometer. J Strength Condition Res 2011; 25: 1065–1071.

24.

Harris

Heer

Roy

, et al. Reliability of a measurement of neck flexor muscle endurance. Phys Ther 2005; 85: 1349–1349.

25.

Liaw

L-J

Hsieh

C-L

S-K

, et al. The relative and absolute reliability of two balance performance measures in chronic stroke patients. Disabil Rehabil 2008; 30: 656–661.

26.

Bland

Altman

. Measurement error and correlation coefficients. BMJ 1996; 313: 41–41.

27.

Beckerman

Roebroeck

Lankhorst

, et al. Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res 2001; 10: 571–578.

28.

Gibson

Bath

Murphy

. G-CSF reduces infarct volume and improves functional outcome after transient focal cerebral ischemia in mice. J Cerebr Blood Flow Metab 2005; 25: 431–439.

29.

Chen

Wang

, et al. Therapeutic benefit of intravenous administration of bone marrow stromal cells after cerebral ischemia in rats. Stroke 2001; 32: 1005–1011.

30.

Koizumi

Yoshida

Nakazawa

, et al. Experimental studies of ischemic brain edema, I: A new experimental model of cerebral embolism in rats in which recirculation can be introduced in the ischemic area. Jnp J Stroke 1986, pp. 8–8.

31.

Chen

Sanberg

, et al. Intravenous administration of human umbilical cord blood reduces behavioral deficits after stroke in rats. Stroke 2001; 32: 2682–2688.

32.

Hua

Schallert

Keep

, et al. Behavioral tests after intracerebral hemorrhage in the rat. Stroke 2002; 33: 2478–2484.

33.

Chen

Constantini

Trembovler

, et al. An experimental model of closed head injury in mice: Pathophysiology, histopathology, and cognitive deficits. J Neurotrauma 1996; 13: 557–568.

34.

Schallert

Kozlowski

Humm

, et al. Use-dependent structural events in recovery of function. Adv Neurol 1997; 73: 229–238.

35.

Chopp

Chen

, et al. Intrastriatal transplantation of bone marrow nonhematopoietic cells improves functional recovery after stroke in adult mice. J Cerebr Blood Flow Metab 2000; 20: 1311–1319.

36.

Schallert

Fleming

Leasure

, et al. CNS plasticity and assessment of forelimb sensorimotor outcome in unilateral rat models of stroke, cortical ablation, parkinsonism and spinal cord injury. Neuropharmacology 2000; 39: 777–787.

37.

Whishaw

Kolb

. The Behavior of the Laboratory Rat: A Handbook with Tests, Oxford: Oxford University Press, 2004.

38.

Chao

Pum

J-S

, et al. The grid-walking test: Assessment of sensorimotor deficits after moderate or severe dopamine depletion by 6-hydroxydopamine lesions in the dorsal striatum and medial forebrain bundle. Neuroscience 2012; 202: 318–325.

39.

Nunnally

. Psychometric Theory, 2nd ed. New York, NY: McGraw-Hill, 1978.

40.

Damstra

Slater

JJH

Fourie

, et al. Reliability and the smallest detectable differences of lateral cephalometric measurements. Am J Orthodont Dentofac Orthoped 2010; 138: 546, e1–e8.

41.

Atkinson

Nevill

. Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Med 1998; 26: 217–238.

42.

Bland

Altman

. Statistics notes: Measurement error. BMJ 1996; 313: 744–744.

43.

Cui

L-l

Golubczyk

Jolkkonen

. Top 3 behavioral tests in cell therapy studies after stroke: Difficult to stop a moving train. Stroke 2017; 48: 3165–3167.

44.

Bouët

Freret

Toutain

, et al. Sensorimotor and cognitive deficits after transient middle cerebral artery occlusion in the mouse. Exp Neurol 2007; 203: 555–567.

45.

Chang

Wendland

, et al. Erythropoietin improves functional and histological outcome in neonatal stroke. Pediatr Res 2005; 58: 106–106.

46.

Juul

McPherson

Bauer

, et al. A phase I/II trial of high-dose erythropoietin in extremely low birth weight infants: Pharmacokinetics and safety. Pediatrics 2008; 122: 383–391.

47.

Priest

. Essentials of nursing research: Methods, appraisal, and utilization. Nurse Res 2006; 13: 91–93.

48.

Maher

Sherrington

Herbert

, et al. Reliability of the PEDro scale for rating quality of randomized controlled trials. Phys Ther 2003; 83: 713–721.