Test–Retest Reliability,Criterion-Related Validity,and Ecological Validity of the Test of Visual–Motor Skills,Third Edition,in Kindergarten Children With Developmental Coordination Disorder

Abstract

Importance: Establishing empirical evidence on the psychometric properties of the Test of Visual–Motor Skills (3rd ed.; TVMS–3) is helpful for guiding its use as an assessment of visual–motor integration (VMI) skills in kindergarten children with developmental coordination disorder (DCD).

Objective: To investigate the test–retest reliability, criterion-related validity, and ecological validity of the TVMS–3 in Taiwanese kindergarten children with DCD.

Design: A nonexperimental, descriptive, correlational design.

Setting: A hospital in Central Taiwan.

Participants: Fifty-seven kindergarten children with DCD were recruited in the study.

Outcomes and Measures: Intraclass correlation coefficient, percentage of minimal detectable change, and paired t test (Wilcoxon signed rank test) were used to investigate the test–retest reliability of the TVMS–3. The correlations (Pearson’s r) between the TVMS–3 accuracy score and the scores of each of the four domains and the adaptive behavior composite score of the Vineland Adaptive Behavior Scales (3rd ed.; Vineland–3) were calculated, respectively, to examine criterion-related validity and ecological validity.

Results: The accuracy score of the TVMS–3 had excellent test–retest reliability and acceptable random measurement error. Moreover, it showed good criterion-related validity and sufficient ecological validity with the Vineland–3 in Taiwanese kindergarten children with DCD.

Conclusions and Relevance: The accuracy score of the TVMS–3 is applicable to Taiwanese kindergarten children with DCD in clinical and research settings.

Plain-Language Summary: The accuracy score of the Test of Visual–Motor Skills (3rd ed.; TVMS–3) is a useful assessment tool to detect deficits in visual–motor integration for Taiwanese kindergarten children with developmental coordination disorder. The TVMS–3 has excellent test–retest reliability, good criterion-related validity, and sufficient ecological validity.

The Test of Visual–Motor Skills is a useful assessment tool to detect deficits in visual–motor integration for Taiwanese kindergarten children with developmental coordination disorder.

Developmental coordination disorder (DCD) is a neurodevelopmental disorder primarily characterized by motor coordination impairments that result in performance difficulties in daily life activities and academic settings (American Psychiatric Association, 2013). Such motor coordination difficulties cannot be explained by intellectual disability or any specific congenital or acquired neurologic condition (American Psychiatric Association, 2013). DCD generally occurs in the early developmental period and affects about 5% to 6% of school-age children; however, it is largely underrecognized by health care and education professionals (American Psychiatric Association, 2013; Blank et al., 2012, 2019; Lee & Zwicker, 2021).

Visual-motor integration (VMI) is defined as the degree of coordination between visual perception and upper extremity movement when performing a task (Beery & Beery, 2010; Valverde et al., 2020). Occupational therapists often assess the VMI skills of children because VMI is crucial for participating in school- related tasks, such as handwriting, using scissors, and riding a bicycle or tricycle (Carsone et al., 2021). It has been reported that poor VMI performance was found in preschool and school-age children with DCD in both Western countries such as the Netherlands (Lust et al., 2022; Van Hoorn, Maathuis, Peters, & Hadders-algra, 2010) and Eastern countries such as Taiwan (Hsu & Tsai, 2009; Shen, 2010), which may be the cause of difficulties in school-related tasks in children with DCD (Biotteau et al., 2019; Rodger et al., 2003; Wang et al., 2009). Additionally, a previous imaging study also showed that the white matter deficits and altered structural connectivity found in the brains of children with DCD are associated with their VMI deficits (Debrabant et al., 2016). Therefore, it is essential to assess VMI in children with DCD, especially in earlier stages, to determine the deficits and further provide proper interventions.

The Test of Visual–Motor Skills (3rd ed.; TVMS–3; Martin, 2010) was developed to detect deficits in VMI by assessing how well a person (ages 3 to ≥90 yr) can coordinate visually guided fine motor movements while copying a design that is in sight. Compared with another commonly used measure of VMI for children—the Beery-Buktenica Developmental Test of Visual–Motor Integration (6th ed.; BEERY™ VMI; Beery & Beery, 2010)—the TVMS–3 is more suitable for children with DCD for a critical reason: The scoring of the TVMS–3 includes analysis of the nine types of errors in the copied design (Beery & Beery, 2010), which is not provided by the Beery VMI. Accordingly, by using the TVMS–3, clinicians can gather more information regarding how motor difficulties in children with DCD may affect their VMI performance. As a result, the TVMS–3 has great potential for use in children with DCD, especially in the early developmental period (i.e., kindergarten period), to provide better management in the future.

Regarding the psychometric properties of the TVMS–3, it shows good construct validity and good discriminant validity to differentiate between children with attention difficulties and/or learning disabilities and typically developing children (TD children; Martin, 2010). Furthermore, some evidence found that the reliability of the TVMS–3 was sufficient (i.e., high internal consistency and small random measurement error) in TD children ages 4 and 5 yr (Martin, 2010). However, this evidence was established on children in the United States only. No evidence on the psychometric properties of this measure has been gathered from TD children or children with special needs (e.g., DCD) in Eastern countries (e.g., Taiwan). Such a limitation can restrict the utility of the TVMS–3 in clinical and research settings across countries.

Test–retest reliability reflects the extent of consistency between repeated assessments that are free from random measurement error (Portney & Watkins, 2009). Ideally, a measure of VMI with high test–retest reliability and small random measurement error ensures the precision of the results in repeated assessments (Portney & Watkins, 2009). In addition, criterion-related validity states how closely the evaluated test relates to another existing instrument (criterion) that measures the same (or similar) constructs. Ideally, tests measuring the same or similar constructs should be highly correlated (Portney & Watkins, 2009). A measure of VMI with good criterion-related validity is assumed to be useful for determining deficits in children’s VMI performance. Furthermore, ecological validity refers to the degree to which test performance corresponds to real-world performance (Chaytor & Schmitter-Edgecombe, 2003). Evidence on the ecological validity of the TVMS–3 can assist clinicians in interpreting test findings with regard to the predictive relationship between a child’s performance on a measure of VMI and their behavior (e.g., adaptive behavior) in a variety of real-world settings (Portney & Watkins, 2009). Because impaired VMI is a possible problem in children with DCD (Van Hoorn, Maathuis, Peters, & Hadders-algra, 2010), which may lead to deficit adaptive behavior (Biotteau et al., 2019), an ecologically valid measure of VMI for children with DCD is useful for predicting how well they can function in the real world. It is critical for the TVMS–3 to have sufficient test–retest reliability, criterion-related validity, and ecological validity in kindergarten children with DCD to identify their VMI deficits and plan further treatment.

Because psychometric properties are generally sample dependent (Hobart & Cano, 2009), the purpose of this study was to investigate the test–retest reliability, criterion-related validity, and ecological validity of the TVMS–3 in kindergarten children with DCD in Taiwan. The results of this study should also be helpful for establishing new evidence regarding the utility of the TVMS–3 in clinical and research settings.

Method

Participants

For this study, a convenience sample was recruited from a pool of kindergarten children who received assessments for early intervention in a hospital in Yunlin, Taiwan, from June 2, 2018, to January 30, 2021. The inclusion criteria were as follows:

▪ age between 4 and 6 yr

▪ the presence of significant movement difficulties with a high probability of DCD (at or below the fifth percentile, as identified by the Movement Assessment Battery for Children, Second Edition (Movement ABC-2)

▪ enrollment in the regular education system without evidence of highlighted delay, as well as cognitive development within the expected range for the chronological age, according to the report of caregivers.

Children who had any past medical history of neurologically related diagnoses, such as attention deficits, as reported by their caregivers, were excluded. Additionally, because the recommended sample size for examining test–retest reliability is 50 (Atkinson & Nevill, 2000; Hopkins, 2000), we recruited at least 50 kindergarten children with DCD.

The study protocol was approved by the St. Joseph’s Hospital Institutional Review Board (IRB; approval/application number CCH IRB No.171220). We obtained informed consent from the caregivers and the assent of the participating children before inclusion.

Procedure

First, we assessed the kindergarten children individually with the Movement ABC–2, and their caregivers were interviewed to ensure that all participants in the study met the inclusion criteria. Participants who met the inclusion criteria were then individually assessed with the TVMS–3 twice (with a 2-wk interval) by the same rater. The Vineland Adaptive Behavior Scales, Third Edition (Vineland–3; Sparrow et al., 2016), were administered through in-person interviews with the caregivers after the TVMS–3 was administered at the first time point. Two specially trained occupational therapists were in charge of administering the Movement ABC–2, TVMS–3, and Vineland–3.

Measures

TVMS–3

The TVMS–3 was designed to assess the integration of visual–perceptual, motor planning, and motor execution abilities during the execution of one skill: copying (Martin, 2010). Examinees were asked to copy a series of 39 designs (arranged in a developmental sequence from easiest to most difficult) one by one until they failed to produce recognizable designs four times in a row. Each copied design was scored on a 3-point scale (0 = not recognizable, 1 = one or more types of errors made, and 2 = correctly copied) in relation to how accurately it was rendered. One overall accuracy score (total raw score) was then derived from the sum of the individual scores. The total raw score can be further transformed into a standard score (M = 100, SD = 15) and percentile rank. A score below 1 SD (85) indicated below-average VMI performance. In this study, the standard score was used in the examination of test–retest reliability for the TVMS–3 accuracy score. In addition, the scoring of the TVMS–3 also included the analysis of the nine types of errors in the copied design. The nine types of errors are as follows:

▪ incorrect closures

▪ incorrect angles

▪ line quality

▪ line lengths

▪ line connections

▪ modification of a size or a part

▪ addition or deletion of a part

▪ rotation or reversal

▪ shape overlap error.

Because of the different types of errors, nine separate error scores are provided, with those in the lowest quartile (25th percentile and below) indicative of the need for remediation (demonstrating deficits in VMI). In this study, the nine separate error scores were also analyzed for the test–retest reliability of the TVMS–3.

Vineland–3

The Vineland–3 is a comprehensive adaptive behavior measure that comprises four core domains (and 11 subdomains):

▪ communication (receptive, expressive, and written adaptive language functions)

▪ daily living skills (personal, domestic, and community skills)

▪ socialization (interpersonal relationships, play and leisure time, and coping abilities)

▪ motor skills (gross and fine motor skills; Sparrow et al., 2016).

All domains except the motor skills domain are included in an overall score, referred to as the adaptive behavior composite. The Vineland–3 has shown excellent internal consistency and acceptable test–retest reliability (Sparrow et al., 2016). In the present study, the adaptive behavior composite score obtained from the Vineland–3 (representing the kindergarten children’s adaptive behavior) was used as a criterion to examine the ecological validity of the TVMS–3.

Data Analysis

Normality of the Distribution

The normality of the distribution of the accuracy score and error scores at test and retest were examined by using the one-sample Kolmogorov–Smirnov test.

Test–Retest Reliability

The test–retest reliability of the accuracy score and nine separate error scores of the TVMS–3 were determined through the calculation of the intraclass correlation coefficient (ICC) between the first and the second assessments, respectively, on the basis of a two-way random-effects model with absolute agreement (Lexell & Downham 2005). ICC values greater than .90 indicated excellent reliability, .75 to .90 indicated good reliability, .74 to .50 indicated moderate reliability, and less than .50 indicated poor reliability (Koo & Li, 2016).

We also calculated the standard error of measurement (SEM) and the minimal detectable change (MDC) on the basis of the ICC, with Equations 1 and 2: $S E M = S D_{all testing scores} \times \sqrt{1 - I C C}$ (1) $M D C = 1.96 \times \sqrt{2} \times S E M$ (2)

In these two equations, SD represents the standard deviation of all scores of the first and the second assessments; the ICC value was obtained from the test–retest reliability; 1.96 is the z score at the 95% confidence level; and the multiplier of $\sqrt{2}$ indicates the additional uncertainty caused by the use of different scores from measurements at two time points (Haley & Fragala-Pinkham, 2006).

We calculated the MDC percentage (MDC%) as shown in Equation 3. The MDC% represents the relative amount of random measurement error. An MDC% below 30% was considered an acceptable error, and less than 10% was considered excellent for a measure (Smidt et al., 2002). $M D C % = \frac{M D C}{the mean score of all the trials} \times 100 %$ (3)

We also examined whether a practice effect was present between the first and second assessments. We used the paired t test (or Wilcoxon signed rank test for nonparametric data) to determine the significance of the change scores (practice effect), and we used the effect size to determine the magnitude of the change scores (practice effect). The effect size was calculated as the mean change score between the first and the second assessments divided by the SD of the first assessment (Kazis et al., 1989). Effect sizes of .2, .5, and .8 reflected small, medium, and large practice effects, respectively, in terms of Cohen’s criteria (Cohen, 2013).

Criterion-Related Validity

The criterion-related validity of the TVMS–3 was examined by correlating the TVMS–3 accuracy score at the first assessment with the scores of each of the four domains (i.e., communication, daily living skills, socialization, and motor skills) of the Vineland–3 using Pearson’s r (or Spearman’s ρ for nonparametric data). We hypothesized that the motor skills domain of the Vineland–3 would correlate more highly than the other domains (Portney & Watkins, 2009).

Ecological Validity

The ecological validity of the TVMS–3 was assessed by examining the relationship between the TVMS–3 accuracy score and the adaptive behavior composite score in the Vineland–3 at the first time point. The correlation between the TVMS–3 accuracy score and the adaptive behavior composite score in the Vineland–3 was assumed to be around moderate (Pearson’s r or Spearman’s ρ = .50; Hinkle et al., 2003), indicating good ecological validity (Jarus et al., 2011).

Results

Demographic and Clinical Characteristics of Participants

We approached 104 kindergarten children; among them, 47 kindergarten children did not meet our inclusion criteria and were excluded. Finally, 57 participants were recruited and completed both assessments of the TVMS–3.

Table 1 presents the characteristics of the participants (M age = 4.4 yr; 64.9% were male). Moreover, around 37% of the participants demonstrated VMI deficits (standard score was below average). However, no participants demonstrated VMI deficits on the basis of the results of error analysis.

Table 1.

Characteristics of the Participants

Characteristic	n (%) or M (SD)
Gender, n (%)
Male	37 (64.9)
Female	20 (35.1)
Age, yr, M (SD)	4.4 (0.3)
TVMS–3 baseline raw score, M (SD)	15.5 (7.8)
Proportion of participants with VMI deficits identified by the accuracy score of the TVMS–3, n (%)	21 (36.8)

Note. n = 57. TVMS–3 = Test of Visual–Motor Skills, Third Edition; VMI = visual–motor integration.

Normality of the Distribution

The normality of the distribution for the accuracy scores at test and retest, respectively, were not rejected at the .05 significance level, as determined by using the one-sample Kolmogorov–Smirnov test. However, the normality of the distribution for most of the error scores at test and retest, respectively, were rejected at the .05 significance level.

Test–Retest Reliability

Accuracy Score

The ICC for the TVMS–3 accuracy score between the two assessments was .92, with a 95% confidence interval (CI) of .86 to .95. The value of MDC (MDC%) for the TVMS–3 accuracy score was 10.2 points (11.6%).

On the basis of the examination results of the normality of distribution for the accuracy score, we chose the paired t test to determine whether practice effect was present between repeated assessments. The paired t test showed that the TVMS–3 accuracy score was not significantly different at test and retest (p = .063). The absolute effect size between the two assessments of the TVMS–3 was .10.

Error Scores

Table 2 presents the ICC, MDC, MDC%, and p values and effect sizes for the error scores. The ICCs for the nine separate error scores of the TVMS–3 ranged from .58, 95% CI [.38, .73] to .92, 95% CI [.86, .95]. The MDC values for the nine separate error scores of the TVMS–3 ranged from 1.1 to 3.1 (number of errors); the MDC% values for the nine separate error scores of the TVMS–3 ranged from 42.7% to 312.4%.

Table 2.

Test–Retest Reliability Indices of the Error Scores of the Test of Visual–Motor Skills, Third Edition

Type of Error	ICC [95% CI]	SEM	MDC	MDC%	p Value From Wilcoxon Signed Rank Test	Effect Size
1. Incorrect closures	.91 [.84, .94]	1.10	3.04	42.7	.048	.08
2. Incorrect angles	.86 [.78, .92]	1.55	4.29	57.5	.137	.08
3. Line quality	.63 [.44, .76]	0.41	1.125	312.4	.822	.03
4. Line lengths	.82 [.72, .89]	1.47	4.09	66.0	.206	.10
5. Line connections	.92 [.86, .95]	0.85	2.36	66.4	.025	.12
6. Modification of a size or part	.86 [.77, .92]	1.70	4.71	64.7	.017	.16
7. Addition or deletion of a part	.72 [.57, .83]	0.49	1.35	179.9	.858	.02
8. Rotation or reversal	.58 [.38, .73]	1.07	2.97	220.4	.341	.02
9. Shape overlap error	.75 [.61, .85]	1.00	2.78	143.8	.061	.10

Note. n = 57. CI = confidence interval; ICC = intraclass correlation coefficient; MDC = minimal detectable change; MDC% = MDC percentage.

On the basis of the examination results of the normality of distribution for the error scores, we chose the Wilcoxon signed rank test to determine whether a practice effect was present between repeated assessments. The Wilcoxon signed rank test showed that most of the error scores of the TVMS–3 were not significantly different at test and retest (p $\geq$ .061), except for the scores of Errors 1, 5, and 6.

Criterion-Related Validity

The motor skills domain of the Vineland–3 had the highest correlation with the accuracy score of the TVMS–3 (Pearson’s r = .44, p < .01), compared with the other domains (communication: Pearson’s r = .41, p < .01; daily living skills: Pearson’s r = .37, p < .01; socialization: Pearson’s r = .34, p < .05).

Ecological Validity

The correlation between the TVMS–3 accuracy score and the adaptive behavior composite score on the Vineland–3 was around moderate (Pearson’s r = .47, p < .01).

Discussion

The results of this study showed that the ICC (.92) for the TVMS–3 accuracy score was high, indicating that the TVMS–3 has excellent test–retest reliability over a 2-wk interval when the accuracy score was applied. However, our results showed that the ICCs for the nine separate error scores of the TVMS–3 were moderate to high (ranging from .58 to .92), demonstrating different levels of test–retest reliability among these error scores. Because ICCs represent the extent of consistency between two assessments (Portney & Watkins, 2009), our results suggest that the TVMS–3 accuracy score, compared with its error scores, produces more consistent results over repeated assessments on the same subject under identical conditions (i.e., same participant, same rater, and similar conditions). Moreover, the commonly cited minimal standards for reliability coefficients (e.g., ICC values) are .70 for group comparisons and .90 for individual comparisons (Aaronson et al., 2002). Therefore, the TVMS–3 accuracy score is more useful than its error scores when applied in clinical and research settings.

It is becoming clear that the use of only ICCs for the analysis of test–retest reliability is not sufficient (Portney & Watkins, 2009). In the present study, we also calculated the values of MDC and MDC%. Our results showed that the MDC value (10.2 points) for the TVMS–3 accuracy score was <30% of the mean of all scores from the two assessments, indicating acceptable random measurement error between the test and retest assessments. However, the MDC values for the nine separate error scores of the TVMS–3 were all >30% of the mean of all scores from the two assessments, indicating substantial random measurement error between the test and retest assessments. Because it is highly unlikely that a child’s VMI performance could improve by more than 30% in a short period (i.e., a 2-wk interval), the high MDC% value indicates that the error scores of the TVMS–3 are unstable. Therefore, to avoid over- or underestimating a real change in VMI performance in kindergarten children with DCD, the TVMS–3 accuracy score is recommended for use in clinical and research settings when monitoring the change in a child’s VMI performance.

The calculation of MDC value is beneficial to clinicians and researchers, because it provides a threshold for determining a real change in a patient (Haley & Fragala-Pinkham, 2006). For example, a change score exceeding the MDC value can be interpreted as a real change with the corresponding certainty (e.g., 95%). Furthermore, to apply the MDC value appropriately, practice effect should be considered (Koh et al., 2011). Our results showed that the value of MDC for the TVMS–3 accuracy score was 10.2 points. In addition, no practice effect was found between repeated assessments when the TVMS–3 accuracy score was applied. These findings mean that only a change >10.2 points in the TVMS–3 accuracy score between repeated assessments can be interpreted as a real change with 95% certainty. Not only is such a value useful for clinicians to identify whether the change in the score of a kindergarten child with DCD is a real improvement, but it is also useful for researchers who are conducting efficacy studies to identify the effectiveness of interventions by calculating the percentage of participants with change scores larger than the MDC value (MDC proportion).

We found that the motor skills domain of the Vineland–3 had the highest correlation with the accuracy score of the TVMS–3, compared with the other domains, supporting our hypothesis. Thus, good criterion-related validity was demonstrated for the accuracy score of the TVMS–3. Our results do support the use of the TVMS–3 accuracy score for assessing VMI performance in kindergarten children with DCD in Taiwan.

Our results also demonstrated that the accuracy score of the TVMS–3 had a moderate correlation with the information on adaptive behavior obtained from the Vineland–3. This finding indicates that the TVMS–3 is an ecologically valid measure of VMI that can predict adaptive behavior in kindergarten children with DCD. Thus, the TVMS–3 accuracy score is useful for clinicians to understand whether kindergarten children with DCD can function effectively in daily life.

Five limitations should be noted in our study. First, a convenience and homogeneous sample (i.e., kindergarten children living in Central Taiwan) was adopted in our study. This might have limited the generalizability of our findings. Second, we recruited only kindergarten children (i.e., ages 4–6 yr) with DCD in this study. Thus, the results of this study (e.g., MDC value) are not suitable for children with DCD at different ages (e.g., school-age). Third, less than a half of the participants demonstrated VMI deficits on the basis of the accuracy score results. In general, children with DCD demonstrate heterogeneous patterns of performance (Lust et al., 2022). Future research may recruit more participants with VMI deficits to validate our findings. Fourth, only the Vineland–3 composite score was used to determine ecological validity in this study. Such limited evidence might not be robust enough to describe the ecological validity of the TVMS–3. Future studies adopting other criterion measures (e.g., The Functional Independence Measure for Children) are needed to improve the utility of the TVMS–3 in clinical and research settings. Fifth, we adopted the original norms of the TVMS–3 in this study to identify the VMI performance of our participants, because specific norms for children living in Taiwan have not been developed. Further research is required to provide Taiwanese norms so that the findings of this study can be confirmed.

Implications for Occupational Therapy Practice

The results of this study have the following implications for occupational therapy practice:

▪ The findings of our study support the use of the TVMS–3 accuracy score for assessing VMI skills in kindergarten children with DCD in Taiwan. This assessment tool has the potential to support the provision of timely intervention targeting VMI deficits in young Taiwanese children with DCD before secondary sequelae develop.

▪ The MDC value for the TVMS–3 accuracy score is useful for future assessments in Taiwan, not only to determine whether the change score of an individual is beyond random measurement error but also to determine the treatment efficacy of VMI training programs by calculating the MDC proportion.

Conclusion

Our results indicate that the test–retest reliability, criterion-related validity, and ecological validity of the TVMS–3 accuracy score is sufficient for assessing Taiwanese kindergarten children with DCD in clinical and research settings. However, all nine separate error scores showed substantial random measurement error, and some of them demonstrated practice effect. Accordingly, we recommend that clinicians and researchers use the accuracy score to identify the VMI performance of kindergarten children with DCD in Taiwan. Moreover, the MDC value for the TVMS–3 accuracy score is useful for future assessments in Taiwan to determine whether the change score is beyond random measurement error.

Footnotes

Acknowledgments

We are grateful to all participants for their involvement. This study was supported by St. Joseph’s Hospital (Grant no. 10703). En-Chi Chiu and Wan-Hui Yu contributed to this article equally.

References

Aaronson

Alonso

Burnam

Lohr

K. N.

Patrick

D. L.

Perrin

, & Stein

R. E.

(2002). Assessing health status and quality-of-life instruments: attributes and review criteria. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 11, 193–205. https://doi.org/10.1023/A:1015291021312

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). https://doi.org/10.1176/appi.books.9780890425596

Atkinson

, & Nevill

(2000). Typical error versus limits of agreement. Sports Medicine, 30, 375–381. https://doi.org/10.2165/00007256-200030050-00005

Beery

, & Beery

(2010). The Beery Buktenica Developmental Test of Visual–Motor Integration. Pearson.

Biotteau

Danna

Baudou

É.

Puyjarinet

Velay

J. L.

Albaret

J. M.

, & Chaix

(2019). Developmental coordination disorder and dysgraphia: Signs and symptoms, diagnosis, and rehabilitation. Neuropsychiatric Disease and Treatment, 15, 1873–1885. https://doi.org/10.2147/NDT.S120514

Blank

Barnett

A. L.

Cairney

Green

Kirby

Polatajko

et al. Vinçon, S. (2019). International clinical practice recommendations on the definition, diagnosis, assessment, intervention, and psychosocial aspects of developmental coordination disorder. Developmental Medicine and Child Neurology, 61, 242–285. https://doi.org/10.1111/dmcn.14132

Blank

Smits-Engelsman

Polatajko

, & Wilson

; European Academy for Childhood Disability .(2012). European Academy for Childhood Disability (EACD): Recommendations on the definition, diagnosis and intervention of developmental coordination disorder (long version). Developmental Medicine and Child Neurology, 54, 54–93. https://doi.org/10.1111/j.1469-8749.2011.04171.x

Carsone

Green

Torrence

, & Henry

(2021). Systematic review of visual motor integration in children with developmental disabilities. Occupational Therapy International, 2021, 1801196. https://doi.org/10.1155/2021/1801196

Chaytor

, & Schmitter-Edgecombe

(2003). The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills. Neuropsychology Review, 13, 181–197. https://doi.org/10.1023/B:NERV.0000009483.91468.fb

10.

Cohen

(2013). Statistical power analysis for the behavioral sciences. Academic Press. https://doi.org/10.4324/9780203771587

11.

Debrabant

Vingerhoets

Van Waelvelde

Leemans

Taymans

, & Caeyenberghs

(2016). Brain connectomics of visual-motor deficits in children with developmental coordination disorder. Journal of Pediatrics, 169, 21–7.e2. https://doi.org/10.1016/j.jpeds.2015.09.069

12.

Haley

S. M.

, & Fragala-Pinkham

M. A.

(2006). Interpreting change scores of tests and measures used in physical therapy. Physical Therapy, 86, 735–743. https://doi.org/10.1093/ptj/86.5.735

13.

Hinkle

D. E.

Wiersma

, & Jurs

S. G.

(2003). Applied statistics for the behavioral sciences (Vol. 663). Houghton Mifflin College Division.

14.

Hopkins

W. G.

(2000). Measures of reliability in sports medicine and science. Sports Medicine, 30, 1–15. https://doi.org/10.2165/00007256-200030010-00001

15.

Hsu

Y. W.

, & Tsai

C. L.

(2009). Visual perception and visual-motor integration in children with developmental coordination disorder. Formosa Journal of Mental Health, 22, 111–138.

16.

Hutcheon

J. A.

Chiolero

, & Hanley

J. A.

(2010). Random measurement error and regression dilution bias. BMJ, 340, c2289. https://doi.org/10.1136/bmj.c2289

17.

Jarus

Lourie-Gelberg

Engel-Yeger

, & Bart

(2011). Participation patterns of school-aged children with and without DCD. Research in Developmental Disabilities, 32, 1323–1331. https://doi.org/10.1016/j.ridd.2011.01.033

18.

Kazis

L. E.

Anderson

J. J.

, & Meenan

R. F.

(1989). Effect sizes for interpreting changes in health status. Medical Care, 27(Suppl.), S178–S189. https://doi.org/10.1097/00005650- 198903001-00015

19.

Koh

C. L.

W. S.

Chen

H. C.

Hsueh

I. P.

Hsieh

J. J.

, & Hsieh

C. L.

(2011). Test–retest reliability and practice effect of the oral-format Symbol Digit Modalities Test in patients with stroke. Archives of Clinical Neuropsychology, 26, 356–363. https://doi.org/10.1093/arclin/acr029

20.

Koo

T. K.

, & Li

M. Y.

(2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15, 155–163. https://doi.org/10.1016/j.jcm.2016.02.012

21.

Lee

E. J.

, & Zwicker

J. G.

(2021). Early identification of children with/at risk of developmental coordination disorder: A scoping review. Developmental Medicine and Child Neurology, 63, 649–658. https://doi.org/10.1111/dmcn.14803

22.

Lexell

J. E.

, & Downham

D. Y.

(2005). How to assess the reliability of measurements in rehabilitation. American Journal of Physical Medicine and Rehabilitation, 84, 719–723. https://doi.org/10.1097/01.phm.0000176452.17771.20

23.

Lust

J. M.

Steenbergen

Diepstraten

J. A. E. M.

Wilson

P. H.

Schoemaker

M. M.

, & Poelma

M. J.

(2022). The subtypes of developmental coordination disorder. Developmental Medicine and Child Neurology, 64, 1366–1374. https://doi.org/10.1111/dmcn.15260

24.

Martin

N. A.

(2010). Test of Visual Motor Skills (3rd ed.). Western Psychological Services.

25.

Portney

L. G.

, & Watkins

M. P.

(2009). Foundations of clinical research: Applications to practice. Prentice Hall.

26.

Rodger

Ziviani

Watter

Ozanne

Woodyatt

, & Springfield

(2003). Motor and functional skills of children with developmental coordination disorder: A pilot investigation of measurement issues. Human Movement Science, 22, 461–478. https://doi.org/10.1016/j.humov.2003.09.004

27.

Shen

C. Y.

(2010). The study of visual motor ability of children with developmental coordination disorder. [Unpublished master’s thesis]. National Yang-Ming University.

28.

Smidt

van der Windt

D. A.

Assendelft

W. J.

Mourits

A. J.

Devillé

W. L.

de Winter

A. F.

, & Bouter

L. M.

(2002). Interobserver reproducibility of the assessment of severity of complaints, grip strength, and pressure pain threshold in patients with lateral epicondylitis. Archives of Physical Medicine and Rehabilitation, 83, 1145–1150. https://doi.org/10.1053/apmr.2002.33728

29.

Sparrow

Cicchetti

, & Saulnier

(2016). Vineland Adaptive Behavior Scales –Third Edition (Vineland-3). American Guidance Service.

30.

Valverde

A. A.

Araújo

C. R. S.

Magalhães

L. C.

, & Cardoso

A. A.

(2020). Relationship between visual-motor integration and manual dexterity in children with developmental coordination disorder. Cadernos Brasileiros de Terapia Ocupacional, 28, 890–899. https://doi.org/10.4322/2526-8910.ctoAO1999

31.

van Hoorn

J. F.

Maathuis

C. G.

Peters

L. H.

, & Hadders-Algra

(2010). Handwriting, visuomotor integration, and neurological condition at school age. Developmental Medicine and Child Neurology, 52, 941–947. https://doi.org/10.1111/j.1469-8749. 2010.03715.x

32.

Wang

T. N.

Tseng

M. H.

Wilson

B. N.

, & Hu

F. C.

(2009). Functional performance of children with developmental coordination disorder at home and at school. Developmental Medicine and Child Neurology, 51, 817–825. https://doi.org/10.1111/j.1469-8749.2009.03271.x