Development of an Objective Structured Clinical Examination Station for Pediatric Occupational Therapy and an Evaluation of Its Quality

Abstract

Importance: The Objective Structured Clinical Examination (OSCE) is a highly valued measure of students’ clinical competencies in medical education. However, few studies have reported on the administration of the OSCE in pediatric occupational therapy education.

Objective: To describe the development of a pediatric occupational therapy OSCE station to evaluate students’ use of a standardized assessment and examine its standard setting, failure rates, and psychometric properties.

Design: Prospective, cross-sectional, observational study design.

Setting: Three OSCE stations in a university clinical skills center.

Participants: Five experienced occupational therapists, 60 examinees, 44 child standardized patients, 44 chaperones, and 15 examiners.

Measures: The sum of the rating scale and the global performance scores were used. The rating scale measured the examinee’s clinical competences in administering a standardized assessment. The 5-point global performance score was used to evaluate the examinee’s whole performance.

Results: The OCSE station’s expert validity was acceptable (item-level content validity index [CVI] = 0.8–1.0; scale-level CVI = 0.98). Passing scores according to the Angoff method (passing score = 14) and the contrasting-groups M–SD method (passing score = 13) were similar. Failure rates were high (61.7%–73.3%). Internal consistency was acceptable (Cronbach’s α = .78). No significant examiner effect was found (p = .554), and interexaminer reliability was acceptable (item score = 0.58–1.00; sum of the rating scale score = 0.97; global performance score = 0.79).

Conclusions and Relevance: The OSCE station for using a standardized assessment is a reliable and valid measure of students’ interpersonal communication skills and assessment skills.

What This Article Adds: The OSCE for education in pediatric occupational therapy is both effective and rigorous.

The OSCE station for using a standardized assessment is a reliable and valid measure for evaluating students’ interpersonal communication skills and assessment skills.

One primary goal of occupational therapy departments is to educate students to become qualified occupational therapy practitioners, which requires professional knowledge, skills, and attitudes. The use of an appropriate assessment to evaluate these competencies has been a matter of concern (Lee, 2013). Among the methods of assessment, the Objective Structured Clinical Examination (OSCE) is highly valued in medical education because of its reliable, effective, and holistic evaluation of students’ clinical knowledge, skills, and attitudes (Harden & Gleeson, 1979; Lee, 2012, 2013).

An OSCE is a performance-based test that consists of a circuit of short stations at which students are examined individually, with one examiner and one standardized patient (SP). A detailed script is provided to the SP to ensure that consistent information is given to the examinee. The rating scales to assess students’ performance are also designed through a standardized procedure to establish objective and reliable evaluation. The OSCE examines “showing how to do” instead of “knowing how to do” (Harden, 1988; Lee, 2012, 2013).

Administration of an OSCE in two specialty areas of occupational therapy education has previously been explored with patients with physical disabilities (Feng et al., 2016; Jaywant & Pai, 2009; Kanada et al., 2012, 2015; Sakurai et al., 2013a, 2013b, 2014a, 2014b) and patients with mental health disorders (Hung & Lin, 2013; Pan & Liu, 2017). However, only one study has reported on the administration of an OSCE in pediatric occupational therapy education (Fu et al., 2017). Although our previous study (Fu et al., 2017) supported the administration of an OSCE in pediatric occupational therapy education, many relevant issues still remain unclear. For example, appropriate use of standardized assessments is a core competency of pediatric occupational therapists. However, few studies have examined how to design an OSCE station to evaluate students’ use of a standardized assessment. Moreover, examination of the post hoc psychometrics of a rating scale used to evaluate students’ performance at an OSCE station is critical to improving the OSCE’s quality (Khan et al., 2013; Pell et al., 2010; Tavakol & Dennick, 2011, 2012).

The objective of our study was to provide more information about the application of an OSCE station in pediatric occupational therapy education. Specifically, our purposes were to (1) describe the development of an OSCE station to evaluate examinees’ clinical competency in using a standardized assessment; (2) examine the standard setting and failure rates of the rating scale used at the OSCE station; and (3) investigate the psychometrics of the rating scale, including its expert validity, internal consistency, and interexaminer reliability.

Method

Development of the OSCE Station

The OSCE station for evaluating clinical competency in using a standardized assessment was developed according to the Association for Medical Education in Europe Guide No. 81, Part 2 (Khan et al., 2013). First, a committee of six experts was established, consisting of an assistant professor of a Department of Occupational Therapy (Chung-Pei Fu), four experienced pediatric occupational therapists (including Hsin-Yu Chi and Ming-Wei Li), and the director of a clinical skills center (Jiann-Horng Yeh). These experts met to establish a blueprint for the examination and the number, duration, topics, and types of stations. They determined to include a 15-min OSCE station to evaluate students’ clinical competency in using parts of a standardized assessment, the Peabody Developmental Motor Scales–Second Edition (PDMS–2; Folio & Fewell, 2000).

Second, a pediatric occupational therapist (Ming-Wei Li) with 16 yr of experience wrote instructions on how to apply the standardized assessment in the OSCE station (see the Supplemental Appendix, available online with this article at https://research.aota.org/ajot). Third, the expert committee reviewed and modified the OSCE station. Fourth, five pediatric occupational therapists with an average of 12.6 yr of experience rated the fitness (including clarity, concreteness, centrality, and importance) of each item on the rating scale to examine its expert validity. Fifth, the OSCE station was pretested by nine examinees. The duration of the station, the arrangement of the space, and the rating scale were modified, and the hybrid assessment method was added according to the results of the pilot study. The final version of the OSCE station was then determined.

Examination of the Quality of the OSCE Station

In the pediatric occupational therapy OSCE, each examinee rotated through three stations. The stations were set up in 12 testing rooms of the clinical skill center; thus, the final version of the OSCE station for using a standardized assessment was used in 4 testing rooms. The content of the three stations for each examinee was identical, but the order could differ. Sixty examinees, 44 child SPs (5 SPs used in each of 8 testing rooms, plus 2 backup child SPs in the morning and 2 backup child SPs in the afternoon), 44 chaperones (simulated parents of child SPs), 4 adult SPs (1 used in each of 4 testing rooms), and 15 examiners (pediatric occupational therapists with an average of 13.5 yr of experience) participated in the OSCE. Each examiner rated 15 examinees. Each adult SP played the role of patient for 15 examinees, and each child SP played the role of patient for 3 examinees.

Measures

The relevant information regarding the OSCE station (see Supplemental Appendix) consisted of five parts: (1) information on the OSCE station, (2) information for the site organizers, (3) instructions for the examinees, (4) script for the child SPs and their chaperones, and (5) the rating scale for the examiners.

The examinee was allotted 2 min to read the instructions and 13 min to demonstrate the use of the standardized assessment, that is, one item in the Visual–Motor Integration subtest of the PDMS–2. The activity tested was building blocks. The OSCE station was intended to evaluate examinees’ clinical competency in using this assessment. Examinees’ interpersonal communication skills with children and their assessment skills (including calculation of the child’s chronological age, administration of the standardized assessment, and scoring) were evaluated. The child SPs were preschoolers older than 5 yr. The examinees used a hybrid assessment method that included interaction with the child SPs and video scoring of a video of a child with a developmental disability building blocks to evaluate examinees’ scoring abilities because the child SPs were typically developing children, and they always built blocks correctly.

Two types of scoring rubrics were used in this station, the sum of the rating scale score and the global performance score. The rating scale contained 11 items, and the total score could range from 0 to 22. Higher scores indicated better interpersonal communication skills with children and better assessment skills using the PDMS–2. To test interexaminer reliability, two examiners simultaneously observed and completed the rating scale without discussing their ratings with each other. They determined the examinee’s 5-point global performance score (excellent, good, pass, borderline, fail) on the basis of the examinee’s whole performance without consideration of the rating scale score. Examiners determined the criteria for the global performance score in a consensus meeting.

Data Analysis

The data were analyzed using IBM SPSS Statistics Version 21.0. We examined the mean, median, mode, skewness, and kurtosis of the sum of the rating scale score and then explored the expert validity, standard setting, failure rates, item analysis, relationship between the global performance score and the sum of the rating scale score, intergrade discrimination, between-groups variation, and interexaminer reliability.

Expert validity was examined with the item-level content validity index (i-CVI) and the scale-level content validity index, average calculation method (s-CVI/Ave; Lee, 2013; Polit & Beck, 2006). The standard setting was examined by using the Angoff method (Angoff, 1971; Khan et al., 2013; Lee, 2012, 2013) and the contrasting-groups M–SD method (Lee, 2012, 2013). The failure rates, used to review the effectiveness of teaching (Pell et al., 2010), were obtained by three different indicators on the basis of the global performance and the passing scores determined, respectively, by the Angoff method and the contrasting-groups M–SD method. Item analysis included internal consistency, item difficulty index, item discrimination index, critical ratio, and statistical significance (George & Mallery, 2003; Lee, 2012; Nunnally, 1978; Pell et al., 2010; Tavakol & Dennick, 2011).

The relationship between the global performance score and the sum of the rating scale score was determined by using the coefficient of determination (R ²) and the curve estimation plot (Pell et al., 2010). The intergrade discrimination was computed as the average increase in the sum of the rating scale score corresponding to an increase of one grade on the global performance score (Pell et al., 2010). Between-groups variation (examiner effect) was examined with one-way analysis of variance (ANOVA; Pell et al., 2010). Interexaminer reliability was examined using intraclass coefficients (ICCs) with a two-way random effects model (Shrout & Fleiss, 1979).

Results

The mean of the sum of the rating scale score was 11.30 (SD = 4.58). Both the median and the mode were 11.00. The distribution of the sums of the rating scale score was right skewed (0.28), with negative kurtosis (−0.66). The mean of fitness and the i-CVI (range = 0.8–1.0) for each item is shown in Table 1. For standard setting, the passing score determined by using the Angoff method was 14. The passing score set by the contrasting-groups M–SD method was 13, or M₁ − [S₁(M₁ − M₂)/(S₁ + S₂)] = 16.78 − [2.602(16.78 − 8.95)/(2.602 + 2.930)] = 13.10. Moreover, when the global performance score was used, the failure rate was 70%. When the Angoff and contrasting-groups M–SD methods of determining the sum of the rating scale score were used, the failure rates were 73.3% and 61.7%, respectively.

Table 1.

Psychometrics of the Rating Scale

Rating Scale Items	Expert Validity		Internal Consistency			Item Analysis				Interexaminer Reliability
Rating Scale Items	Mean of Fitness	i-CVI	M (SD)	Corrected Item–Total Correlation	α if Item Deleted	Item Difficulty Index	Item Discrimination Index	Critical Ratio	Statistical Significance (t[df])	ICC	Wilcoxon Signed-Rank Test (Z)
1. Calculate the child’s age correctly.	4.0	1.0	1.67 (0.75)	−0.02	.81	0.84	0.28	1.13	1.11 (22.73)	1.00*	0.00
2. Choose the correct starting items.	4.0	1.0	0.97 (0.97)	0.64	.74	0.49	1.88	15.04	15.00* (15.00)	0.97*	−1.00
3. Choose the materials and toys correctly.	3.6	1.0	0.85 (0.90)	0.67	.73	0.43	1.74	14.37	14.45* (25.72)	1.00*	0.00
4. Introduce herself/himself and check the child’s and her/his chaperone’s identities.	4.0	1.0	1.15 (0.55)	0.27	.78	0.58	0.58	3.09	3.06* (26.31)	1.00*	0.00
5. Guide the child and their chaperone to the assessment area. At the end, tell them the assessment is finished.	4.0	1.0	1.08 (0.59)	0.46	.77	0.54	0.90	4.89	4.89* (29.00)	1.00*	0.00
6. Communicate with the child and her/his chaperone using appropriate attitude (including appropriate volume, tone, speed, and eye contact).	4.0	1.0	1.50 (0.62)	0.47	.76	0.75	1.01	5.26	5.23* (17.28)	0.58*	−2.53*
7. Appropriately encourage the child before and after the assessment.	4.0	1.0	1.32 (0.77)	0.48	.76	0.66	1.28	6.94	6.92* (21.22)	0.61*	−1.13
8. Instruct the child (“build the steps like I did”) using age-appropriate verbal commands.	4.0	1.0	0.62 (0.80)	0.56	.75	0.31	1.25	6.45	6.46* (15.00)	0.61*	−1.41
9. Demonstrate the activity correctly.	3.8	1.0	0.23 (0.56)	0.50	.76	0.12	0.69	3.16	3.15* (15.00)	0.95*	−1.00
10. Score the child’s performance correctly.	4.0	1.0	1.08 (0.67)	0.49	.76	0.54	0.96	4.81	4.77* (29.00)	1.00*	0.00
11. Guide the child to put the toys in order after assessment.	3.6	0.8	0.83 (0.83)	0.33	.78	0.42	1.10	4.49	4.47* (29.00)	1.00*	0.00

Note. Scale-level content validity index, average calculation method = 0.98. Cronbach’s α = .78. ICC = intraclass coefficient; i-CVI = item-level content validity index.

p < .05.

The results for internal consistency (low and negative corrected item-total correlation for Item 4 and Item 1, respectively) and item analysis (item difficulty index [lowest for Item 9 and highest for Item 1], item discrimination index [lowest for Item 1 and highest for Item 2], critical ratios [lowest for Item 1 and highest for Item 2], and statistical significance [only Item 1 was not significant]) are also presented in Table 1.

The Pearson correlation coefficient for the global performance score and the sum of the rating scale score was 0.88 (p < .001). The R ² was .78, implying that 78.0% of the variation in the examinees’ global performance scores was accounted for by variation in the sums of their rating scale scores. Figure 1 presents a scatterplot of the linear correlation between the global performance score and the sum of the rating scale score. Moreover, the intergrade discrimination of the rating scale was 3.13, which was 14% of the maximum available rating scale score. Also, the one-way ANOVA indicated no significant examiner effect among the four examiners, F(3, 56) = 0.704, p = .554.

Figure 1.

Scatterplot of the linear correlation between global performance scores and the sum of rating scale score.

The results for interexaminer reliability are presented in Table 1 (ICCs ranged from 0.58 to 1.00). The sum of the rating scale score was 0.97, and the global performance score was 0.79. Systematic bias between the two examiners was found only for Item 6 (Z = −2.53, p < .05) and the global performance score (Z = −2.11, p = .04).

Discussion

To the best of our knowledge, this study is the first to develop an OSCE station using child SPs in pediatric occupational therapy and to examine the quality of the station.

The results show that passing scores were similar whether the Angoff method (passing score = 14) or the contrasting-groups method (passing score = 13) was used. The failure rates were high whether the global performance score (70%) or the Angoff method (73.3%) or contrasting-groups M–SD method (61.7%) of the sum of the rating scale score was used. This may be because this was the first time the OSCE was conducted in pediatric occupational therapy. Although most of the examinees (93.4%) felt that the content of the OSCE had been taught in their curriculum, they were also anxious because they were unfamiliar with the OSCE (Fu et al., 2017). This finding supports the importance of providing students with more opportunities to practice clinical skills and attitudes.

The psychometrics of the rating scale were solid, as determined by classical test theory. The i-CVI (range = 0.8–1.0) and the s-CVI/Ave (0.98) of the rating scale indicated acceptable expert validity.

As shown in Table 1, the rating scale’s internal consistency was acceptable. The easiest, most difficult, most discriminating, and least discriminating items were, respectively, Item 1, Item 9, Item 2, and Item 1. Moreover, all the indices for Item 1 suggested deletion of this item. The reason may be that most of the examinees (83.3%) received full scores on Item 1, which measured their arithmetic abilities rather than their personal communication or assessment skills. However, the expert committee thought that Item 1 should be preserved because incorrect calculation of the child’s age was a common student mistake in clinical training. The corrected item–total correlation for Item 4 was small (0.27). The ability tested by this item may be relevant to basic courtesy, which was not emphasized in the curriculum. These two items also support the importance of teaching appropriate self-introduction and checking the client’s and caregiver’s identities in occupational therapy training.

The correlation between the global performance score and the sum of the rating scale score (R ² = .88) indicated a reasonable relationship between them. A higher sum of the rating scale score generally corresponds with a higher global performance score. However, as shown in Figure 1, a wide range of marks for failing or borderline grades (1 or 2 on the x-axis) was found. This indicates that although some examinees received high marks on the rating scale, their global performance scores raised concerns among the examiners—for example, if an examinee received a score of 20 on the rating scale but a score of 0 for the global performance. This may imply that some items that examined the same competency were weighted implicitly in the sum of the rating scale score. However, this weighted competency was not as important in the global performance judged by the examiners.

The intergrade discrimination was a little high (14% of the maximum available rating scale score). It is possible that a small number of examinees with very poor scores steepened the regression line. Also, between-groups variation indicated that no significant examiner effect (i.e., the hawk–dove effect—the effect of examiners’ leniency or stringency) was found.

The ICCs showed that the interexaminer reliability of each item was fair to excellent. The interexaminer reliability of the sum of the rating scale score and the global performance score was excellent. However, the Wilcoxon signed-rank test demonstrated that systematic bias between the two examiners existed for Item 6 and the global performance score. We suggest using the OSCE video, which records the examinee’s performance in the OSCE station, in the examiner consensus meeting to increase interexaminer reliability of the rating scale (Gormley et al., 2012; Reid et al., 2016).

On the basis of the sound reliability and validity indicated by our data, we are confident that using this station in an OSCE would indicate areas of improvement for students in clinical assessment of children with the PDMS–2.

Limitations and Suggestions

This study has three limitations. First, we used only classical test theory to examine the psychometrics of the rating scale because of the limited number of examinees. However, the psychometrics examined with classical test theory are sample dependent. We recommend that future research use generalizability theory and item response theory (Glas, 2012). Second, the examinees in this study were undergraduate students with no clinical training. In future studies, a wider perspective could be gained by using undergraduate students who have received clinical training and postgraduate occupational therapists as examinees to examine the rating scale’s discriminant validity. Third, the examinees in this study were recruited from only one university. Our results should thus be generalized with caution. Administration of the OSCE to undergraduate students who both receive and do not receive clinical training, to postgraduate occupational therapists, and to examinees from different schools and hospitals is needed in future research. Moreover, we recommend using a similar approach with other parts of the PDMS–2, or with other assessment tools, when examining students’ core competency in using a standardized assessment.

Implications for Occupational Therapy Education

The OSCE helps to monitor students’ learning of knowledge, skills, and attitude; examine whether teachers’ curriculum mapping matches the core competencies; evaluate showing how to do rather than knowing how to do; and decide which student may need remedial teaching. The OSCE station for using a standardized assessment is reliable and valid for evaluating interpersonal communication and assessment skills. This study encourages the application of OSCE in occupational therapy education for a more holistic training and evaluation of occupational therapy students.

Conclusion

The OSCE station for using a standardized assessment has acceptable expert validity, acceptable internal consistency, good correlation between the global performance score and the sum of the rating scale score, no significant examiner effect, and acceptable interexaminer reliability. Students’ interpersonal communication and assessment skills were appropriately evaluated. These results support the effectiveness and rigorousness of applying an OSCE for education in pediatric occupational therapy.

Supplemental Material

Supplemental Information

Supplementary material for Development of an Objective Structured Clinical Examination Station for Pediatric Occupational Therapy and an Evaluation of Its Quality

Supplementary material, sj-pdf-1-aot-10.5014_ajot.2022.043521.pdf for Development of an Objective Structured Clinical Examination Station for Pediatric Occupational Therapy and an Evaluation of Its Quality by Chung-Pei Fu, Hsin-Yu Chi, Ming-Wei Li, Chien-Hsiou Liu, Jiann-Horng Yeh, Chih-Chia Wang and Chia-Ting Su in The American Journal of Occupational Therapy

Footnotes

Acknowledgments

We appreciate the support of the experienced pediatric occupational therapists on the expert committee, the child standardized patients, their parents, the chaperones, the playroom managers, the examiners, the examinees, and the Objective Structured Clinical Examination assistants who participated in this study. This study was supported by research grants from the Ministry of Science and Technology, Taiwan, R.O.C. (105-2511-S-030-004 and 106-2511-S-030-005).

References

Angoff

W. H.

(1971). Scales, norms, and equivalent scores (2nd ed.). American Council on Education.

Feng

C. T.

Wang

K. M.

Y. C.

Liu

C. W.

C. P.

Yeh

J. H.

, . . . Liu

C. H.

(2016). Construction of a valid and reliable checklist for Objective Structured Clinical Examination—An example of Brunnstrom motor recovery stage III/IV assessment for cerebral vascular accident. Journal of Occupational Therapy Association R.O.C. , 34 , 230–249. https://doi.org/10.6594/JTOTA.2016.34(2).05

Folio

M. R.

, & Fewell

R. R.

(2000). Peabody Developmental Motor Scales (2nd ed.). Pro-Ed.

C. P.

Yeh

J. H.

C. T.

Liu

C. H.

Chang

W. Y.

Chen

Y. L.

, . . . Wang

C. C.

(2017). Using children as standardized patients in OSCE in pediatric occupational therapy. Medical Teacher , 39 , 851–858. https://doi.org/10.1080/0142159X.2017.1320540

George

, & Mallery

(2003). SPSS for Windows step by step: A simple guide and reference (4th ed.). Allyn & Bacon.

Glas

C. A. W.

(2012). Generalizability theory and item response theory. In Eggen

T. J. H. M.

& Veldkamp

B. P.

(Eds.), Psychometrics in practice at RCEC. Research Centre for Examination and Certification.

Gormley

G. J.

Johnston

Thomson

, & McGlade

(2012). Awarding global grades in OSCEs: Evaluation of a novel eLearning resource for OSCE examiners. Medical Teacher , 34 , 587–589. https://doi.org/10.3109/0142159X.2012.682745

Harden

R. M.

(1988). What is an OSCE? Medical Teacher , 10 , 19–22. https://doi.org/10.3109/01421598809019321

Harden

R. M.

, & Gleeson

F. A.

(1979). Assessment of clinical competence using an Objective Structured Clinical Examination (OSCE). Medical Education , 13 , 41–54. https://doi.org/10.1111/j.1365-2923.1979.tb00918.x

10.

Hung

C. H.

, & Lin

C. Y.

(2013). Application of Structured Clinical Examination to diagnose clinical competencies in occupational therapy. Journal of Occupational Therapy Association R.O.C. , 31 , 145–173. https://doi.org/10.4236/ce.2013.410A008

11.

Jaywant

S. S.

, & Pai

A. V.

(2009). Evaluation and comparison of Objective Structured Clinical Examination and conventional clinical examination. Indian Journal of Occupational Therapy , 41 , 15–19.

12.

Kanada

Sakurai

, & Sugiura

(2015). Difficulty levels of OSCE items related to examination and measurement skills. Journal of Physical Therapy Science , 27 , 715–718. https://doi.org/10.1589/jpts.27.715

13.

Kanada

Sakurai

Sugiura

Motoya

Tomita

Sawa

, . . . Okanisi

(2012). Standardizing the assessment of the clinical abilities of physical therapists and occupational therapists using OSCE. Journal of Physical Therapy Science , 24 , 985–989. https://doi.org/10.1589/jpts.24.985

14.

Khan

K. Z.

Gaunt

Ramachandran

, & Pushkar

(2013). The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part II: Organisation and administration. Medical Teacher , 35 , e1447–e1463. https://doi.org/10.3109/0142159X.2013.818635

15.

Lee

S. T.

(2012). The Objective Structured Clinical Examination: Establishment, implementation, quality control (2nd ed.). Elsevier Taiwan.

16.

Lee

S. T.

(2013). Guideline of clinical competence assessment (2nd ed.). Elsevier Taiwan.

17.

Nunnally

J. C.

(1978). Psychometric theory (Vol. 2). McGraw-Hill.

18.

Pan

A. W.

, & Liu

L. T.

(2017). Construction and development of the Objective Structured Clinical Examination (OSCE) in occupational therapy. Journal of Medical Education , 21 , 9–25. https://doi.org/10.6145/jme201702

19.

Pell

Fuller

Homer

, & Roberts

; International Association for Medical Education. (2010). How to measure the quality of the OSCE: A review of metrics—AMEE Guide No. 49. Medical Teacher , 32 , 802–811. https://doi.org/10.3109/0142159X.2010.507716

20.

Polit

D. F.

, & Beck

C. T.

(2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing and Health , 29 , 489–497. https://doi.org/10.1002/nur.20147

21.

Reid

Smallwood

Collins

Sutherland

, & Dodds

(2016). Taking OSCE examiner training on the road: Reaching the masses. Medical Education Online , 21 , 32389. https://doi.org/10.3402/meo.v21.32389

22.

Sakurai

Kanada

Sugiura

Motoya

Wada

Yamada

, . . . Okanishi

(2014a). OSCE-based clinical skill education for physical and occupational therapists. Journal of Physical Therapy Science , 26 , 1387–1397. https://doi.org/10.1589/jpts.26.1387

23.

Sakurai

Kanada

Sugiura

Motoya

Wada

Yamada

, . . . Okanishi

(2014b). Reliability of the OSCE for physical and occupational therapists. Journal of Physical Therapy Science , 26 , 1147–1152. https://doi.org/10.1589/jpts.26.1147

24.

Sakurai

Kanada

Sugiura

Motoya

Yamada

Tomita

, . . . Okanishi

(2013a). Standardization of clinical competency evaluation in the education of physical therapists and occupational therapists—Establishment of an OSCE compliant education system. Journal of Physical Therapy Science , 25 , 101–107. https://doi.org/10.1589/jpts.25.101

25.

Sakurai

Kanada

Sugiura

Motoya

Yamada

Tomita

, . . . Okanishi

(2013b). Standardization of clinical skill evaluation in physical/occupational therapist education—Effects of introduction of an education system using OSCE. Journal of Physical Therapy Science , 25 , 1071–1077. https://doi.org/10.1589/jpts.25.1071

26.

Shrout

P. E.

, & Fleiss

J. L.

(1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin , 86 , 420–428. https://doi.org/10.1037/0033-2909.86.2.420

27.

Tavakol

, & Dennick

(2011). Post-examination analysis of objective tests. Medical Teacher , 33 , 447–458. https://doi.org/10.3109/0142159X.2011.564682

28.

Tavakol

, & Dennick

(2012). Post-examination interpretation of objective test data: Monitoring and improving the quality of high-stakes examinations: AMEE Guide No. 66. Medical Teacher , 34 , e161–e175. https://doi.org/10.3109/0142159X.2012.651178

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.37 MB