What do sports coaches actually know? Development and validation of physical and technical common content knowledge test packs

Abstract

While it is widely accepted that coach knowledge plays a foundational role in coaching effectiveness, there are few validated tools to determine pre- and- in-service coaches’ content knowledge (CK) levels. Thus, the aim of the present study was to develop and validate two CK test packs for developing physical and technical competence in athletes. Using convenience sampling, 2457 pre-service coaches participated in this study voluntarily. Several steps were followed for development of nine CK tests including facilitating panels, and processes for establishing content and face validities. Winstep was used to conduct Rasch analyses. Content validity was established, and the infit and outfit statistics confirmed that, with minor exceptions, all of the tests met the standards as the difficulties of questions matched respondents’ knowledge levels. The item and person separation indices showed excellent and acceptable confidence levels in item difficulties and respondents, respectively. The Wright map further demonstrated that all tests had a good range of difficulty levels to discriminate different knowledge levels. Data generated through the use of these CK test packs can be used to understand the actual knowledge levels of coaches, to monitor their improvements and to inform the development of educational programs.

Keywords

Assessment coaching effectiveness physical competence technical competence

Introduction

Côté and Gilbert¹ proposed that coaching effectiveness is dependent on “the consistent application of integrated professional, interpersonal, and intrapersonal knowledge to improve athletes’ competence, confidence, connection, and character in specific coaching contexts.” (p.316). This integrative definition focuses on coach knowledge, athlete outcome and coaching contexts. While interrelated, it is the element of coach knowledge that plays a foundational role in this trilogy. It underpins coaches’ work across contexts and trying to impact the athlete outcomes in and through sport positively.

More specifically, Côté and Gilbert¹ highlight the critical role of professional, interpersonal, and intrapersonal knowledge in coach effectiveness. Professional knowledge refers to knowledge of sport-specific demands and techniques, pedagogical knowledge to teach sports skills, and knowledge of sports sciences such as kinesiology, physiology and biomechanics. Interpersonal knowledge relates to interaction and connection with athletes, parents, coaches and other stakeholders in the sports setting. Intrapersonal knowledge is about openness to continued learning, self-awareness and reflection. While each of these categories of knowledge is accepted as being crucial for effective coaching, questions remain about how well these knowledge bases have been developed through contemporary coach education.

Although there are different perspectives and possibilities related to coaches’ education, formal coach education programs, and tertiary degrees in particular, are seen as an essential step toward the professionalization of coaching.^2–6 Despite their importance, tertiary programs around the world vary considerably in content and form.^7–9 This issue was identified by the International Council for Coaching Excellence (ICCE), which is a prominent, coach-focused and multi-sport global organization specifically working on coach learning and education. The ICCE has subsequently proposed standards for international higher education sport coaching bachelor degrees. ICCE standards are the only available standards for higher education sports coaching bachelor's degrees globally. They are flexible and non-compulsory guidelines, and acknowledge differences in specific national and local contexts and domains of practice. Three higher education institutions (HEIs) have been endorsed in the world so far. While this is a low number compared to the number of HEIs worldwide, it is still possible to make use of these proposed standards when appraising HEI offerings in sports coaching. Moreover, there is also potential value in using these standards as a basis for the examination of specific contexts as they are bound and shaped by their particular cultures, policies and values, which is appreciated and respected by ICCE.

In the Turkish coach education context, there are three ways in which a person can obtain a coaching certification: (a) through courses that are offered by the Turkish Ministry of Youth and Sport in cooperation with sports federations, (b) through “Coaching Education” degrees from universities in cooperation with sports federations, or (c) through exemptions due to international success as an athlete or a coach. As one of the best regarded and preferred options in Türkiye, the Departments of Coaching Education in Faculties of Sports Sciences provide a range of coach education modules as part of a bachelor's degree program. Over four years, pre-service coaches gain professional knowledge through compulsory and elective courses (e.g., motor learning, kinesiology, long-term athlete development) aiming to build a comprehensive skill set for their future careers. In the last year of their education process, they do an internship in a sports club to put their knowledge and skills into practice in a real-world setting. The internship took place at least one day a week throughout two semesters until 2023.¹⁰ With a new regulation and suggestion by The Council of Higher Education,¹¹ the internship takes place three days a week throughout the last two semesters since 2023. In particular, The Council of Higher Education¹¹ in Türkiye proposed a common curriculum for coaching education departments. Such calls for standardization reflect a desire to have a common knowledge base from which all Turkish coaches can subsequently operate.

The standard of knowledge has been a concern for researchers globally as well. For example, there are several research studies examining the knowledge levels of coaches on several subjects such as sports nutrition,^12,13 physical training methods^14,15 and sports science.¹⁶ However, many of the data collection tools have not been validated by a robust validation process such as Griffin et al.¹⁷ and many research projects investigated perceived knowledge with Likert-type questionnaires instead of directly questioning the knowledge levels of coaches with a test such as Blumenstein et al.,¹⁸ De Ste Croix et al.,¹⁹ Mawson et al.²⁰ and Quinaud et al..²¹ One of the main contributors to this situation is the lack of validated content knowledge (CK) tests available in the literature. The development and validation of CK tests can provide tools for educators to evaluate the knowledge levels of course attenders and the effects of the delivered instruction.²² Later on, the data generated through such CK tests can be used to support evidence-based coach education practices. The tests can also be used to support the design of professional development programs for coaches as in other fields. For instance, in the education field, Desimone²³ suggested that quality data is crucial to guide the design of teacher education for both pre-service and in-service professional development programs. Accordingly, tests are quite important to establish coach knowledge, to accurately determine current levels, to monitor improvements, and to make the required adjustments to education processes.

Considering the abovementioned potential utility along with the relative absence, developing CK tests for both pre-service and in-service coaches in several knowledge bases seems crucial to facilitate their education processes. Thus, in this study we developed and validated two CK test packs for developing physical and technical competence. While determining the content of the packs, ICCE standards⁸ and the proposed coaching education curriculum by Council of Higher Education¹¹ in Türkiye were used. For developing physical competence, ICCE standards⁸ highlighted planning theories (e.g., periodization), theories of physiological demands (e.g., strength and conditioning, sports nutrition and sports injuries) and biological theories (e.g., anatomy and exercise physiology). In Turkish coaching education curriculum in universities, courses are taught in anatomy and kinesiology, exercise and sport physiology, methodology of training, physical fitness, sports nutrition, sport injuries and first aid. For developing technical competence, theories of technique (e.g., biomechanics and skill acquisition) and theories of movement demands (e.g., motor development and functional movement) are highlighted in the professional knowledge domain by ICCE standards.⁸ These theories are delivered to Turkish pre-service coaches in sports biomechanics, motor development and motor learning courses. Thus, two CK test packs were developed and validated for each one of those subjects in Turkish coach education context to measure knowledge levels of pre-service coaches.

Materials & method

Akdeniz University Clinical Research Ethics Committee approved the study and consent forms were collected from the participants.

Participants

Convenience sampling method was used, and 2457 pre-service coaches participated in this study voluntarily (1599 participants in the CK Test Pack for Physical Competence Development and 858 participants in the CK Test Pack for Technical Competence Development). Their ages ranged from 18 to 38 years, and they were from coaching education bachelor's degree programs at 16 state universities across all seven geographical regions of Türkiye. The study's primary inclusion criterion was that each participant had previously completed the compulsory course during their undergraduate education relevant to the test they undertook. Considering that participation is voluntary, taking more than one test might cause fatigue and affect the data quality, each participant took one test. They were allocated randomly by the research team considering the number of students in universities and the primary inclusion criterion. Demographics of participants are presented in Table 1.

Table 1.

Demographics of participants.

	Content	Number of Participants	Age	Gender
CK Test Pack for Developing Physical Competence	Anatomy and Kinesiology	301	M = 21.20 (SD = 2.24)	Male = 58.14% Female = 40.53% Not disclosed = 1.33%
	Exercise and Sport Physiology	251	M = 21.37 (SD = 2.05)	Male = 57.77% Female = 41.43% Not disclosed = 0.80%
	Methodology of Training	240	M = 21.92 (SD = 1.69)	Male = 60.00% Female = 37.50% Not disclosed = 2.50%
	Physical Fitness	255	M = 21.73 (SD = 1.62)	Male = 55.69% Female = 42.35% Not disclosed = 1.96%
	Sports Injuries and First Aid	273	M = 21.66 (SD = 3.20)	Male = 55.31% Female = 43.59% Not disclosed = 1.10%
	Sports Nutrition	279	M = 21.81 (SD = 1.91)	Male = 66.67% Female = 32.26% Not disclosed = 1.08%
CK Test Pack for Developing Technical Competence	Motor Development	339	M = 21.04 (SD = 2.03)	Male = 54.28% Female = 45.13% Not disclosed = 0.59%
	Motor Learning	281	M = 21.66 (SD = 2.01)	Male = 45.20% Female = 53.38% Not disclosed = 1.42%
	Sports Biomechanics	238	M = 21.94 (SD = 1.53)	Male = 52.52% Female = 46.22% Not disclosed = 1.26%

M = Mean, SD = Standard deviation.

Development of the tests

The following steps were used to develop and validate the tests for each subject separately. First, a panel that included two professors with a PhD in the relevant study area, a coach with a Level 3 coaching certificate, and a curriculum developer was created. Both professors had knowledge of the related subjects and were coach educators in the coaching education departments. The coach had 19 years of coaching experience in a sports club and 6 years of experience in the national team. The curriculum developer had experience in national physical education and coaching organizations for 21 years. In addition, there was one Turkish language expert in the panel to check the final version of the tests regarding language clarity and grammar.

The second step involved the facilitation of the panel. The lead author who is a PhD candidate in Sport Pedagogy and Coaching organized and participated in the panel meetings to facilitate the discussions as well. Morrow et al.²⁴ suggested using a table of specifications to ensure content validity, and having other content experts review items as an addition. Accordingly, tables of specifications were created by each expert group and these tables were checked by two lecturers who had teaching experience on related subjects in the coaching education departments. The group met and developed questions that represented the core knowledge of the content as agreed on in the table of specifications. The CK test packs consisted of multiple-choice questions with four alternative options (only one correct answer).

In the third step, and as also suggested by Morrow et al.,²⁴ 10 coaches were invited for a content validity check via email. Considering that the subjects in the tests are compulsory for each student from different sporting backgrounds in Turkish tertiary coaching education programs, the research team aimed to get different perspectives from coaches in different sports in order to avoid any bias to one specific sport. Six of the invited coaches who graduated from the Department of Coaching Education and held Level 3 coaching certificates in either badminton, orienteering, tennis, handball, basketball or volleyball replied to the email and accepted to take part in the content validity check. They were provided the draft tests via email with the table of specifications for them to consider, and given their familiarity with the content from their experiences in university education, all confirmed the appropriateness of the content.

In the fourth step, the draft tests were checked by the same two lecturers in the second step as suggested by Morrow et al.²⁴ to ensure content validity. These lecturers approved all questions, stated that the developed tests were appropriate, and confirmed that coaching education students completing the relevant courses could answer these questions successfully.

To establish face validity, the draft tests were pilot tested with 10 pre-service coaches (4 females, 6 males) who were last year students in coach education program and had already successfully completed the related courses. These pre-service coaches completed the tests between 17 and 29 minutes (M = 21.66 minutes) confirmed the content representativeness of the tests based on their course experience and tables of specifications, and gave feedback regarding clarity for two questions in the anatomy and kinesiology CK test, one question in each of methodology of training, sports biomechanics, motor learning, and sport injuries and first aid tests. All comments were reviewed by the panel members and minor adjustments were made for wording and language clarity which were checked again and agreed upon by the same pre-service coaches. For the content and face validity processes, there is no gold standard for the number of participants involved in checking the tests. Thus, the authors agreed on not sending the tests to more people when the replies had started to be similar to each other. Example questions from each test are provided in Online Appendix 1.

Finally, tables of specifications created by two professors in the panel and confirmed by two lecturers taking part in content validity process, were used to determine the distribution of difficulty of the questions in each test. The questions of CK test pack for developing physical competence were rated 28% easy, 43% moderate and 29% difficult, and the questions of CK test pack for developing technical competence were rated 25% easy, 45% moderate and 30% difficult.

Procedures and data analysis

The pre-service coaches completed the developed tests as a paper and pencil tests during a regularly scheduled class. Morrow et al.²⁴ suggest 11 minute for one or two multiple choice questions for high school age and older individuals. Considering Morrow et al.'s²⁴ suggestion, test completion time range of 17 minutes to 29 minutes in pilot testing (M = 21.66 minutes), and the typical class time of 40–45 minutes in Türkiye, the participants were given a time limit of 30 minutes to complete the test, and no one needed additional time. Data for the completed tests were entered into Excel spread sheets, and then exported to Winstep (version 3.73) for the Rasch analysis.²⁵

Rasch modeling²⁴ is an alternative to classical test development approaches. It addresses issues with existing measures, including limited discrimination and negative skewness.²⁶ Boone²⁷ noted the advantages of Rasch modelling, including its capacity to make crucial modifications and convert nonlinear data to linear scale. Additionally, it has several significant instrumentation strengths such as step ordering/disordering, differential item/test functioning, and item/person reliability. Linacre²⁸ stated that the model is underpinned by the assumption that an individual's likelihood of responding to a question depends on the item's difficulty and ability. Contrary to traditional methods of test assessments like item response theory, which requires model to “fit the data”, Rasch model demands the data to “fit the model”.²⁸ In this study, we analysed the validity and reliability of each test in both CK packs through four basic levels of the Rasch model.^25,28

Item fit

Item fit statistics, which show how well each item fits the model, are provided via Rasch analysis.²⁹ The mean square residuals between the observed and expected responses are called infit and outfit statistics, and they are used to assess “fit to the model” (p.41).³⁰ Infit statistics are sensitive to expectations regarding potential responses from participants. Thus, it would suggest that the data fit the model if higher-knowledge respondents could answer most of the questions, especially the more challenging ones. Unexpected trends in the data can have a significant impact on outfit statistics. For instance, there would be a poor fit to the model if respondents who had lower knowledge on the content could answer the somewhat difficult questions correctly. The Rasch model's difference between what is observed and what is expected provides the basis for item fit statistics, including the mean square residual (MNSQ) and the standardized mean square residual (ZSTD).³⁰ ZSTD is a normalized t-score of the residual, while MNSQ is a squared residual based on the difference between the observed and expected response patterns.³⁰ Since the ZSTD value is based on the MNSQ, we examined the MNSQ values first to evaluate the fit of the test to the model.²⁸ A good model fit is achieved by the instrument tool when the MNSQ values, in both the infit and outfit statistics, range from 0.5 (indicating slight variance in answers) to 1.5 (indicating substantial variation).²⁸ Only when MNSQ values were outside that range we considered ZSTD values (the acceptable range −2.0 to 2.0).²⁸

Person fit

Person-fit statistics aim to decide the item-score pattern in the model.^28,29 Person-fit statistics were utilized to determine the relevance of responders’ answers. As in item fit, the acceptable range for MNSQ values is between 0.5 and 1.5. Person-fit statistics are considered poor if they are lower than 0.5 (e.g., slight variation in responses), or higher than 1.5 (e.g., large variation in responses) in MNSQ.²⁸ Again, ZSTD values were considered within the same range as stated above only when MNSQ values were not within the acceptable range.

Person item/wright maps

The visual representation of a distribution of item difficulties and respondents’ performance is provided by a Wright map, also called a person item map.²⁸ The Wright map consists of a left and a right side, with the item difficulties ranked and displayed on the right side. The most difficult item is plotted at the top of the map, while the easiest ones are at the bottom. The participants’ responses are represented on the left side, with the highest scores at the top and the lowest at the bottom. In an ideal test, the items and respondents are distributed on the Wright map from the highest to the lowest.

Separation index and separation-reliability index

The separation indices consist of two types: item and person. The item separation index is utilized to confirm the range of difficulty from easy to difficult, and it is associated with the model's construct validity. Meanwhile, the person separation index categorizes respondents into those with a high level of knowledge and those lacking the necessary level of knowledge. Both separation index scores follow these criteria: (a) 3.00 or higher indicates an excellent level, (b) 2.00 signifies a good level, and (c) 1.50 represents an acceptable level.²⁹

The separation-reliability index serves as a reliability measure for predicting the likelihood of repeating placements of an item or person.²⁹ Both item and person values fall between 0 and 1. A value near 1 indicates high confidence, while a value near 0 indicates low confidence (0.9 or higher = 3 or 4 levels; 0.8 = 2 or 3 levels; 0.5 or lower = 1 or 2 levels [not acceptable]).²⁸ When the reliability value is less than .5, it suggests that the variation in measurements is mainly caused by errors in measurement, which is not considered acceptable.²⁹ According to Linacre,²⁸ person reliability can also be interpreted as being similar to the more traditional reliability indices in classical test theory (i.e., KR-20 and Cronbach's alpha). However, a wide range of different qualitative descriptors were used to interpret alpha values such as excellent (0.93–0.94), fairly high (0.76–0.95), good (0.71–0.91), relatively high (0.70–0.77), adequate (0.64–0.85), satisfactory (0.58–0.97), acceptable (0.45–0.98) and not satisfactory (0.4–0.55).³¹ Due to differing views in literature regarding the qualitative interpretation of alpha value, we accepted > .5 as a benchmark for acceptability as suggested by Linacre²⁸ and Bond and Fox²⁹ for both item and person reliability in Rasch analysis specifically.

Results

Table 2 shows each test's MNSQ and ZSTD values for both infit and outfit in the CK Test Pack for Developing Physical Competence. The MNSQ values were in the acceptable range of .50 and 1.50,²⁸ except for one item in the Methodology of Training and Sports Nutrition tests each (values = 1.52 and 1.51, respectively; see Online Appendix 2). Items with unacceptable values were then checked for ZSTD values. These two items were also noted not in the acceptable range of ZSTD values (values = 3.1 and 2.8, respectively; see Online Appendix 2).

Table 2.

Summary statistics of the Rasch analysis for CK test pack for developing physical competence.

Content		Number measured	Infit		Outfit		Reliability	Separation
Content		Number measured	MNSQ M (SD)	ZSTD M (SD)	MNSQ M (SD)	ZSTD M (SD)	Reliability	Separation
Anatomy and Kinesiology	Item	25	1.00 (.10)	−.3 (1.7)	1.02 (.16)	−.1 (1.6)	.96	5.00
Anatomy and Kinesiology	Person	301	1.00 (.17)	.0 (.9)	1.02 (.30)	.0 (1.0)	.76	1.76
Exercise and Sport Physiology	Item	22	1.00 (.07)	−.3 (1.3)	1.01 (.13)	.0 (1.1)	.97	5.88
Exercise and Sport Physiology	Person	251	1.00 (.24)	−.1 (1.2)	1.01 (.38)	.0 (1.2)	.72	1.60
Methodology of Training	Item	27	1.00 (.08)	−.1 (1.3)	1.01 (.14)	.0 (1.3)	.96	4.96
Methodology of Training	Person	240	1.00 (.14)	.0 (.8)	1.01 (.25)	.0 (.9)	.75	1.71
Physical Fitness	Item	22	1.00 (.07)	−.2 (1.2)	.99 (.11)	−.2 (1.1)	.94	4.14
Physical Fitness	Person	255	1.00 (.14)	.0 (.9)	.99 (.23)	0 (.9)	.74	1.68
Sports Injuries and First Aid	Item	28	.99 (.05)	−.1 (1.0)	1.05 (.13)	.4 (.9)	.98	7.33
Sports Injuries and First Aid	Person	273	1.00 (.19)	.0 (1.0)	1.05 (.51)	.0 (1.0)	.82	2.17
Sports Nutrition	Item	26	.99 (.07)	−.3 (1.2)	1.05 (.17)	.3 (1.4)	.96	5.07
Sports Nutrition	Person	279	1.00 (.15)	−.1 (1.9)	1.05 (.31)	.0 (1.0)	.79	1.96

M = Mean; MNSQ = Mean square residual; SD = Standard deviation; ZSTD = Standardized mean square residual.

Reliability values of items of the CK Test Pack for Developing Physical Competence ranged from .94 to .98. These values showed that items of all tests had high-reliability levels. Person reliability values ranged from .72 to .82, within the acceptable reliability scores.

Item separation index scores ranged from 4.14 to 7.33 indicating that developed items of all tests had very high separation index values. Test items could discriminate between the participants who knew the subject matter and those who did not. Regarding the person separation index, there were acceptable and good scores from 1.68 to 2.17. These values demonstrated that participants who filled out the test had acceptable and good separation index scores, and they could discriminate between valid and invalid items.

The Wright map was used to check the distribution of the items (See Online Appendix 3). The Wright maps for all tests in the pack for developing physical competence illustrated that the difficulty of the items and person responses were well distributed.

Table 3 shows MNSQ and ZSTD values of each test in CK Test Pack for Developing Technical Competence. All of the MNSQ values were in the acceptable range of .50 and 1.50 except for one item in the Motor Learning test (value = 1.61; see Online Appendix 4). The ZSTD value of that item was checked, and it was not in the acceptable range too (value = 3.2; see Online Appendix 4).

Table 3.

Summary statistics of the Rasch analysis for CK test pack for developing technical competence.

Content		Number measured	Infit		Outfit		Reliability	Separation
Content		Number measured	MNSQ M (SD)	ZSTD M (SD)	MNSQ M (SD)	ZSTD M (SD)	Reliability	Separation
Motor Development	Item	24	1.000 (.04)	−.1 (.7)	1.003 (.10)	.3 (.9)	.97	5.30
Motor Development	Person	339	1.000 (.15)	.0 (.8)	1.003 (.35)	.0 (.9)	.78	1.087
Motor Learning	Item	20	.98 (.06)	−.2 (.8)	1.009 (.18)	.6 (1.2)	.98	6.55
Motor Learning	Person	281	1.000 (.22)	−.1 (1.0)	1.009 (.60)	.0 (1.1)	.76	1.076
Sports Biomechanics	Item	25	1.000 (.08)	−.2 (1.3)	1.001 (.12)	.0 (1.2)	.95	4.20
Sports Biomechanics	Person	238	1.000 (.14)	.0 (.9)	1.001 (.24)	.0 (.9)	.74	1.068

M = Mean; MNSQ = Mean square residual; SD = Standard deviation; ZSTD = Standardized mean square residual.

Reliability values of items in the CK Test Pack for Developing Technical Competence were .97, .98 and .95 for Motor Development, Motor Learning and Sports Biomechanics, respectively. Items in the tests had very high-reliability levels. Person reliability values ranged from .74 to .78. Moreover, values of person reliability were also within the acceptable reliability scores.

Separation index scores for items were 5.30, 6.55 and 4.20 for the three tests, respectively. Test items demonstrated very high separation index scores, which show the alignment of the item difficulties and the knowledge levels of the participants. Items of CK Test Pack for Developing Technical Competence were found to discriminate participants who knew the subject matter and those who did not. Person separation index scores were 1.87, 1.76 and 1.68 for the three tests, respectively. Similar to the CK Test Pack for Developing Physical Competence, values were acceptable, and participants could discriminate between valid and invalid items.

As in the CK Test Pack for Developing Physical Competence, the distributions of the items were checked using the Wright maps (See Online Appendix 5). The Wright maps for all tests illustrated that the difficulty of the items and person responses were well distributed.

Discussion

As noted earlier, coaching effectiveness as suggested by Côté and Gilbert¹ is founded on three components: coach knowledge, athlete outcomes and coaching contexts. Notably, questions remain about whether coach knowledge is being developed through contemporary coach education. Thus, this study was designed to develop two CK Test Packs (i.e., for Developing Physical Competence and for Developing Technical Competence) to evaluate the knowledge levels of pre-service coaches. Content validity was established, and the findings showed that all of the CK tests in both packs met the required standards. The infit and outfit statistics confirmed that all of the tests met the standards as difficulties of items matched with respondents’ knowledge levels (except for one item in each of the Methodology of Training, Sports Nutrition and Motor Learning tests). One outlier item in each of these three tests was removed due to their infit and outfit values. The item and person separation indices showed excellent and acceptable levels of confidence in a range of item difficulties and respondents, respectively. The Wright map further demonstrated that all tests had a good range of difficulty levels to discriminate different knowledge levels. In conclusion, Rasch analysis indicated that all of the tests in both CK Test Packs are valid and reliable measures of pre-service coaches’ CK with robust psychometric properties.

The two CK Test Packs developed in this study contribute to the literature in a number of ways including both in practice and in research. First, these tests can be used to measure pre-service coaches’ learning gains in the relevant courses delivered in the Coaching Education Departments. Given that previous studies have shown that coaches do not have strong knowledge bases on different contents such as physical training methods^14,15 and sports nutrition,³² it is valuable for educators to be able to test such knowledge using validated instruments. The findings in the literature that coaches who had already been through several coach education programs did not learn as intended should serve as a clear reminder to educators that the development of CK cannot be assumed, even in comprehensive education programs. Data generated through the use of these CK Test Packs can be used to understand the actual knowledge levels of coaches and inform the improvement of educational programs. Moreover, these tests can help coach educators to define the knowledge about these contents for preparing pre-service coaches to coach on the field. Actions could be taken to ensure alignment of educational objectives, pedagogical instructions, and assessment of and for learning in education programs.³³ In addition, researchers can use these tests to determine which sub-components are weak among pre-service coaches and where to focus instruction on improving that knowledge while designing development programs.

This study has several limitations. First, the tests were developed and validated in the context of Türkiye with pre-service coaches in the Coaching Education Departments. Thus, using these measures needs to be viewed cautiously in other contexts. There is great potential to engage in further testing and validation of CK Test Packs in different countries and with varying groups of participants such as pre- and in-service coaches. Second, the number of questions in the final version of each test ranged from 19 to 28, which was decided by the panel based on reducing participant fatigue and maximizing completion rates. Having a larger set of questions might provide more comprehensive coverage of the CK. Considering the knowledge bases required for coaches to develop athletes’ competence holistically (i.e., physical, technical, tactical and mental skills, as mentioned in Côté et al.³⁴), we narrowed our focus on CK for developing physical and technical competence. Development of CK tests for other components is highly recommended for future studies to reach a holistic perspective. Finally, while we used convenience sampling in this study and the number of participants was sufficient for validation, we recommend a random sampling with more participants for future studies that replicate this study. This random sampling would allow generalizability and decrease the possibility of selection bias and sampling error such as overrepresentation of certain groups (e.g., participants with high or low CK levels).

Conclusion

Considering the current era of accountability and calls for evidence-based practices, coach educators and researchers need valid instruments to ascertain whether educational programs and interventions are making a measurable difference in the learning of participants. This study provides valid and reliable CK tests for developing physical and technical competence with robust psychometric properties to be used as a pack or separately in both Coaching Education Programs and professional development settings.

Supplemental Material

sj-pdf-1-spo-10.1177_17479541251358190 - Supplemental material for What do sports coaches actually know? Development and validation of physical and technical common content knowledge test packs

Supplemental material, sj-pdf-1-spo-10.1177_17479541251358190 for What do sports coaches actually know? Development and validation of physical and technical common content knowledge test packs by Mert Bilgiç, Erhan Devrilmez, Steven Rynne, M Levent İnce, Alpay Güvenç, Selçuk Akpınar, Abdurrahman Aktop, Emel Çetin Özdoğan, İlkay Orhan, Özgür Özdemir, Y Gül Özkaya and Neşe Toktaş in International Journal of Sports Science & Coaching

Supplemental Material

sj-pdf-2-spo-10.1177_17479541251358190 - Supplemental material for What do sports coaches actually know? Development and validation of physical and technical common content knowledge test packs

Supplemental material, sj-pdf-2-spo-10.1177_17479541251358190 for What do sports coaches actually know? Development and validation of physical and technical common content knowledge test packs by Mert Bilgiç, Erhan Devrilmez, Steven Rynne, M Levent İnce, Alpay Güvenç, Selçuk Akpınar, Abdurrahman Aktop, Emel Çetin Özdoğan, İlkay Orhan, Özgür Özdemir, Y Gül Özkaya and Neşe Toktaş in International Journal of Sports Science & Coaching

Supplemental Material

sj-pdf-3-spo-10.1177_17479541251358190 - Supplemental material for What do sports coaches actually know? Development and validation of physical and technical common content knowledge test packs

Supplemental material, sj-pdf-3-spo-10.1177_17479541251358190 for What do sports coaches actually know? Development and validation of physical and technical common content knowledge test packs by Mert Bilgiç, Erhan Devrilmez, Steven Rynne, M Levent İnce, Alpay Güvenç, Selçuk Akpınar, Abdurrahman Aktop, Emel Çetin Özdoğan, İlkay Orhan, Özgür Özdemir, Y Gül Özkaya and Neşe Toktaş in International Journal of Sports Science & Coaching

Supplemental Material

sj-pdf-4-spo-10.1177_17479541251358190 - Supplemental material for What do sports coaches actually know? Development and validation of physical and technical common content knowledge test packs

Supplemental material, sj-pdf-4-spo-10.1177_17479541251358190 for What do sports coaches actually know? Development and validation of physical and technical common content knowledge test packs by Mert Bilgiç, Erhan Devrilmez, Steven Rynne, M Levent İnce, Alpay Güvenç, Selçuk Akpınar, Abdurrahman Aktop, Emel Çetin Özdoğan, İlkay Orhan, Özgür Özdemir, Y Gül Özkaya and Neşe Toktaş in International Journal of Sports Science & Coaching

Supplemental Material

sj-pdf-5-spo-10.1177_17479541251358190 - Supplemental material for What do sports coaches actually know? Development and validation of physical and technical common content knowledge test packs

Supplemental material, sj-pdf-5-spo-10.1177_17479541251358190 for What do sports coaches actually know? Development and validation of physical and technical common content knowledge test packs by Mert Bilgiç, Erhan Devrilmez, Steven Rynne, M Levent İnce, Alpay Güvenç, Selçuk Akpınar, Abdurrahman Aktop, Emel Çetin Özdoğan, İlkay Orhan, Özgür Özdemir, Y Gül Özkaya and Neşe Toktaş in International Journal of Sports Science & Coaching

Footnotes

Acknowledgements

We thank all of the pre-service coaches and panel members/experts for their participation and support.

Consent to participate

Written informed consent forms were collected from the participants.

Consent for publication

Written informed consent forms were collected from the participants.

Data availability

All data will be made available at request to the corresponding author.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical considerations

Akdeniz University Clinical Research Ethics Committee approved the study (Approval date: 22.03.2023/ Approval code: KAEK-225)

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The lead author and the project were funded by 2211-A and 2214-A programs of The Scientific and Technological Research Council of Türkiye (TÜBİTAK).

ORCID iDs

Mert Bilgiç

Erhan Devrilmez

Steven Rynne

M Levent İnce

Alpay Güvenç

Selçuk Akpınar

Abdurrahman Aktop

Emel Çetin Özdoğan

İlkay Orhan

Özgür Özdemir

Y Gül Özkaya

Neşe Toktaş

Supplemental material

Supplemental material for this article is available online.

References

Côté

Gilbert

. An integrative definition of coaching effectiveness and expertise. Int J Sports Sci Coach 2009; 4: 307–323.

Carter

Bloom

. Coaching knowledge and success: going beyond athletic experiences. J Sport Behav 2009; 32: 419–437.

Erickson

Bruner

MacDonald

, et al. Gaining insight into actual and preferred sources of coaching knowledge. Int J Sports Sci Coach 2008; 3: 527–538.

Lara-Bercial

Mallett

. The practices and developmental pathways of professional and Olympic serial winning coaches. Int Sport Coach J 2016; 3: 221–239.

Rynne

Mallett

Tinning

. High performance sport coaching: institutes of sport as sites for learning. Int J Sports Sci Coach 2006; 1: 223–234.

Mallett

Lara-Bercial

. Learning from serial winning coaches: Caring determination. New York: Routledge, 2024, pp.41–42.

Callary

Culver

Werthner

, et al. An overview of seven national high performance coach education programs. Int Sport Coach J 2014; 1: 152–164.

International Council for Coaching Excellence. ICCE Standards for Higher Education Sport Coaching Bachelor Degrees. ICCE, 2016.

Mesquita

Borges

Rosado

, et al. Self-efficacy, perceived training needs and coaching competences: the case of Portuguese handball. Eur J Sport Sci 2012; 12: 168–178.

10.

Işık

Moustakas

. Coaching and coach education in Turkey: history, Status, and future. Int Sport Coach J 2022; 9: 390–396.

11.

Council of Higher Education. Spor Bilimleri Fakültesi Antrenörlük Eğitimi Bölümü Ulusal Çekirdek Eğitim Programı (ANE-UÇEP) [Faculty of Sport Sciences Department of Coaching Education National Common Education Program]. https://www.yok.gov.tr/kurumsal/idari-birimler/egitim-ogretim-dairesi/ulusal-cekirdek-egitimi-programlari (2023, accessed 8 August 2024).

12.

Cockburn

Fortune

Briggs

, et al. Nutritional knowledge of UK coaches. Nutrients 2014; 6: 1442–1453.

13.

Trakman

Forsyth

Devlin

, et al. A systematic review of athletes’ and coaches’ nutrition knowledge and reflections on the quality of current nutrition knowledge measures. Nutrients 2016; 8: 570.

14.

Hammad

Alnawayseh

Alshaar

, et al. Assessing the knowledge level of karate coaches regarding physical training modalities. Sport Mont 2024; 22: 67–73.

15.

Hammad

Djemai

, et al. The knowledge level of taekwondo coaches regarding physical training methods in Jordan. Sci Sports 2022; 37: 631–6e1.

16.

Brink

Kuyvenhoven

Toering

, et al.

What do football coaches want from sport science?

Kinesiology 2018; 50: 150–154.

17.

Griffin

Ranson

Moore

, et al. Concussion knowledge and experience among Welsh amateur rugby union coaches and referees. BMJ Open Sport & Exercise Medicine 2017; 3: e000174.

18.

Blumenstein

Orbach

Bar-Eli

, et al. High-level coaches’ perceptions of their professional knowledge, skills, and characteristics. Sport Sci Rev 2012; 21: 5–27.

19.

De Ste Croix

Ayala

Sanchez

, et al. Grass-root coaches knowledge, understanding, attitude and confidence to deliver injury prevention training in youth soccer: a comparison of coaches in three EU countries. J Sci Sport Exerc 2020; 2: 367–374.

20.

Mawson

Creech

Peterson

, et al. Lower limb injury prevention programs in youth soccer: a survey of coach knowledge, usage, and barriers. J Exp Orthop 2018; 5: 1–7.

21.

Quinaud

Backes

Nascimento Junior

JRAD

, et al. Development and validation of the coach knowledge questionnaire: measuring coaches’ professional, interpersonal and intrapersonal knowledge. Int J Sport Exerc Psychol 2022; 20: 302–318.

22.

Santiago

Morrow

. A study of preservice physical education teachers’ content knowledge of health-related fitness. J Teach Phys Educ 2020; 40: 118–125.

23.

Desimone

. Improving impact studies of teachers’ professional development: toward better conceptualizations and measures. Educ Res 2009; 38: 181–199.

24.

Morrow Jr

Mood

Disch

, et al. Measurement and evaluation in human performance. 5th ed. Champaign: Human Kinetics, 2016.

25.

Rasch

. Probabilistic Models for some Intelligence and Attainment Tests. Chicago: University of Chicago Press, 1980.

26.

Edelsbrunner

Dablander

. The psychometric modeling of scientific reasoning: a review and recommendations for future avenues. Educ Psychol Rev 2019; 31: 1–34.

27.

Boone

. Rasch analysis for instrument development: why, when, and how? CBE—Life Sci Educ 2016; 15: rm4.

28.

Linacre

. A user’s guide to Winsteps Rasch-model computer program. Version 3.73, 2008.

29.

Bond

Fox

. Applying the Rasch model: Fundamental measurement in the human sciences. 2nd ed. London: Erlbaum, 2007.

30.

Liu

. Using and developing measurement instruments in science education: A Rasch modeling approach. 2nd ed. New York: Information Age Publishing, 2020.

31.

Taber

. The use of cronbach’s alpha when developing and reporting research instruments in science education. Res Sci Educ 2018; 48: 1273–1296.

32.

Botsis

Holden

. Nutritional knowledge of college coaches. Sport Sci Rev 2015; 24: 193–200.

33.

Devrilmez

Dervent

Ward

, et al. A test of common content knowledge for gymnastics: a Rasch analysis. Eur Phy Educ Rev 2019; 25: 512–523.

34.

Côté

Erickson

Duffy

PAT

. Developing the expert performance coach. In: Farrow

Baker

MacMahon

(eds) Developing sport expertise. 2nd ed. London: Routledge, 2013, pp.96–112.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.44 MB

0.14 MB

0.43 MB

0.07 MB

0.39 MB