Exploring the Validity and Robustness of a Competency Self-Report Instrument for Vocational and Higher Competence-Based Education

Abstract

Research on the effectiveness of competence-based education (CB-education) across educational contexts and levels requires a new evaluation measurement. This study explores the face validity, construct validity, and robustness of a competency self-report instrument that is aligned with contemporary competence theory and with current educational practice based on CB-qualification frameworks. A pilot study showed face validity of the competency constructs and indicators according to students from various levels in tertiary education. The results of the principal components analyses and parallel analyses, using data from 351 secondary vocational education and academic students, show more construct validity and robustness for competency constructs that are concrete and easy to relate to specific situations (e.g., “applying expertise”) compared with the abstract competencies (e.g., “deciding and initiating”). This article sets out implications for designing and administrating uniform competency self-reports across educational levels and discusses suggestions for subsequent research.

Keywords

competency competence-based education self-report self-assessment vocational education higher education

Within competence-based education (CB-education), there is a gradual paradigm shift from thinking in task-specific qualifications to more general competencies (Sturing, Biemans, Mulder, & Bruijn, 2011). In the 1970s and 1980s, the CB-education movement led to formulating endless lists of detailed, narrowly formulated, task-specific performance criteria (Bowden & Masters, 1993; Grant et al., 1979) and ignored the importance of how to apply knowledge in various working situations (Argüelles & Gonczi, 2000). During the past two decades, several countries, including Germany, France, and Austria, have developed a more comprehensive approach toward CB-education in which learning situations address essential knowledge, skills, and attitudes in an integrated manner (Biemans et al., 2009). In a contemporary CB-curriculum, students are confronted with a variety of core problems that they may encounter in their professional lives, situated in meaningful and recognizable contexts, with the aim of developing competencies that are portable from one context to another (Wesselink, Biemans, Mulder, & van den Elsen, 2007). These competencies are included in the qualification frameworks of many countries (e.g., the United Kingdom, Australia, Germany). This raises a number of questions: What specific learning settings and contexts are effective for developing competencies? What effects does CB-education have on vocational and higher education students’ competency development? How can competencies best be assessed (e.g., Blömeke, Zlatkin-Troitschanskaia, Kuhn, & Fege, 2013; Schaap, Baartman, & de Bruijn, 2012)?

Researchers who try to answer such questions need a competency measurement instrument that allows them to explain variation in the development of different competencies across educational settings and levels. Such an instrument also needs to be aligned with contemporary CB-education theory and practice. The aim of this study is to construct a competency measurement and to test its face validity, construct validity, and robustness across educational settings and levels. This study argues that such a competency measurement consists of (a) a variety of competencies from a qualification framework, (b) incorporating, for each competency, several indicators that include relevant knowledge, skills, and attitudes. Arguments for constructing the competency measure this way are the following:

First, competencies are the foundation of many countries’ qualification frameworks (e.g., the U.K. National Vocational Qualifications Framework, Australian Standards Framework, European Qualifications Framework, and the Bologna Qualifications Framework). These qualification frameworks consist of outcome standards for reaching a common approach to qualifications and assessments across disciplines. The idea behind qualification frameworks is similarity; all qualifications share core competencies that are generic across professional sectors and educational levels (Young, 2009). In formulating and working toward a common set of outcomes, the aim is to improve mobility of labor and transferability between educational systems (Brockmann, Clarke, Méhaut, & Winch, 2008). Competencies in a qualification framework include not only functional and behavioral requirements (e.g., applying expertise) but also more complex cognitive abilities for functioning in the profession (e.g., problem solving) as well as social abilities to function as a person (e.g., showing tolerance and caring for others; Le Deist & Winterton, 2005). The competencies in a qualification framework can be a guideline for teachers in designing their CB-learning context, adapted to the students’ level (Young, 2009). It would be efficient to align competency effectiveness studies with a qualification framework and incorporate a set of competencies from a qualification framework in a competency measurement applicable to different educational programs and levels.

Second, competencies are coherent clusters of knowledge, skills, and attitudes that can be utilized in real performance contexts (Mulder, 2014). Traditional CB-education aimed to enable students to acquire qualifications that led to competencies that basically consisted of a summing-up of fragmented knowledge, skills, or attitudes related to a specific occupation (Boyatzis, 1982). In contrast, the aim of contemporary CB-education is the development of competencies that students need in their future professional career and in society as a whole (Biemans et al., 2009). Therefore, an integration of knowledge, skills, and attitudes in learning and assessment is necessary (Wesselink, de Jong, & Biemans, 2010).

Third, research argues that it is possible to measure different kinds of competencies via self-reports under certain conditions: (a) The instrument should include multiple indicators per competency to address a competency in its full complexity; (b) context should be given for the competencies and indicators; and (c) the indicators should concern concrete behavior. Braun, Woodley, Richardson, and Leidner (2012) review seven examples of competency self-reports frequently used in educational settings around the world. According to the authors, competency self-reports tend to include vague and abstract expressions, which increases the likelihood of personal interpretation and decreases the validity of the measurement. One example of abstract wording is the Cognitive Development Scale of the Cooperative Institutional Research Program (see http://www.heri.ucla.edu/abtcirp.php). Without further explanation of the concepts, this questionnaire instructs students to rate themselves on competencies such as “my critical thinking skills” or “my analytical and problem-solving skills.” Competencies are complex constructs; without context, they can be open to multiple interpretations (Hodkinson & Issitt, 1995). To avoid misinterpretation and to cover a given competency construct in its full complexity, self-reports should at least include multiple indicators that concern specific behavior (Braun et al., 2012).

This Study

The present study uses the competency framework used for vocational education programs as described in the Dutch Association for Voctional education expertise centre (COLO, 2006, see Appendix A), which is based on the Uniform Competency Framework of SHL (Bartram, 2005). Our framework is of an explorative nature and aims to investigate the possibility of constructing a competency self-report for vocational and higher education based on a generic competency framework. Because the concept of competencies is sensitive to personal interpretations, we (a) have assured face validity with pilot groups from vocational and higher education–level students, and (b) focus in the present study on examining the construct validity and robustness of the competency constructs. Robustness refers here to the possibility of using the instrument across educational levels. The educational field would benefit from a uniform competency self-report because it allows for comparing CB-learning context and thereby offers better insights into the effectiveness of specific CB-learning contexts. This allows for more targeted use of courses for training specific sets of competencies across various levels. The research questions guiding this study are as follows:

Research Question 1 (RQ1): What is the construct validity of a competency self-report instrument with distinguishing competencies and indicators?

Research Question 2 (RQ2): Is a competency measurement with such a self-report instrument robust across educational levels?

Method

Instrument Development

Instrument development consisted of formulating the indicators for all 25 competencies from the theoretical qualification framework and testing the face validity with student groups.

Initially, the first two authors carefully compared indicators documented by various authors and organizations developing indicators for the theoretical qualification framework (e.g., Groene Kennis Coöperatie, 2008; van den Herik & Winkler, 2008). The authors identified which indicators were mentioned most frequently per competency and formulated for each competency a comprehensive description of the competency and a set of indicators in the form of behavior-related wording. Next, the descriptions of competencies and their underlying indicators were presented to independent researchers in the field of competencies, for content validity, face validity, clarity, and readability. Based on the reviews of the independent researchers, unclear indicators were reformulated and irrelevant indicators were eliminated. This resulted in a self-report with a Likert-type scale ranging from 1 (not at all applicable to me) to 10 (completely applicable to me) per indicator. The instrument was labeled Competentie Ontwikkelings Meter (COM)—or, in English, “The Competency Development Meter.”

Second, the COM was pilot-tested in January and February 2011 with six student groups from secondary vocational education and higher education in the life sciences, the latter consisting of the higher vocational level and the academic level. Students filled out the questionnaire individually; directly following, they took part in a 1½-hr group debriefing group interview per educational level to investigate face validity and readability of the competency indicators (Czaja & Blair, 2005). During interviews, the students were asked whether they (a) understood the competency and the indicators, (b) thought the indicators fit the competency, (c) recognized the competency and indicators from their school and/or working situations, (d) could name specific situations in which they worked on the competency and indicators, and (e) could specify how they worked on the competency and indicators. Last, each indicator was specifically discussed regarding its readability.

Students’ reactions, interpretations, and suggestions were ordered per competency and put in an overview. Face validity was ensured by reformulating indicators that the students found unclear and omitting those that none of the students recognized in practice. Finally, the indicators were corrected for readability by two independent researchers. This resulted in the last version of the COM consisting of 25 competencies from the theoretical qualification framework, with 5 to 9 indicators per competency.

Procedure

In 2011 and 2012, new groups of students from secondary vocational education and academic education were assigned to fill out the COM. Within the context of a certain educational module, students assessed themselves on a selection of the competencies that, according to the teaching staff, were relevant. For the purpose of the present study, only those competencies filled out by both groups were used in the analyses. The competencies “deciding and initiating,” “cooperating,” “applying expertise,” and “planning and organizing,” and their related indicators, were included in the analyses. See Appendix B for all the indicators as translated from Dutch to English.

Subjects

A total of 351 life-sciences students completed the COM (n = 195 for the secondary vocational education group and n = 146 for the academic education group, see Table 1). The secondary vocational education students were studying animal husbandry, animal care and management, horse equipment, and commercial entrepreneurship in a learning environment that intertwines school and workplace learning. The academic students were working in a project-based setting with multidisciplinary groups: land use planning; international development studies; management, economics, and consumer studies; forest and nature conservation; and animal science.

Table 1.

Characteristics of the Subjects.

	Secondary vocational education sample	Academic education sample
n	195	146
Gender (%) male	116 (59.5)	75 (51.4)
Age M (SD)	19.03 (1.34)	21.0 (4.0)
Level (%)^a	3 (3.1)	BSc 111 (76)
	4 (96.9)	MSc 35 (24)

The Dutch secondary vocational educational system distinguishes Levels 1, 2, 3, and 4. For more information about the Dutch educational system, refer to Wesselink, Biemans, Mulder, and van den Elsen (2007).

Statistical Analysis

Construct validity of the COM was explored in both groups by a principal components analysis (PCA) with orthogonal rotation (varimax). Prior to final component extraction, indicators with communalities below 0.5 were omitted. The suitability of the data was assessed with the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy (>.5) and Barlett’s Test of Sphericity (Field, 2009). With PCA, there are several debatable decision rules for component extraction, such as a low reliability of data interpretation and a high risk of over-factoring (O’Connor, 2000). Therefore, we performed a parallel analysis (PA) for each data set (O’Connor, 2000) to determine the number of components, as PA is currently the most accurate method for deciding on numbers of component extraction in PCA (Schmitt, 2011). Missing cases were excluded list-wise. The robustness of the components was explored by comparing component patterns of indicators across educational levels. All analyses were conducted in SPSS version 19.

Results

Secondary Vocational Education Group

Four indicators had communalities under 0.5 and were therefore omitted from the analysis (see Appendix B). The KMO measure confirmed the sampling adequacy for the analysis with KMO = .81. Barlett’s Test of Sphericity, χ²(231) = 2,123.98, p > .001, indicated that correlations between the indicators were sufficiently large for PCA. The analysis resulted in six components with an eigenvalue more than 1, which in combination explained 70.42% of the variance. The PA suggested four components (Table 2). Therefore, extraction was restricted to four components, explaining 54.38% of the total variance. Table 3 shows the factor loadings after varimax rotation.

Table 2.

Parallel Analysis for the PCA of the Secondary Vocational Education Group Data.

Component	PCA eigenvalue	PA eigenvalue^a	Difference
1	5.563	1.764	3.800
2	3.776	1.617	2.159
3	2.260	1.522	0.738
4	1.590	1.440	0.150
5	1.288	1.364	−0.007
6	1.016	1.300	−0.284

Note. PCA = principal components analysis; PA = parallel analysis.

Random data eigenvalues for 100 replications over 22 indicators and 194 subjects.

Table 3.

Structure Matrix Obtained by PCA After the Varimax Rotation on Indicators for the Secondary Vocational Education Group (n = 194).

Competency	Competency indicator	Component 1 “Planning and organizing”	Component 2 “Applying expertise”	Component 3 “Task-specific shared responsibility”	Component 4 “Peer collaboration”
Planning and organizing	64537	0.851 0.844 0.817 0.809 0.764




Applying expertise	42135		0.840 0.800 0.791 0.765 0.695		0.445




Cooperating	2			0.821
	4			0.809
	3			0.788
	5			0.582
	9				0.880
	8				0.858
	6				0.565
Deciding and initiating	342	0.442


Eigenvalues		5.563	3.776	2.260	1.590
% of variance		25.29	17.16	10.27	7.22

Note. All loadings >0.40 are depicted; factor loadings >0.50 are in boldface. PCA = principal components analysis.

Indicators of the competency “planning and organizing” had high factor loadings on Component 1, and “applying expertise” had high factor loadings on Component 2. These two components were in line with the theoretical competency framework. The indicators for the theoretical competency “cooperating” were divided between two components, a task-oriented component and a social-oriented component. Indicators 2, 3, 4, and 5 were mainly about helping others, performing duties, and contributing to the common result, and were labeled as Component 3—“task-specific shared responsibility.” Component 4 included Indicators 6, 8, and 9 of the theoretical competency “cooperating” and represented only the social and interactive aspects of working together. Therefore, Component 4 was labeled as “peer collaboration.” In the secondary vocational group, no component was found reflecting the competency and items of “deciding and initiating.”

Academic Education Group

Initial analysis showed one indicator with a communality below 0.5 and was omitted from further analysis. Refactoring showed sampling adequacy for the analysis (KMO = .81) and sufficiently large correlations between the indicators, Barlett’s Test of Sphericity, χ²(300) = 1,861.09, p < .001. Seven components had eigenvalues greater than Kaiser’s criterion of 1 and, in combination, explained 72.57% of the variance. The PA suggested extraction of only four components (Table 4). Therefore, four components explaining 59.52% of the total variance were extracted. Table 5 shows the structure matrix after varimax rotation.

Table 4.

PA for the PCA of the Academic Group Data.

Component	PCA Eigenvalue	PA Eigenvalue^a	Difference
1	7.074	2.033	5.041
2	3.278	1.839	1.439
3	2.269	1.695	0.574
4	1.760	1.588	0.202
5	1.401	1.503	−0.102
6	1.338	1.432	−0.094
7	1.021	1.354	−0.333

Note. PA = parallel analysis; PCA = principal components analysis.

Random data eigenvalues for 100 replications over 24 variables and 135 subjects.

Table 5.

Structure Matrix Obtained by PCA After the Varimax Rotation on Indicators for the Academic Education Group (n = 135).

Competency	Competency Indicator	Component 1 “Applying expertise”	Component 2 “Planning and organizing”	Component 3 “Task-specific shared responsibility”	Component 4 could not be labeled
Applying expertise	2	0.900
	4	0.894
	3	0.867
	1	0.854
	5	0.844
Planning and organizing	64532		0.835 0.832 0.756 0.722 0.561




Cooperating	2			0.791
	4			0.788
	3			0.694
	5			0.628
Deciding and initiating	514				0.877
					0.742
					0.590
Cooperating	6
	9
	8
	7			0.415
Deciding and initiating	32
Deciding and initiating	32
Planning and organizing	71		0.4680.484
Planning and organizing	71		0.4680.484
Eigenvalues		7.074	3.278	2.269	1.760
% of variance		28.30	13.11	9.07	7.04

Note. All loadings >0.40 are depicted; factor loadings >0.50 are in boldface. PCA = principal components analysis.

For the academic group, indicators for the competencies “applying expertise” had high factor loadings on Component 1, whereas indicators for the competency “planning and organizing” had high loadings on Component 2. Components 1 and 2 were labeled as “applying expertise” and “planning and organizing,” consistent with the theoretical competency framework. Component 3 appeared to reflect the shared responsibility students have when performing a task together and was labeled as “task-specific shared responsibility.” Component 4 consisted of three items of the theoretical competency “deciding and initiating.” However, these items were not interpretable and we decided not to label this component.

Construct Validity

The explorative analyses on the COM suggest construct validity of the theoretical competency constructs “planning and organizing” and “applying expertise.” The analyses also suggest that the theoretical construct “cooperating” actually is made up of two components: “task-specific shared responsibility” and “peer cooperation,” whereby “task-specific shared responsibility” was found in both groups and “peer collaboration” was only found in the secondary vocational education group. As the competency “deciding and initiating” was not a meaningful construct in both analyses, this was not a valid construct.

Robustness

The results show robustness of some competency constructs from the original competency framework. The empirical patterns can be seen as signs of robustness across indicators on “applying expertise” and “planning and organizing.” These indicators cover the same competency constructs on both educational levels. Indicators reflecting the “shared responsibility” part of cooperating were extracted as a separate component in both groups, while the other indicators of the cooperating construct were only extracted as a separated component (“peer collaboration”) in the vocational education group. Thus, “task-specific shared responsibility” seems to be an additional robust construct. The theoretical competency construct “deciding and initiating” could not be considered robust, as this was not an interpretable separate component in both groups.

Conclusion and Discussion

This study explored the possibility of constructing a competency self-report aligned with the practice and theory of contemporary CB-education. The competency self-report instrument with multiple indicators per competency, COM, which has shown face validity according to vocational and higher educational students, was examined for its construct validity and robustness. The performance of the COM was mixed. Two constructs—“planning and organizing” and “applying expertise”—showed construct validity and robustness. The indicators loaded on the same extracted components in both groups. Construct validity of the theoretical competency “cooperating” varied between groups, but an additional robust construct, “task-specific shared responsibility,” was found. No robust construct reflecting the competency “deciding and initiating” could be found. These results show that, under certain circumstances, it is possible to construct a competency self-report instrument based on a qualification framework. The reasons for the mixed findings and the implications for assessing competencies using a self-report instrument can be found in the formulation and context specificity of the indicators and the misalignment between selected competencies and their actual implementation.

First, there is a possibility that the formulation of the indicators of the competencies “deciding and initiating” and “cooperating” was not specific enough for valid measurements. Schwarz (1999) advises self-assessment only for concrete and specific behaviors related to particular situations. Although we formulated the indicators of the COM as concrete as possible, it was also our goal to allow comparison and differentiation between educational situations, and therefore to develop items that are generically applicable. Indicators such as those associated with the theoretical competency construct “deciding and initiating,” could have been more abstract in wording and consequently more ambiguous. Indicators such as “I am able to justify my choices” may still have been too abstract for the students. In that respect, the present study underpins the statement of Braun et al. (2012) that concrete and straightforward wording is necessary when validly self-assessing competencies. Ackerman, Beier, and Bowen (2002) state that self-assessment of capacity is markedly improved when using concrete items instead of broadly defined concepts. Because our study showed validity and robustness of competency measurements that are generally easier to relate to a specific context (“applying expertise” and “planning and organizing”) than the more abstract ones are (“cooperating” and “deciding and initiating”), there is a possibility that, for improving valid measurements, abstract competency constructs need more context-specific wording than concrete competency constructs do. Three questions remain from this study: To what extent should indicators of competency constructs be concretized for valid and robust measurements? How context-specific should competency measurements be? And can abstract competency constructs be evaluated with a self-report in a valid way across educational levels?

A second explanation may be the misalignment between selected competencies and their actual implementation. Benett (1993) attributes difficulties with standardized self-report instruments to the complexity and variety of learning situations students encounter in work-related learning. Benett claims that it is possible to use competency standards for comparisons between groups but only if the competencies and their associated indicators are representative of the situation to which the self-report instrument refers. A recent study on CB-assessment shows that the intended outcomes are often described in terms of competencies, but in practice, the competencies are not sufficiently addressed (Baartman, Gulikers, & Dijkstra, 2013). There is a possibility that the students in our study did not consciously work on the competency “deciding and initiating actions”, although the teachers selected relevant competencies prior to the learning situation. As a result, students may have found it hard to imagine indicators such as “I take responsibility for the choices I make” because in reality they never had to deal with this consciously.

Implications

The present study demonstrated that there are possibilities for using a generic instrument to explain variation in the development of various competencies across educational setting and levels. However, two important conditions must be met for a valid measurement. First, formulations for indicators should be as concrete and straightforward as possible when designing a self-report; otherwise, interpretation problems are expected. Second, researchers have to critically overthink which competencies they want to assess and are advised to assess only competencies that are actually addressed in the learning context under study. Competencies that students do not specifically work on in their learning activities cause noise and ought to be excluded from self-reports. Such a self-report instrument is a valuable addition to the CB-education research and practice: It offers ample opportunities for examining and comparing the effectiveness of various CB-learning contexts in relation to qualification frameworks, and it offers opportunities for more evidence-based development and improvement of learning contexts with the aim of developing specific competencies of this framework. These insights can provide information for teachers to improve learning situations for developing certain competencies.

Limitations and Future Research

One limitation of the current study is the inclusion of only the secondary vocational and academic-level samples, although the COM was constructed for all tertiary educational levels (secondary vocational education, higher vocational education, and academic education). Furthermore, this study used a relatively small sample size and was of a more explorative, rather than confirmatory, nature. Nonetheless, this study has taken the first step in establishing validity of a contemporary competency self-report instrument: We have found evidence of face validity and construct validity of the competency self-report. The next steps in the construct validation process of the COM would be (a) examining its convergent and discriminant validity by comparing scores that should and should not be related to COM measurements, (b) examining the predictive validity of the COM (e.g., do higher competency scores lead to higher performance during internships or other work experiences?) to add to the lacking evidence of predictive validity of self-assessed competency measurements (Braun et al., 2012), (c) confirming construct validity by confirmatory factor analysis using a larger sample from all levels of tertiary education, and (d) then directly comparing the nature of the questionnaire responses between groups.

To further validate competency self-report instruments in general, it might also be interesting to test the other competencies and indicators from the Dutch Qualification framework included in the COM. In addition, we also suggest examining the construct validity and robustness of other existing competency self-reports used in different countries, from other qualification frameworks. It would be valuable to examine whether validation research on similar competency self-reports lead to the same findings.

Footnotes

Appendix A

Appendix B

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Ackerman

P. L.

Beier

M. E.

Bowen

K. R.

(2002). What we really know about our abilities and our knowledge. Personality and Individual Differences, 33, 587-605. doi:10.1016/S0191-8869(01)00174-X

Argüelles

Gonczi

(2000). Competency based education and training: A world perspective. Balderas, Mexico: Editorial Limusa S.A. de C.V. Grupo Noriego Editores.

Baartman

Gulikers

Dijkstra

(2013). Factors influencing assessment quality in higher vocational education. Assessment & Evaluation in Higher Education, 36, 978-997. doi:10.1080/02602938.2013.771133

Bartram

(2005). The great eight competencies: A criterion-centric approach to validation. Journal of Applied Psychology, 90, 1185-1203. doi:10.1037/0021-9010.90.6.1185

Benett

(1993). The validity and reliability of assessments and self-assessments of work-based learning. Assessment & Evaluation in Higher Education, 18, 83-94. doi:10.1080/0260293930180201

Biemans

Wesselink

Gulikers

Schaafsma

Verstegen

Mulder

(2009). Towards competence-based VET: Dealing with the pitfalls. Journal of Vocational Education & Training, 61, 267-286. doi:10.1080/13636820903194682

Blömeke

Zlatkin-Troitschanskaia

Kuhn

Fege

(2013). Modeling and measuring competencies in higher education. Rotterdam, The Netherlands: Sense Publishers.

Bowden

J. A.

Masters

G. N.

(1993). Implications for higher education of a competency-based approach to education and training. Canberra: Australian Government Publishing Service.

Boyatzis

R. E.

(1982). The competent manager: A model for effective performance. New York, NY: John Wiley.

10.

Braun

Woodley

Richardson

J. T. E.

Leidner

(2012). Self-rated competences questionnaires from a design perspective. Educational Research Review, 7, 1-18. doi:10.1016/j.edurev.2011.11.005

11.

Brockmann

Clarke

Méhaut

Winch

(2008). Competence-based vocational education and training (VET): The cases of England and France in a European perspective. Vocations and Learning, 1, 227-244. doi:10.1007/s12186-008-9013-2

12.

COLO. (2006). Competentiemodel kenniscentra beroepsonderwijs bedrijfsleven [Competency Framework vocational education and workplace]. Available from http://www.kwalificatiesmbo.nl

13.

Czaja

Blair

(2005). Designing surveys: A guide to decisions and procedures. Thousand Oaks, CA: Pine Forge Press.

14.

Field

(2009). Discovering statistics using SPSS. Thousand Oaks, CA: Sage.

15.

Grant

Elbow

Ewens

Gamson

Kohli

Neumann

. . . Riesman

(1979). On competence: A critical analysis of competence-based reforms in higher education. San Francisco, CA: Jossey-Bass.

16.

Groene Kennis Coöperatie. (2008). 4Cyourway. Available from http://www.4cyourway.nl/

17.

Hodkinson

Issitt

(1995). The challenge of competence: Professionalism through vocational education and training. London, England: Cassell.

18.

Le Deist

F. D.

Winterton

(2005). What is competence? Human Resource Development International, 8, 27-46. doi:10.1080/1367886042000338227

19.

Mulder

(2014). Conceptions of professional competence. Dordrecht, The Netherlands: Springer.

20.

O’Connor

B. P.

(2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, 32, 396-402. doi:10.3758/BF03200807

21.

Schaap

Baartman

de Bruijn

(2012). Students’ learning processes during school-based learning and workplace learning in vocational education: A review. Vocations and Learning, 5, 99-117. doi:10.1007/s12186-011-9069-2

22.

Schmitt

T. A.

(2011). Current methodological considerations in exploratory and confirmatory factor analysis. Journal of Psychoeducational Assessment, 29, 304-321. doi:10.1177/0734282911406653

23.

Schwarz

(1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93-105. doi:10.1037/0003-066X.54.2.93

24.

Sturing

Biemans

Mulder

Bruijn

(2011). The nature of study programmes in vocational education: Evaluation of the model for comprehensive competence-based vocational education in the Netherlands. Vocations and Learning, 4, 191-210. doi:10.1007/s12186-011-9059-4

25.

van den Herik

Winkler

(2008). Competentiepocket: Lexicon voor deelnemers en docenten [Competency pocket: Lexicon for participants and teachers]. Leeuwarden, The Netherlands: Eisma.

26.

Wesselink

Biemans

H. J. A.

Mulder

van den Elsen

E. R.

(2007). Competence-based VET as seen by Dutch researchers. European Journal of Vocational Training, 40, 38-51. Retrieved from http://www.researchgate.net/publication/236593405_Competence-based_VET_as_seen_by_Dutch_researchers/file/5046351827adb90a38.pdf#page=40

27.

Wesselink

de Jong

Biemans

H. J.

(2010). Aspects of competence-based education as footholds to improve the connectivity between learning in school and in the workplace. Vocations and Learning, 3, 19-38. doi:10.1007/s12186-009-9027-4

28.

Young

(2009). National qualification frameworks: An analytical overview. In Maclean

Wilson

(Eds.), International handbook of education for the changing world of work (pp. 2867-2880). Dordrecht, The Netherlands: Springer.