Development of the Ethical and Legal Issues in Counseling Self-Efficacy Scale

Abstract

The authors present the development of the Ethical and Legal Issues in Counseling Self-Efficacy Scale (ELICSES). The purpose of this article is threefold: (a) present a rationale for the ELICSES, (b) review statistical analysis procedures used to develop the ELICSES, and (c) offer implications for future research and counselor education.

Keywords

self-efficacy scale development exploratory factor analysis replication analysis

The strength of individuals’ self-efficacy is a powerful agent regarding their effort or willingness to attempt or complete a challenging task (Bandura, 1977, 1997). Self-efficacy is the extent to which an individual feels able to complete a task, activity, or reach a goal (Bandura, 1977, 1997). Thus, self-efficacy, whether low or high, may affect the decision-making process that occurs in a stressful situation. For example, a counselor who may feel incompetent with a counseling skill or intervention might avoid using it. Similarly, counselors may encounter situations in which they feel uncomfortable and/or lack confidence to resolve, resulting in avoidant behavior or poor decision making. Consequently, counselors’ self-efficacy may determine the likelihood and accuracy regarding how they act in unclear situations (e.g., ethical dilemmas).

Counselors abide by ethical standards and laws that guide their profession (e.g., American Counseling Association [ACA], 2005; American School Counselor Association [ASCA], 2010). Effective practitioners integrate the knowledge of laws and ethical standards into their work with clients (Kocet, 2006). Counselors’ ethical practice is an interactional process, involving the decision-making procedures that necessitate counselors to have the knowledge and confidence to make the appropriate decision. However, ethical and legal issues are diverse and complex (Corey, Corey, & Callanan, 2010), covering a multitude of topics that often lack a clear solution. Therefore, the development of ethical knowledge and its integration into clinical work is an important issue to examine.

Both ACA (2005) and the Council for Accreditation of Counseling and Related Educational Programs (CACREP; 2009) acknowledge the importance of preparing astute counselors who have the knowledge and confidence to handle unique ethical and legal circumstances. Counselors-in-training develop their ethical and legal knowledge during their initial coursework. Standards for the training of counselors are outlined by CACREP (2009), including the expectation that counselor preparation programs deliver content that fosters students’ development of ethical and legal knowledge, including the application of their acquired knowledge. Furthermore, counselor educators provide ongoing evaluation and appraisal of their counselors-in-training (ACA, 2005, Standard F.9.a). Therefore, an accurate evaluation method for assessing counseling trainees’ ethical and legal knowledge, and their confidence in applying their knowledge, is warranted.

Measures of Counselors’ Self-Efficacy and Ethical Processes

Counselors’ self-efficacy is linked to effective counseling (e.g., Bodenhorn, 2005; Larson & Daniels, 1998; Larson et al., 1992; Lent, Hill, & Hoffman, 2003). Researchers have developed self-efficacy scales to assess different domains in counseling. Larson et al. (1992) developed the Counseling Self-Estimate Inventory to measure counselors’ self-efficacy concerning (a) microskills, (b) attending to process, (c) dealing with difficult clients, (d) cultural competence, and (e) awareness of values. In addition, Lent et al. (2003) constructed the Counselor Activity Self-Efficacy Scale to assess counselors’ performance of helping skills, management of the counseling session, and resolution of challenging processes. Furthermore, Bodenhorn (2005) developed the School Counselor Self-Efficacy Scale to measure self-efficacy regarding specific behaviors and activities performed by school counselors. These three counseling-related self-efficacy scales measure specific domains regarding the performance of activities on the behalf of the counselor; however, no self-efficacy assessment instruments were identified that measure counselors’ self-efficacy in resolving ethical or legal issues.

Historically, researchers interested in assessing counselors’ levels of legal and ethical related behaviors and decision-making have measured them through indirect measures (e.g., Defining Issues Test [Rest, 1986] assessing counselors’ moral reasoning; Dufrene & Glosoff, 2004). However, Lambie, Hagedorn, and Ieva (2010) developed the Ethical and Legal Issues in Counseling Questionnaire (ELICQ) to measure counseling students’ and practitioners’ levels of ethical and legal knowledge. In addition, Dufrene and Glosoff (2004) developed the Ethical Decision-Making Scale–Revised (EDMS-R) to measure counselors’ ability to make ethical decisions. However, no assessment instruments were identified that measure both practicing counselors and trainees’ confidence in applying their ethical and legal knowledge (ethical and legal issues counseling self-efficacy).

We infer that counselor educators measure their trainees’ level of competency and/or understanding of ethical and legal issues through measures of knowledge recall (e.g., Counselor Preparation Competency Examination) and/or subjective supervisory evaluation of the appropriateness of ethical decision-making (e.g., programmatic internship student evaluations). Nevertheless, counseling students’ and practitioners’ self-efficacy in navigating ethical and legal issues also includes their ability to integrate cognitive, social, and behavioral skills to develop and facilitate an appropriate course of action (Bandura, 1982, 1997). Moreover, counselors’ willingness to engage in activities is influenced by their self-efficacy regarding the activity (e.g., ethical behavior; Bandura, 1997). Counselors’ lack of comfort and/or confidence regarding specific ethical or legal topics may lead to avoidant behavior and/or poor decisions. Assessing counseling students’ and practitioners’ self-efficacy regarding ethical and legal issues provides information that assists in tailoring pedagogy and clinical supervision to support the delivery of professional development, thus enhancing ethical service delivery to clients.

Development of the ELICSES

The development of the Ethical and Legal Issues in Counseling Self-Efficacy Scale (ELICSES) is in response to the limited availability of instruments that can be used to research and assess issues regarding ethical and legal topics in counseling and counselor education. As noted, a few measures assess ethical and legal constructs in counselors; however, no instruments measure self-efficacy regarding ethical and legal issues in counseling. Therefore, the ELICSES contributes to the literature and profession by providing (a) an assessment for evaluating counseling student self-efficacy regarding ethical and legal issues and (b) a mechanism for additional research on the topic of ethical and legal issues in counseling.

Initially, we sought to develop an instrument to measure the construct of confidence regarding ethical and legal issues in counseling. Self-efficacy is a primary factor in determining an individual’s motivation and agency regarding specific tasks or goals (Bandura, 1977, 1982, 1997); thus, it is a creative and appropriate method to research and assess this construct. Therefore, we decided to create a self-efficacy scale related to the construct we are seeking to measure (ethical and legal issues in counseling).

Prior to the data collection, the ELICSES was developed using scale development procedures (e.g., Crocker & Algina, 1986; DeVellis, 2012; Dimitrov, 2012). The steps in developing the ELICSES included the following: (a) decide on what to measure, (b) produce an item pool, (c) create the format for measurement, (d) have the initial item pool reviewed by experts, (e) think about inclusion of validation items, (f) administer items to a developmental sample, (g) evaluate the items, and (h) optimize scale length.

After conducting a comprehensive search in the counseling ethical and legal issues literature (e.g., textbooks, published articles, and codes of ethics) and a review of comparable measures (e.g., Ethical and Legal Issues in Counseling Questionnaire; Lambie et al., 2010), an initial item pool was developed. The initial ELICSES consisted of 40 items and 10 domain areas of ethical and legal issues, including (a) professional identity, (b) ethical and legal terms/concepts, (c) ethical decision-making, (d) confidentiality, privileged communication and privacy, (e) suicide and client violence, (f) abuse and neglect, (g) counseling and educational records, (h) educational and civil rights laws, (i) counselor development and wellness, and (j) discrimination laws and ethics. In addition, these domain areas were developed to assure a comprehensive breath of topics based on the literature.

After constructing the initial item pool, we created the format for the ELICSES. Both the question style and scoring method (e.g., 0–100) of the ELICSES were based on recommendations for self-efficacy scale development (Bandura, 2006). In addition, we used other self-efficacy scales as models (e.g., Bodenhorn & Skaggs, 2005). The overall appearance and layout of the ELICSES was derived from Tailor Design Method recommendations (Dillman, Smyth, & Christian, 2009). Figure 1 provides an example of ELICSES questions, appearance, and response options, including participant instructions.

After the ELICSES item pool was developed and formatted, we contacted expert reviewers to assess the appropriateness of each item. The reviewer pool consisted of 16 experts on the topic of ethical and legal issues within the field of counseling. Of the 16 reviewers, 14 were counselor educators at different CACREP-accredited counselor education programs, one was a counselor educator at a non-CACREP program, and one was a staff member for ACA. In addition, the expert reviewers represented diverse geographical locations (e.g., 14 different states; addressing different state-based legal issues). Furthermore, two expert reviewers serve on an ethics committee for two different national counseling associations (i.e., ACA, ASCA). Overall, the size and consistency of the expert review pool exceeded the recommended size identified in literature (e.g., DeVellis, 2012).

After including the input and recommendations from these 16 expert reviewers, the number of items increased to 54, and the number of ethical and legal issue domains increased to 12 (the two additional domains were [a] boundaries and relationships and [b] assessment and evaluation). The number of ELICSES items per ethical and legal domain area varied, ranging from three to six items per domain.

Next, we examined ELICSES through statistical data analysis procedures, which was the focus of our investigation. In this data analysis process, DeVellis’s (2012) recommendations to consider the use of validation items were followed (e.g., Marlowe–Crowne Social Desirability Scale; Strahan & Gerbasi, 1972). Therefore, the ELICSES and Marlowe–Crowne Social Desirability Scale (short version; Strahan, & Gerbasi, 1972) were administered to a developmental sample. The results of the ELICSES administration were analyzed (e.g., exploratory factor analysis and reliability analysis). After examining the ELICSES results, the scale length was optimized.

In summary, the ELICSES was developed to assess counselors’ self-efficacy regarding ethical and legal issues. The ELICSES offers an innovative method to examine issues related to ethical and legal issues in counseling. Our study investigated the psychometric properties (validity and reliability) of the ELICSES.

Method

Participants

The population targeted to develop the ELICSES included two groups: (a) counseling and therapist practitioners (mental health counselors [MHC], marriage and family therapist [MFT], and school counselors [SC]) and (b) counselors-in-training (specializing in mental health counseling, marriage and family therapy, and school counseling). To identify and invite practicing MHCs and MFTs to participate in the investigation, we accessed a public listing of all mental health practitioners in a southeastern state in the United States. The list of MHCs and MFTs was provided by the state’s department of health and included various forms of contact information (e.g., physical address, telephone number, and e-mail addresses). However, some contact information was missing due to the individual MHCs and MFTs choosing to exclude their information from publication. We invited all the MHCs and MFTs who provided an e-mail address to the state’s department of health to participate in the investigation. After combining the licensed MHCs and MFTs with the entry-level counseling practitioners (under 2 years of practice), we identified an accessible population of 7,771 MHCs and 1,540 MFTs. From this population, we randomly selected 1,500 participants for each specialty group, totaling 3,000 potential MHC and MFT participants. It should be noted that some participants reported having dual licensure in MHC and MFT. These participants with dual licensure were segregated in the results section when we examine the participant breakdown; however, their duel licensure status did not affect their participation in the investigation.

To identify and invite practicing SCs, we randomly selected school districts in the same southeastern state to acquire the potential participants’ contact information (e.g., e-mail address) from their respective school website, including 16 (23% of the state’s total) school districts, which resulted in an accessible population of 1,500 school counselors at the elementary, middle, and high school levels. To identify and invite counselors-in-training participants, we administered the data collection packet during four master’s-level counseling courses at a CACREP counselor education program in the same southeastern state (N = 97).

In total, we invited 4,597 counseling practitioners and students to participate in this study. The counseling practitioners (n = 4,500) were invited to participate through e-mail, and the students (n = 97) were invited to participate in person. Of the 4,500 potential e-mail participants, 222 emails did not work properly, resulting in 4,278 potential working e-mail addresses. The response rate for the counseling practitioners receiving the e-mail invitation to participate in the investigation was 15.5% (n = 664). Of these 664 responses, 487 (11.4% usable response rate for e-mailed counseling practitioners) completed all portions of the data collection instruments. As compared to mail survey research, e-mail surveys have a considerable lower response rate but can still be considered a viable tool (Shih & Fan, 2009). Nevertheless, the response rate was lower than normal for counseling and clinical psychology journals (Van Horn, Green, & Martinussen, 2009). Furthermore, many of the e-mail addresses may have been out of use or inaccurate, thus affecting the overall response rate. Based on the counseling practitioners who opened the e-mail to participate in the investigation (n = 664), most completed all the data collection instruments (n = 487; 73%). The counselors-in-training participants had a 100% (n = 97) response rate and completed all sections of the data collection packet. The total completed and usable data includes 584 participants (out of 4,375 possible) with a total usable response rate of 13.3%. Table 1 presents the demographic characteristics of the participants by subsample group (graduate student counselors-in-training and counseling practitioners).

Table 1.

Demographic Characteristics of the Sample.

	% (n)
Counselors-in-training
Gender
Female	80.4 (78)
Male	9.6 (19)
Discipline
School counseling	15.5 (15)
Mental health counseling	46.4 (45)
Marriage and family therapy	38.1 (37)
Ethnicity
African American	8.2 (8)
Asian American	8.2 (8)
Hispanic	13.4 (13)
Caucasian	68 (66)
Other ethnicity	2.1 (2)
Age
21-29	86.6 (84)
30-57	13.4 (13)
Counseling practitioners
Gender
Female	77 (375)
Male	23 (112)
Discipline
School counseling	28.3 (138)
Mental health counseling	39 (190)
Marriage and family therapy	15.8 (77)
Counselor educators	3.9 (19)
Mental health counseling & marriage and family therapy	4.3 (21)
School counseling and mental health counseling	3.9 (19)
Ethnicity
African American	7.6 (37)
Asian American	0.6 (3)
Hispanic	0.6 (3)
Caucasian	77.8 (379)
Multiracial	0.6 (3)
Native American	0.6 (3)
Pacific Islander	2 (1)
Other ethnicity	2.5 (12)
Age
24-29	9.7 (47)
30-39	20.3 (99)
40-49	23.8 (116)
50-59	22.8 (111)
60-69	20.3 (99)
70+	3.1 (15)

Procedure

This research project received approval from the institutional review board at our university before initiating research activity. All participants who were practicing counselors and therapists were contacted by e-mail with an invitation to participate in this study. To use e-mail survey, the data collection instruments were input into Survey Monkey (www.surveymonkey.com). Next, the practitioners received three emails: (a) initial explanation of the research and invitation to participate, (b) second invitation 7 days after the initial email, and (c) a final request for participation 7 to 10 days after the second invite. The survey, e-mail content and structure, and timings of delivery were based on the Tailor Design Method (Dillman et al., 2009). Each participant was assigned a random user identification code to ensure anonymity. Participants opened a link in the e-mail that directed them to the Survey Monkey page to complete the data collection instruments. Each participant was instructed to read a consent form before completing the instruments. The participants were removed from any future invitations once they started (but not necessarily completed) the data collection instruments.

The counselor-in-training participants were also invited to complete the same data collection instruments. The only difference between the instrumentation packets for the two groups of participants was the slight change in the demographics form to account for different professional roles. For the counseling students, four graduate-level courses in a counselor education program were invited to participate. The combination of courses produced a range of students who were at varying semesters in the program (first semester to eighth semester). For the counseling student administration, the first author visited four individual classes during the end of a semester to invite each student to participate in the study. All class visits and participant recruitment was completed with the course instructor’s approval. Each participant received an envelope with the study materials. Once completed, the student participants sealed the envelope and returned it to the researchers. Overall, we compiled 584 completed data collection packets from a wide range of participants who constitute the developmental population for the ELICSES.

Data Screening

Prior to data analysis, the data set was screened for missing data. Due to participant noncompletion of instrument or demographic form, there were 172 cases that were identified and marked missing data; thus, these cases were removed. The final sample size was 584 with an initial participant per item ratio greater than 10:1, which results in a moderately strong ratio of items to participants (Dimitrov, 2012; Hair, Black, Babin, Anderson, & Thatham, 2006). Therefore, the minimum amount of data for conducting an EFA with the ELICSES was satisfied.

Before conducting the EFA, we explored the normality of the data to determine the appropriate extraction method. To assess for normality, we (a) checked univarite normality of each item and (b) if item univarite normality was satisfied, we checked multivariate normality using the Mardia test (West, Finch, & Curran, 1995). The univarite normality of the individual ELICSES items was necessary for multivariate normality; thus, when we identified several items were not normally distributed, we assumed the multivariate data had severe nonnormality (West et al., 1995).

Instrumentation

We employed three data collection instruments: (a) the ELICSES, (b) the Marlowe–Crowne Social Desirability Scale–Short Form X1 (MCSDS-X1; Strahan & Gerbasi, 1972), and (c) a general demographics questionnaire. We introduce these three data collect instruments to set the context for our investigation.

ELICSES

The ELICSES is the instrument being examined in this study. It consists of 54 items and 12 domain areas of ethical and legal issues. Participants record their level of confidence regarding specific ethical and legal issues, including ratings from Cannot Do At All, Moderately Certain Can Do, and Highly Certain Can Do (ranging from 0 to 100). Figure 1 provides sample items and participant instructions. The ELICSES calculates a total score (e.g., overall self-efficacy) and subscale scores (e.g., based on identified factors) that give guidance regarding participants’ self-efficacy toward ethical and legal issues in counseling. The resulting psychometrics for the ELICSES is presented in the Results section. The internal consistency reliability for the ELICSES (initial 54 item version) was .98 with these data.

Figure 1.

Sample from the Ethical and Legal Issues in Counseling Self-Efficacy Scale.

Marlowe–Crowne Social Desirability Scale (Short-Form)

Participants completed the MCSDS-X1 (Strahan & Gerbasi, 1972), which is a shortened 10-item instrument that is derived from the original 33-item Marlowe–Crowne Social Desirability Scale (Crowne & Marlowe, 1960). Form X1 is correlated with a large effect size with the original 33-item version (e.g., .96; Cohen, 1992; Fischer & Fink, 1993). MCSDS is one of the most widely researched social desirability scales (Beretvas, Meyers, & Leite, 2002). A sample item from the MCSDS-X1 is, “I like to gossip at times.” Respondents either indicate the item as being true or false. MCSDS-X1 items that are geared to being more socially desirable responses receive a score of 1 and items that are not socially desirable receive a score of 0 (total scores range 0 to 10). The internal consistency reliability for the MCSDS-X1 have ranged from .50 to .88 (Ballard, 1992; Barger, 2002; Fischer & Fink, 1993; Strahan & Gerbasi, 1972). For this study, the Kuder-Richardson 20 reliability of the MCSDS-X1 was .69.

Demographics Questionnaire

All participants completed a demographics questionnaire. However, due to logical differences in career position, the counselors-in-training questionnaire varied slightly. Both counseling students and practitioners’ questionnaires ask participants general demographic information, such as gender, age, and ethnicity. In addition, both questionnaires inquired about time in the field/profession, current earned degrees, specialty or specialty track, and a Likert-type scale (0–100) on their overall comfort resolving ethical and legal issues. The practicing counselors also provided three to five common ethical dilemmas they face regularly. The researchers, counselor educators, and doctoral counselor education students revised and provided feedback regarding the demographic questionnaire, to support the assessment readability and face validity (no reviewers were potential participants).

Data Analysis

All data were analyzed using the Statistical Package for the Social Sciences (both Windows and Mac Versions 21.0). Initially, the data were screened for missing data. Then, we conducted an exploratory factor analysis (EFA) to examine the construct validity of the ELICSES. Based on the severe nonnormality of the data (Costello & Osborne, 2005; Fabrigar, Wegener, MacCallum, & Strahan, 1999), we used principal axis factoring method for extraction with an oblique rotation (Promax). Next, we examined the internal consistency using a calculation of Cronbach’s alpha, which assessed the degree of correlation among the ELICSES items. In addition, we examined the participant group total score differences using Mann–Whitney U test and Kruskal–Walis test (counselors-in-training and counseling practitioners). Last, we conducted a replication analysis to examine the reliability of the factor structure (Osborne & Fitzpatrick, 2012).

Prior to examining factor loadings, the extraction communalities were examined, we sought to remove items individually until they all were .5 or greater. During the EFA, items with a factor loading larger than .3 were identified (Floyd & Widaman, 1995; Hair et al., 2006) as a means of interpreting the factors. Items that loaded on multiple factors (cross-loading) were removed to simplify the interpretation. In addition, the criteria were operationalized by retaining only those items with factor loadings greater than or equal to .3. Furthermore, in the case of cross-loading ELICSES items, we sought to retain items if the loading factor was at least .2 greater than the next highest factor loading. The results of these data analyses resulted in a 23-item ELICSES.

Results

Social Desirability

All 584 participants completed the MCSDS-X1 (Strahan & Gerbasi, 1972). Mean ELICSES scores were positively correlated with the MCSDS-X1 (r = .315, p > .001), indicating a medium effect size (9.9% of the variance explained; Sink & Stroh, 2006). Therefore, the counseling students and practitioners scoring higher on the ELICSES also responded in a more socially desirable manner. These results identified that the respondents who self-rated higher levels of self-efficacy regarding ethical and legal issues in counseling also self-rated higher on the MCSDS-X1, placing them in a more positive light.

Construct Validity

EFA was used to examine factor loadings for the 54 item ELICSES in an effort to determine (a) how many factors underlie the ELICSES items, (b) which ELICSES items load on which factors, (c) the correlation between individual ELICSES items and factors, (d) the correlation (if any) among the ELICSES factors, and (e) what proportion of the variance in the ELICSES is accounted for by the factors (Dimitrov, 2012). All participant (e.g., practitioner and student) data were aggregated for the EFA. Multiple recognized criteria were used to assess factorability for the ELICSES data. The data were examined for multivariate normality and sampling adequacy using Barlett’s test of sphericity and Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy. Barlett’s test of sphericity produced a statistically significant value (χ² = 27856.66, df = 1431, p < .001), and the KMO measure of sampling adequacy was commendable (.97), well above the required .60 (Dimitrov, 2012; Hair et al., 2006). Thus, the ELICSES data were appropriate for EFA.

A principal axis factoring method was used for the extraction (due to nonnormal data), including an oblique (Promax) rotation due to an expected correlation (Costello & Osborne, 2005; Fabrigar et al., 1999; Hair et al., 2006). We identified criteria for retention of items as a result of the EFA, including (a) significant value for Bartlett’s test of sphericity, (b) a value of .5 or greater for KMO sampling adequacy for the whole instrument, (c) a .5 or greater measurement sampling accuracy (MSA) for each item, (d) a factor loading of .3 or greater (based on sample size), and (e) a .2 or greater difference between factor loadings for the same item (Hair et al., 2006). In addition, all ELICSES items with an extraction communality of .5 or greater were retained.

ELICSES Data Evaluation

Initially, the extraction communalities were examined with the goal of removing ELICSES items less than .5 (Hair et al., 2006), which resulted in the removal of 25 items. When removing extraction communalities, we went item by item seeking the best combination of item removal. Next, we examined items with significant factor loadings (i.e., .3 or higher; Hair et al., 2006) and found that all remaining items were significantly loading on at least one factor. Then, we removed items that loaded significantly (e.g., .3 or higher; Hair et al., 2006) on more than one factor (cross-loading), resulting in the removal of 14 more items (for a total of 39 items removed). This item evaluation process included the inspection of items with the aim to remove any cross-loadings that did not have a difference of .2. If an item held a cross-loading of .2 or less, the item was removed. The inspection and removal of item in relation to factor loading was done in a way that considered many possible combinations of factors to assure that the best solution was derived. Last, we reloaded the removed items individually to assess for the strongest and most inclusive factor solution, resulting in adding eight items back into the solution. Overall, the removal of these items led to a 23-item ELICSES, which resulted in the item to participant ratio increasing to slightly greater than 25:1. Next, the new and reduced ELICSES was examined through an EFA.

After reducing the ELICSES items to 23, the KMO was .96, and Bartlett’s test of sphericity was statistically significant at the .001 level (χ² = 11329.05, df = 253, p < .001). Most extraction communalities were above .5. However, two items (e.g., Item 43 [.44] and 53 [.49]) were below .5. We considered removing these items, but they both have significant factor loadings (e.g., Item 43 = .61 and Item 53 = .67) and contributed to the overall reliability as indicated through the assessment of the Cronbach alpha if the items were removed. All ELICSES items had an MSA over .5, with the lowest MSA at .819 (e.g., Item 6). The EFA yielded three factors with factor loadings ranging from .48 to .95 (see Table 3). Factor 1 had 13 items, Factor 2 had 7 items, and Factor 3 had 3 items, all of which are suitable amount (Hair et al., 2006). No significant cross-loadings occurred.

This rotation of ELICSES factors accounted for 64.59% in total variance, with eigenvalues greater than one for each factor. Factor 1 had an eigenvalue of 12.76, Factor 2 had an eigenvalue of 1.72, and Factor 3 had an eigenvalue of 1.36 (see Table 2). These eigenvalues met the condition for Kaiser’s rule (Mertler & Vannatta, 2005). In addition, accounting for 60% of variance, or less in some cases, is satisfactory for studies in social science (Hair at al., 2006). Examination of the scree plot is the recommended method for identifying the appropriate number of factors (Hair et al., 2006); thus, we examined the scree plot for this EFA. The scree plot provides additional support for a three-factor scale (ELICSES scree plot presented in Figure 2), there is a clear elbow between Points 3 and 4.

Table 2.

Factor Loadings for Exploratory Factor Analysis With Promax Rotation of ELICSES Scale Items.

	Factor			Communality Extracted
Initial Item Number and Topic Domain	1	2	3	Communality Extracted
Question 39—Educational and civil rights laws	0.88	−0.11	0.00	0.64
Question 34—Educational and civil rights laws	0.85	0.00	−0.06	0.67
Question 37—Discrimination laws and ethics	0.81	−0.06	0.09	0.66
Question 33—Ethical decision-making	0.80	0.02	0.02	0.69
Question 51—Boundaries and relationships	0.80	−0.02	−0.03	0.59
Question 25—Ethical and legal terms/concepts	0.78	0.02	−0.04	0.60
Question 52—Assessment and evaluation	0.76	0.02	0.00	0.59
Question 40—Ethical decision-making	0.68	0.21	0.06	0.78
Question 32—Boundaries and relationships	0.67	0.14	0.05	0.65
Question 53—Confidentiality, privileged communication, and privacy	0.67	0.10	−0.10	0.49*
Question 35—Professional identity	0.64	0.09	0.10	0.59
Question 30—Assessment and evaluation	0.60	0.20	−0.05	0.54
Question 23—Assessment and evaluation	0.54	0.22	0.02	0.54
Question 28—Abuse and neglect	−0.08	0.95	0.02	0.80
Question 41—Abuse and neglect	−0.01	0.89	0.03	0.80
Question 1—Abuse and neglect	−0.06	0.82	−0.04	0.56
Question 17—Abuse and neglect	0.08	0.78	−0.05	0.68
Question 29—Suicide and client violence	0.20	0.70	0.02	0.76
Question 31—Suicide and client violence	0.22	0.69	−0.01	0.75
Question 11—Boundaries and relationships	0.23	0.48	0.08	0.75
Question 6—Counselor development and wellness	−0.14	0.00	0.91	0.71
Question 36—Counselor development and wellness	0.08	−0.04	0.88	0.82
Question 43—Counselor development and wellness	0.07	0.03	0.62	0.45*
Eigenvalue	12.76	1.72	1.36

Note. ELICSES = Ethical and Legal Issues in Counseling Self-Efficacy Scale; Factor 1 = General Ethical and Legal Issues Self-Efficacy; Factor 2 = Suicide, Violence, Abuse and Neglect Self-Efficacy; Factor 3 = Counselor Development and Wellness Self-Efficacy. Boldface values indicate significance factor loading (i.e., .3 or higher).

Figure 2.

Scree plot for final EFA with 23 ELICSES items.

Once the EFA was finalized, we examined the data for internal consistency by calculating Cronbach alphas. A Cronbach alpha of .70 or higher is a standard for assessing internal consistency (Mitchell & Jolley, 2004). The Cronbach alpha for the resulting 23 ELICSES items was (.96), indicating strong internal consistency reliability (Drummond & Jones, 2010). Next, we examined the internal consistency for the three separate ELICSES factors. ELICSES Factor 1 (13 items; General ethical and legal issues in counseling self-efficacy) had a Cronbach alpha of .95, Factor 2 (7 items; Suicide, violence, abuse and neglect self-efficacy) had a Cronbach alpha of .94, and Factor 3 (3 items; Counselor development and wellness self-efficacy) had a Cronbach alpha of .85, supporting internal consistency for each factor.

Participant Subgroup Comparison

Additional analysis examined the total mean ELICSES score differences between the subgroups of respondents. First, we compared to total ELICSES score mean difference between practitioners and counselors-in-training. The assumption of normality was tested but not met. Review of the Shapiro–Wilk’s test for normality (p < .000), skewness (−1.444), and kurtosis (1.736) statistics and box plot indicate that normality may not be assumed for the practitioner group. In addition, a review of the Shapiro–Wilk’s test for normality (p < .000), skewness (−.805), and kurtosis (.272) statistics and box plot indicate that normality may not be assumed for the counselor-in-training group. As a result, a Mann–Whitney U test was employed. The resulting analysis identified that there was a statistical difference (U = 5350, z = −12.04, p < .000) in distribution of score rankings between practitioners (M_rank = 330.01) and counselors-in-training (M_rank = 104.15). Practitioner participants scored higher on the ELICSES as compared to the counselors-in-training participants; therefore, these results support that practitioners (e.g., experienced counselors and therapist) report higher levels of self-efficacy with regard to ethical and legal issues in counseling.

Next, we examined the total mean ELICSES score differences between the groups of practitioners (e.g., MHC, MFT, and SC). The assumption of normality was tested but not met. Review of the Shapiro–Wilk’s test for normality (p < .000), skewness (−.902), and kurtosis (.738) statistics and box plot indicate that normality may not be assumed for the mental health counselor group. Review of the Shapiro–Wilk’s test for normality (p = .003), skewness (−.781), and kurtosis (.292) statistics and box plot indicate that normality may not be assumed for the marriage and family therapist group. Also, a review of the Shapiro–Wilk’s test for normality (p < .000), skewness (−1.47), and kurtosis (3.14) statistics and box plot indicate that normality may not be assumed for the school counselor group. Consequently, a Kruskal–Wallis analysis was used to inspect the difference among the total mean ELICSES scores of these groups. The resulting analysis identified a statistically significant difference between the different practitioner specialty groups (χ²[2] = 15.75, p < .000), with a mean rank of 255.42 for MHCs, 194.72 for MFTs, and 221.74 for SCs.

A Mann–Whitney U post hoc test was completed to evaluate the ELICSES differences among the three groups as a result of the aforementioned findings. The resulting analysis identified a significant statistical difference (U = 7974.5, z = −3.85, p < .000) in distribution of score rankings between MHCs (M_rank = 167.53) and MFTs (M_rank = 127.45), a significant statistical difference (U = 12371, z = −2.37, p = .018) in distribution of score rankings between MHCs (M_rank = 183.39) and SCs (M_rank = 157.86), and no significant statistical difference (U = 7668.5, z = −1.69, p = .092) in distribution of score rankings between MFTs (M_rank = 124.77) and SCs (M_rank = 140.88). These results indicate that MHCs respondents scored higher on the ELICSES as compared to both MFTs and SCs; however, there was no difference in the ELICSES responses between MFTs and SCs.

Replication Analysis

Further analysis was conducted to assess reliability of the ELICSES using replication analysis (see Table 3; Osborne & Fitzpatrick, 2012). The internal replication analysis required us to create two separate data sets (n = 292; ELICSES Items = 23; Item/participant ratio slightly greater than 12:1) that were both used to conduct an EFA using the same extraction method (Principal Axis Factoring) and rotation procedure (Oblique, Promax). The initial replication criterion, structural replication, was met (Osborne & Fitzpatrick, 2012). That is, the same ELICSES items were identified for the same number of factors for both the replication analysis samples and the full EFA sample. Additionally, the data split analysis held the same factorial structure. Next, we examined the replication analysis by looking at the squared differences between factor loadings for each split sample.

Table 3.

Internal Replication (IR) Analysis Based on the Results of the EFA.

	Sample 1 (n = 292)				Sample 2 (n = 292)
	Communality Extract	Factor Loading			Communality Extract	Factor Loading			Analysis
	Communality Extract	1	2	3	Communality Extract	1	2	3	Squared Diff.
Q1	0.65	−0.02	0.85	−0.08	0.50	−0.11	0.79	−0.01	0.0038
Q6	0.68	−0.14	−0.01	0.90	0.75	−0.12	0.01	0.91	0.0003
Q11	0.50	0.25	0.46	0.07	0.53	0.16	0.56	0.08	0.0096
Q17	0.70	0.08	0.82	−0.12	0.66	0.08	0.73	0.05	0.0092
Q23	0.63	0.64	0.22	−0.05	0.42*	0.38	0.25	0.08	0.0681*
Q25	0.64	0.77	0.03	0.02	0.56	0.77	0.03	−0.10	0.0000
Q28	0.80	−0.15	0.97	0.08	0.79	0.02	0.90	−0.05	0.0053
Q29	0.77	0.16	0.74	0.03	0.74	0.27	0.63	0.01	0.0108
Q30	0.62	0.56	0.28	−0.02	0.45	0.64	0.08	−0.08	0.0061
Q31	0.76	0.15	0.76	0.00	0.73	0.32	0.59	−0.01	0.0299
Q32	0.69	0.66	0.16	0.06	0.61	0.67	0.12	0.02	0.0000
Q33	0.66	0.77	0.02	0.05	0.73	0.84	0.02	0.00	0.0053
Q34	0.65	0.82	0.02	−0.05	0.69	0.86	−0.01	−0.07	0.0019
Q35	0.65	0.66	0.08	0.14	0.51	0.57	0.14	0.06	0.0085
Q36	0.84	0.06	−0.02	0.89	0.78	0.13	−0.09	0.85	0.0017
Q37	0.68	0.80	−0.08	0.14	0.66	0.83	−0.05	0.05	0.0009
Q39	0.64	0.95	−0.18	−0.04	0.65	0.78	0.00	0.05	0.0276
Q40	0.79	0.68	0.17	0.11	0.76	0.63	0.29	0.01	0.0031
Q41	0.78	−0.05	0.89	0.06	0.82	0.05	0.87	0.00	0.0007
Q43	0.49*	0.11	0.00	0.64	0.39*	−0.05	0.11	0.60	0.0018
Q51	0.61	0.84	−0.02	−0.08	0.58	0.75	0.01	0.02	0.0081
Q52	0.59	0.76	0.05	−0.05	0.60	0.76	−0.01	0.05	0.0000
Q53	0.54	0.67	0.17	−0.12	0.46*	0.69	0.02	−0.05	0.0004
Eigenvalue		13.05	1.84	1.42		12.44	1.73	1.23

Note. Boldface values indicate significance (i.e., .35 or higher; n = 292; Hair et al., 2006). Asterisks indicate items that violated criteria for a strong replication. Factor structure congruent between samples. One significant (i.e., ≥.04) squared different values found (e.g., Item 23). One item in Sample 1 and three items in Sample 2 with low communality (i.e., ≤.5).

The second criterion to verify the replicability of the identified factors is to examine the squared difference between the factor loadings of each item and between the two split samples, with any squared difference .04 or higher (which indicates a .2 difference or higher between replicated factors) being grounds to deem them as volatile (Osborne & Fitzpatrick, 2012). Based on this replication analysis, all the ELICSES items were strong and worth keeping except one item (e.g., Item 23). This item had a squared difference greater than .04 for this replication; thus, it was considered for removal, but we decided to keep them in order to hold true to the theoretical integrity of the topic domains. In future investigations on the ELICSES, researchers can consider replicating this result, removing or modifying this variable to improve the replicability.

We further examined the replicated EFA. The internal replication analysis produced a few ELICSES items in the samples that are below the communality extraction criteria established for the initial EFA (e.g., Item 43 for Sample 1; Items 23, 43, 53 for Sample 2). However, two of these results (e.g., Items 43 and 53) were expected based on the initial EFA results. Also, the low communality in Item 23 is expected considering its high squared difference. This result did not support the inclusion of these three items. Furthermore, neither split sample contained a significant cross-loading (e.g., .35 based on 292 participants; Hair et al., 2006).

Additionally, we examined the reliability of the ELICSES with the split data samples used for the replication analysis. Split Sample 1 had a Cronbach’s alpha of .96, and the Split Sample 2 had a Cronbach of alpha of .96. Regarding the factor loading group Cronbach’s for Split Sample 1, Factor 1 has a Cronbach alpha of .96, Factor 2 has a Cronbach alpha of .95, and Factor 3 has a Cronbach alpha of .82, supporting internal consistency for each factor. The factor loading group Cronbach’s for Split Sample 2 were as follows: Factor 1 has a Cronbach’s alpha of .95, Factor 2 has a Cronbach’s alpha of .94, and Factor 3 has a Cronbach alpha of .84, supporting internal consistency for each factor. The reliability for the replication analysis was consistent between split samples. In addition, both split samples were similar to the full sample. Overall, the replication analysis results supported the reliability of the derived ELICSES items from the original EFA.

Discussion

This study explored the factor structure of the ELICSES with a sample of counseling students and practitioners. Social desirability of the participants was measured using the MCSDS-X1 (Strahan & Gerbasi, 1972) and indicated a medium correlation between overall score on the ELCSES and a desire to be seen in a positive light. The findings from the EFA support a three factor-solution for the ELICSES. Using a principal axis factoring method and promax rotation supported the three ELICSES factors labeled as (a) general ethical and legal issues in counseling self-efficacy; (b) suicide, violence, abuse and neglect self-efficacy; and (c) counselor development and wellness self-efficacy. Within these three ELICSES factors, all 12 ethical and legal issue content domains are represented. However, each ELICSES factor does not represent all of these 12 domain areas (see Table 2).

These results support the use of the ELICSES total score as compared to the individual subscales. However, if a researcher is examining counselors’ ethical and legal confidence regarding suicide and violence or counselor development and wellness, they may find either of these two subscales useful and appropriate. Yet, we suggest that researchers not use Factor 1 independently because the independent factors lack content domain areas crucial to ethical and legal issues (e.g., suicide and violence, counselor development and wellness). Again, our results support the use of the ELICSES total score as an assessment of counselors’ (practitioners and students) overall self-efficacy regarding ethical and legal issues. To use the ELICSES, researchers can examine the overall score or subscale scores. Table 4 provides the ELICSES items, mean scores, and standard deviations. In addition, Table 5 presents the ELICSES measures of central tendency and dispersion scores based on participant pool.

Table 4.

Ethical and Legal Issues Self-Efficacy Statements and Descriptive Statistics.

	Factor and Initial Item Number	Scores
	Factor and Initial Item Number	M	SD
1. Distinguish when it is appropriate to report a case of abuse.	2 (Q1)	90.57	12.79
2. Maintain wellness to support my own counselor functionality.	3 (Q6)	85.39	15.34
3. Determine whether a client and counselor interaction is a boundary violation.	2 (Q11)	89.55	13.10
4. Recognize when it is appropriate to report a case of neglect.	2 (Q17)	90.50	13.00
5. Diagnose a presenting concern with client and/or student.	1 (Q23)	85.43	17.24
6. Define ethical and legal terms (e.g., subpoena, due process, & confidentiality).	1 (Q25)	85.55	17.56
7. Recognize signs of abuse and/or neglect.	2 (Q28)	89.76	11.94
8. Respond to potentially suicidal clients and/or students.	2 (Q29)	89.42	13.89
9. Interpret a formal diagnosis of a mental and/or emotional disorder.	1 (Q30)	83.36	19.61
10. Take appropriate actions regarding potentially suicidal clients.	2 (Q31)	89.79	14.32
11. Circumvent negative dual relationships.	1 (Q32)	86.64	15.42
12. Describe the steps to take when facing an ethical dilemma.	1 (Q33)	85.05	16.00
13. Demonstrate knowledge of civil rights laws regarding clients and/or students.	1 (Q34)	80.26	19.34
14. Explain the professional role of a counselor.	1 (Q35)	91.80	12.47
15. Develop appropriate behaviors to support counselor wellness.	3 (Q36)	88.37	13.21
16. Comprehend discrimination laws and ethics as they apply to my clients and/or students.	1 (Q37)	85.02	15.89
17. Interpret the federal acts or laws to clients and/or students as they pertain to a presenting issue or situation.	1 (Q39)	78.46	19.66
18. Resolve an ethical dilemma.	1 (Q40)	86.44	14.24
19. Identify possible symptoms of abuse and/or neglect.	2 (Q41)	89.04	12.69
20. Recognize possible feelings of burnout within myself.	3 (Q43)	86.56	13.71
21. Recognize appropriate multiple relationships.	1 (Q51)	86.88	15.25
22. Implement and report assessments effectively.	1 (Q52)	83.39	18.38
23. Respond to a subpoena for records.	1 (Q53)	81.83	21.98
Ethical & Legal Issues Self-Efficacy (Total Score)	—	86.48	11.48
General Ethical & Legal Issues Self-Efficacy (Factor 1)	—	84.62	13.67
Suicide, Violence, Abuse & Neglect Self-Efficacy (Factor 2)	—	89.80	11.14
Counselor Development & Wellness Self-Efficacy (Factor 3)	—	86.77	12.24

Note. All items measured on a scale of 0 to 100 (0 = Cannot Do At All and 100 = Highly Certain Can Do). Contact first author to request permission for use.

Table 5.

Participants’ ELICSES Total Score Characteristics.

Participant Pool	Mean	Median	Mode	SD	Range (min/max)
Practicing mental health counselors (n = 190)	90.16	91.52	100	7.76	37.39 (62.61/100)
Practicing marriage and family therapists (n = 77)	90.15	90.3	92.17	7.06	32.61 (67.39/100)
Practicing school counselors (n = 138)	88.97	90.43	90	8.29	46.65 (54.35/100)
Multiple specialties (n = 40)	90.66	91.96	81.3	7.59	30.43 (69.57/100)
Counselors-in-training (n = 97)	71.45	73.91	78.26	14.11	66.52 (29.13/95.65)
Total sample (N = 584)	86.48	85.56	100	11.48	70.87 (29.13/100)

Limitations

Several limitations are apparent. First, the sample population is specific to a single state; thus, it is not representative of diverse geographical locations. Nevertheless, our investigation was the first implementation of ELICSES and may support future investigations to support our results. A second study limitation was the sample demographics. The majority of the sample identified as White, Female participants, thus excluding the perspective from other diverse cultures. In addition, the nonresponse rate was high. However, the lower response rate was somewhat expected as a result of the data collection methodology (e.g., email survey). Furthermore, the specific methodology used (e.g., online survey completion) and instruments used (e.g., ELICSES, Social Desirability Scale, and Demographic information) may have limited the richness and accuracy of the resulting data. Another limitation may be the loss of items for specific topics as a result of item reduction. The items removed may have measured respondent information in a manner that cannot be assessed with their absence. Nonetheless, the removal process was specific and intentional with the hope to assure a concise, reliable, and highly correlated measure.

Implications for Counselor Education and Future Research

The results of our investigation identified three factors underlying the ELICSES’s 12 content domains, offering implications for counselor educators. First, counselor educators can use the ELICSES to measure students’ progress through courses. Specifically, the ELICSES can be used in courses on ethical and legal issues in counseling as a way to measure students’ self-efficacy regarding ethical and legal issues. A second study implication includes the use of the ELICSES as a tool for conducting research on ethical and legal topics. While some measures related to counseling ethics exist, there are limited methods in which researchers can investigate counseling ethics. Thus, the ELICSES contributes to the available resources to research ethical and legal issues in the practice of counseling. In addition, the ELICSES can be a measure to assist in program evaluation. Specifically, whether in a clinical/education (e.g., school district, counseling clinic) or in a counselor education training program (e.g., graduate-level counseling program), the ELICSES can measure counselors’ and/or counseling students’ confidence in resolving issues related to ethical and legal issues. Therefore, research-based program evaluation employing the ELICSES may support continuous programmatic enhancement and refinement.

This investigation was the initial study on the ELICSES. As such, future studies are merited to continue to develop and refine the ELICSES. Researchers interested in the ELICSES can conduct a confirmatory factor analysis to test the fit of the ELICSES model with different samples of counseling professionals. In addition, future research should replicate this study with a more diverse (i.e., geographical and cultural) population. A larger sample may include participants from multiple states and, ideally, higher percentage of culturally diverse participants. In addition, studies may seek to assess test–retest reliability. Another future research area may be assessing the concurrent validity of the ELICSES with other validated instruments. Furthermore, researchers may employ longitudinal research design to investigate changes in counselors and/or students ELICSES scores over time. Last, researcher may wish to develop and/or assess the item pool of the ELICSES in an attempt to assure the representation of broad ethical and legal topics and domains.

Conclusion

In summary, our investigation examined the psychometric properties of the ELICSES. The results provide support for the use of the ELICSES as an assessment of counselors’ and counseling students’ level of confidence when faced with ethical and legal issues. The ELICSES offers counselor educators a tool to aid in the assessment of students and/or practitioners. Thus, counselor educators may tailor their pedagogical strategies to support higher confidence in students regarding ethical and legal issues. In addition, the ELICSES may serve as a tool in research topics related to counseling ethics and laws. Furthermore, the ELICSES may serve as an instrument to aid in program evaluation. Therefore, the ELICSES is an instrument that contributes to both preparing counselors and conducting research in counselor education.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was financially supported by a Southern Association for Counselor Education and Supervision (SACES) research grant.

Author Biographies

Patrick R. Mullen, MA, is a doctoral candidate in the Counselor Education Program at the University of Central Florida. His research interests include counselor education and supervision, professional school counseling, effective interventions for children and adolescents, and research methodology.

Glenn W. Lambie, PhD, is an associate professor and serves as the Associate Chair for the Department of Child, Family, & Community Sciences at the University of Central Florida. He has published over 50 articles in national refereed journals related to counselor education and supervision, counseling children and adolescents, and professional school counseling. In addition, Dr. Lambie developed and/or co-developed five different assessment instruments related to counselor development and supervision.

Abigail H. Conley, PhD, is Assistant Professor of Counselor Education at Virginia Commonwealth University. Her research interests include interpersonal violence survivorship, spiritual diversity, and best practices in program evaluation and research methodology. Dr. Conley currently serves as the Associate Quantitative Editor of the Counseling and Values Journal.

References

American Counseling Association. (2005). Code of ethics. Alexandria, VA: Author.

American School Counselor Association. (2010). Ethical standards for school counselors. Alexandra, VA: Author.

Ballard

(1992). Short forms of the Marlowe–Crowne social desirability scale. Psychological Reports, 71, 1155–1160.

Bandura

(1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall.

Bandura

(1982). Self-efficacy mechanism in human agency. American Psychologist, 37, 122–147.

Bandura

(1997). Self-efficacy: The exercise of control. New York, NY: W. H. Freeman.

Bandura

(2006). Guide for constructing self-efficacy scales. In Pajares

Urdan

T. C.

(Eds.), Self-efficacy beliefs of adolescents (pp. 307–337). Greenwich, CT: Information Age.

Barger

(2002). The Marlowe–Crowne affair: Short forms, psychometric structure, and social desirability. Journal of Personality Assessment, 79, 286–305.

Beretvas

S. N.

Meyers

J. L.

Leite

W. L.

(2002). A reliability generalization study of the Marlowe–Crowne Social Desirability scale. Educational and Psychological Measurement, 62, 570–589.

10.

Bodenhorn

Skaggs

(2005). Development of the school counselor self-efficacy scale. Measurement and Evaluation in Counseling and Development, 38, 14–28.

11.

Cohen

(1992). Power primer. Psychological Bulletin, 112, 155–159.

12.

Corey

M. S.

Callanan

(2010). Issues and ethics in the helping professions. Melbourne, Australia: Brooks/Cole/Thomson Learning.

13.

Council for Accreditation of Counseling and Related Educational Programs. (2009). 2009 CACREP standards. Alexandria, VA: Author.

14.

Costello

A. B.

Osborne

J. W.

(2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, & Evaluation, 10(7). Retrieved from http://pareonline.net/pdf/v10n7.pdf

15.

Crocker

L. M.

Algina

(1986). Introduction to classical and modern test theory. New York, NY: Holt, Rinehart, & Winston.

16.

Crowne

D. P.

Marlowe

(1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349–354.

17.

DeVellis

R. F.

(2012). Scale development: Theory and applications. Thousand Oaks, CA: Sage.

18.

Dillman

D. A.

Smyth

J. D.

Christian

L. M.

(2009). Internet, mail, and mixed-mode surveys: The tailored design method. Hoboken, NJ: Wiley.

19.

Dimitrov

D. M.

(2012). Statistical methods for validation of assessment scale data in counseling and related fields. Alexandria, VA: American Counseling Association.

20.

Drummond

R. J.

Jones (2010). Assessment procedures for counselors and helping professionals. Boston, MA: Pearson.

21.

Dufrene

R. L.

Glosoff

H. L.

(2004). The Ethical Decision-Making Scale–Revised. Measurement and Evaluation in Counseling and Development, 37, 2–37.

22.

Fabrigar

L. R.

Wegener

D. T.

MacCallum

R. C.

Strahan

E. J.

(1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272–299.

23.

Fischer

D. G.

Fick

(1993). Measuring social desirability: Short forms of the Marlowe–Crowne Social Desirability Scale. Educational and Psychological Measurement, 53, 417–424.

24.

Floyd

F. J.

Widaman

K. F.

(1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7, 286–299.

25.

Hair

J. F.

Black

W. C.

Babin

B. J.

Anderson

R. E.

Tatham

R. L.

(2006). Multivariate data analysis. Upper Saddle River, NJ: Pearson.

26.

Kocet

M. M.

(2006). Ethical challenges in a complex world: Highlights of the 2005 ACA code of ethics. Journal of Counseling and Development, 84, 228–234.

27.

Lambie

G. W.

Hagedorn

W. B.

Ieva

K. P.

(2010). Social-cognitive development, ethical and legal knowledge, and ethical decision making of counselor education students. Counselor Education and Supervision, 49, 228–246.

28.

Larson

L. M.

Daniels

J. A.

(1998). Review of the counseling self-efficacy literature. The Counseling Psychologist, 26, 179–218.

29.

Larson

L. M.

Suzuki

L. A.

Gillespie

K. N.

Potenza

M. T.

Bechtel

M. A.

Toulouse

A. L.

(1992). Development and validation of the counseling self-estimate inventory. Journal of Counseling Psychology, 39, 105–120.

30.

Lent

R. W.

Hill

C. E.

Hoffman

(2003). Development and validation of the counselor activity self-efficacy scales. Journal of Counseling Psychology, 50, 97–108.

31.

Mertler

C. A.

Vannatta

R. A.

(2005). Advanced and multivariate statistical methods: Practical application and interpretation. Glendale, CA: Pyrczak.

32.

Mitchell

M. L.

Jolley

J. M.

(2004). Research design explained (5th ed.). Pacific Grove, CA: Wadsworth.

33.

Osborne

J. W.

Fitzpatrick

D. C.

(2012). Replication analysis in exploratory factor analysis: What it is and why it makes you analysis better. Practical Assessment, Research & Evaluation, 17(15). Retrieved from http://pareonline.net/pdf/v17n15.pdf

34.

Rest

J. R.

(1986). Manual for the defining issues test. Minneapolis: University of Minnesota, Center for the Study of Ethical Development.

35.

Shih

Fan

(2009). Comparing response rates in e-mail and paper surveys: A meta-analysis. Educational Research Review, 4(1), 26–40.

36.

Sink

C. A.

Stroh

(2006). Practical significance: Use of effect sizes in school counseling research. Professional School Counseling, 9, 401–411.

37.

Strahan

Gerbasi

(1972). Short, homogeneous versions of the Marlowe-Crowne social desirability scale. Journal of Clinical Psychology, 28, 191–193.

38.

Van Horn

P. S.

Green

K. E.

Martinussen

(2009). Survey response rates and survey administration in counseling and clinical psychology: A meta-analysis. Educational & Psychological Measurement, 69, 389–403.

39.

West

S. G.

Finch

J. F.

Curran

P. J.

(1995). Structural equation models with non-normal variables: Problems and remedies. In Hoyle

R. H.

(Ed.), Structural equation modeling: Concepts, issues and applications (pp. 56–75). Thousand Oaks, CA: Sage.