Abstract
The authors provide psychometric support for the comprehensive counseling skills rubric (CCSR). The CCSR is designed to assess trainee’s counseling proficiency, identify how counseling skills are used within phases of a counseling session, and evaluate the counseling relationship. The CCSR content validity was verified with an overall mean item content validity index of .81. There was a strong domain-level internal consistency reliability (.73 to .84). Fleiss’ κ of .70 demonstrated substantial interrater reliability. Thus, the results indicate initial support for using the CCSR as a transtheoretical instrument designed to assess a counseling trainee’s ability to execute the counseling experience.
Counseling skills, phases of a counseling session, and aspects of the counseling relationship are essential areas of competency for counseling trainees (Cormier, Nurius, & Osborn, 2009; Ivey, Ivey, & Zalaquett, 2010; Young, 2013). Further, the use of external feedback to evaluate these competencies is integral to counselor preparation (Skovholt & Ronnestad, 1992) and supervision (Bernard & Goodyear, 2014), as trainees develop a professional counseling identity (Stoltenberg & Delworth, 1987). Although the Council for Accreditation of Counseling and Related Educational Programs (CACREP) standards (2009, 2015) mandate skill acquisition within all areas of professional practice and specialization (see third section), counselor educators and supervisors may not be evaluating skills at different points in the curriculum, and programs may lack empirically based instruments. A dearth of comprehensive appraisal methods is problematic, as literature notes inconsistent preparation among counseling trainees (Green, 2011; Truax & Carkhuff, 1967), great diversity in the ability of supervisors to evaluate skills (Gaubatz & Vera, 2002; Motley, Reese, & Campos, 2014), and limited standardized and comprehensive evaluation methods (Swank, Lambie, & Witta, 2012) that assess trainees’ developmental progression throughout preparation (Bernard & Goodyear, 2014).
The counseling literature has endorsed using various empirically validated classification systems to assess counseling skills, dispositions, and behaviors (e.g., Hamlet & Burnes, 2013; Swank et al., 2012); attending and multicultural skills (e.g., Larson et al., 1992; Ponterotto & Potere, 2003); counselor effectiveness (e.g., Eriksen & McAuliffe, 2003; Horvath & Greenberg, 1989); cross-cultural counseling competencies (e.g., Kocarek, Talbot, Batka, & Anderson, 2001; Sodowsky, Taffe, Gutkin, & Wise, 1994); trainee verbal response to clientele (Porter, 1943); and facilitative conditions (Truax & Carkhuff, 1967). We found no published research on the development of a counseling assessment instrument designed to measure a trainee’s counseling skill ability and to determine how these skills are utilized within various phases of a counseling session and within the counseling relationship. Therefore, there is a need in counselor education and supervision for a comprehensive counseling skills rubric (CCSR).
Given the lack of available instruments for comprehensive and systematic evaluation of skills, we designed the CCSR to assess trainee skills within various phases of a counseling session and within the counseling relationship. Further, the CCSR was intended to be utilized with all counseling specialty areas (e.g., clinical mental health, school, college, marriage, and family counseling). Having a counseling skills rubric, such as the CCSR, could allow for trainee self-assessment, a formative assessment of changes in trainee’s skill level after each counseling session, comparisons in trainee’s skill level with different clients or client issues, measurement of changes in skill level at various points in training, evaluation of the effectiveness of supervision interventions, and detection of professional impairment and programmatic training deficiencies. The purpose of this study was to describe the item development and validation of the CCSR. We addressed the following research questions: (1) What is the initial item development and rubric design of the CCSR? (2) What is the content validity evidence of the CCSR? (3) To what degree do participants identify appropriate skills in a counseling session using the CCSR? and (4) How do participants rate skill level using the CCSR?
Method
We used an embedded quantitative design (Creswell & Plano Clark, 2010) to evaluate the strength of various item ratings while simultaneously gathering qualitative feedback to enhance the rubric. To address our research questions, we engaged in the following phases during development and initial validation of the CCSR: (1) initial item development; (2) assessment of content validity to further refine the CCSR using two panels (7 counselor educators and 17 students); (3) application of the CCSR for rating a digitally recorded mock counseling session to identify utility of the CCSR and to document agreed-upon skills present in the session, using four independent raters and tabulating interrater agreement and discrepancies; and (4) assessment of interrater reliability of the CCSR, using 63 counselor educators, counselor education doctoral students, and practicing professional counselors to review a mock counseling session and use the CCSR to assess the skills being utilized and the quality of skill execution.
Participants
Using a criterion-based sampling procedure, we identified and selected participants who fit within this study’s objectives. Participants included two expert panels for the content validity portion (Phase 2) and two groups to establish interrater agreement (Phases 3 and 4; see Table 1). For Phase 2, the first panel consisted of seven counselor educators employed in CACREP-accredited programs, who had expertise in test development (e.g., refereed publications in the area of test development, experience in mentoring trainees in the test development process, and experience in writing books on test development) and experience in teaching a skill-based course (e.g., practicum, internship, counseling skills, and prepracticum). We recruited a second panel of 17 professional counseling and school psychology students enrolled in a masters level program. All student participants had taken at least one counseling skills course during their respective programs.
Participant Demographics.
Note. SD = standard deviation; ACES = Association for Counselor Education and Supervision; NARACES = North Atlantic Region Association for Counseling Education and Supervision; SACES = Southern Association for Counselor Education and Supervision; RMACES = Rocky Mountain Association for Counselor Education and Supervision; WACES = Western Association for Counselor Education and Supervision; NCACES = North Central Association for Counselor Education and Supervision. Demographic data may go beyond 100% due to participants providing multiple responses.
For the interrater agreement phase (Phase 3), we recruited four counselor educators with comparable expertise as the seven expert panelists in Phase 2. All interrater agreement participants taught in a CACREP-accredited program, had instrument development expertise, and had experience with counselor skill preparation. Participants (N = 63) for the criterion validity and interrater reliability phase (Phase 4) were counselor educators, advanced counselor education doctoral students in CACREP-accredited programs, or practicing counselors.
Procedures
Prior to engaging in the research process, we received institutional review board (IRB) approval. The first phase, initial CCSR item development, included a thorough review of empirical counseling skill classification systems (e.g., Larson et al., 1992; Swank et al., 2012) and counseling skill preparation textbooks (e.g., Corey, 2013; Cormier et al., 2009; Hackney & Cormier, 2012; Ivey et al., 2010; Young, 2013) available through multiple key word searches. We then created an exhaustive list of all skills promoted within the literature and retained items that were frequently cited. Finally, we came to consensus on whether a skill was better accounted by a previously generated item.
Upon IRB approval, we began Phase 2 and recruited via e-mail seven counselor educators to engage in the content validity process. Participants were asked to complete an informed consent, demographic questionnaire, categorize content skill names using an 8-point relevance scale (i.e., 0 = not representative at all to 7 = totally representative) for the initial 28-item CCSR, note the degree to which rubric items were representative of domains, and provide edits and general suggestions for creating a more effective rubric. We utilized an 8-point relevance scale to compute the content validity indices to avoid having a neutral or ambivalent midpoint (Polit & Beck, 2006) and to maximize scale reliability (Churchill & Peter, 1984). To decide which items were retained, we reviewed participant feedback, revisited the literature and definition of a particular skill, coming to consensus as a research team. Next, we conducted a second content validity procedure by recruiting via e-mail 17 student participants. Participants were asked to complete an informed consent document, demographic questionnaire, and a protocol indicating six response categories and six separate definitions. Participants ranked the order of response categories (i.e., 1 = poor to 5 = superior, not applicable category) and matched the name of each domain (i.e., counseling skills, phases of a counseling session, and sessions of the counseling relationship) to a description.
To address Research Question 3, we first determined the skills present during a 15-min mock counseling session. This allowed us to discover the number of possible correct skills participants were able to identify. The four Phase 3 participants were recruited via e-mail and provided an informed consent document, demographic questionnaire, a blank CCSR, and a 15-min mock digitally recorded counseling session. We tabulated interrater agreement and discrepancies. Items, in which there was less than 100% agreement, were further assessed by examining rater-scored rubrics, and the time-debated skills were identified in the recording.
All Phase 4 participants were recruited via one of the five methods including Counselor Education and Supervision Network and COUNSGRADS listserv posts, face-to-face solicitation, and e-mailing doctoral program CACREP liaisons, and Approved Clinical Supervisor credential holders. Participants were provided an informed consent document, demographic questionnaire, participant-generated identification form (i.e., form that participants fill out to create a code that protects their identity), the 15-min mock counseling session described in Phase 3, and a blank CCSR. To determine the interrater reliability, we asked participants to review the mock session and use the final 30-item version of the CCSR to assess the skills executed.
Data Analysis
To examine content validity, we conducted a content validity index (CVI) on the initial 28-item scale (Polit & Beck, 2006). Following the calculation of the individual item CVIs, domain mean CVIs, sum agreed on by experts, and proportion of items relevant (i.e., overall expert agreement on item relevance), we calculated the mean item-level CVI and the mean expert proportion. We analyzed expert qualitative responses using descriptive coding and frequency counting (Saldana, 2009). Cronbach’s α was used to assess the internal consistency reliability of the CCSR (Cronbach, 1951). We examined interrater reliability of the items by providing the number of correctly and incorrectly identified items and calculating the percentage of correctly identified skills. To control for chance skill endorsement and to determine interrater agreement, we computed Fleiss’ κ for the final 30 CCSR items (Fleiss, 1971).
Results
Item Development
Table 2 displays the original 28 items of the CCSR and the results of initial content validation. The final version of the CCSR is comprised of 30 items with five domains on a 6-point scale. Domains with corresponding items in parentheses are as follows: invitational skills (nonverbal communication, encouragers, vocal tone, observation, and silence), attending skills (goal setting, open-ended questioning, closed-ended questioning, clarification, paraphrasing, summarizing, normalizing, and reflection of feelings), influencing skills (advocacy, immediacy, challenging and pointing out discrepancies, feedback, reflecting meaning and values, reframing, interpretation, self-disclosure, psychoeducation, homework, and directives), phases of a counseling session (opening phase, working phase, and closing phase), and sessions of the counseling relationship (intake, assessment, and termination).
Ratings on a 28-Item Scale by Seven Experts: Items Rated 5 Through 7 Point on a 8-Point Relevance Scale.
Note. CVI = content validity index; D-CVI = domain-level CVI; I-CVI = item-level CVI; IS = invitational skills; AS = attending skills; PC = phases of counseling; Mean I-CVI = .81; Mean expert proportion = .87.
Content Validation
For Research Question 2, based on the first panel’s feedback, we expanded the initial 28-item CCSR to include the skills of goal setting and advocacy. In addition, the original domain entitled phases of counseling was broken into two separate domains: phases of a counseling session and sessions of the counseling relationship (see Table 2). The initial iteration of the CCSR had a 4-point scale (i.e., 1 = poor, 2 = acceptable, 3 = excellent, 4 = superior, and a not applicable category); however, four of the seven panelists suggested there be an additional negative item. In response, the scale was revised to include an item entitled, substandard.
We computed the CVI for the original CCSR using an 8-point relevance scale (see Table 2). There were seven CCSR skills and phases with 100% rater agreement (CVI = 1.00) and 11 items with CVI values within the acceptable range (i.e., .79 to .99); however, there were 10 individual items with CVIs falling below Lynn’s (1986) recommended minimum of .78. Domain mean CVI scores ranged from .79 to .93. With the exception of silence, all scale items had 5 to 7 raters agree on their relevance to the domain and scale. Each expert had an individual relevance percentage (R = .82 to .96), which provided an overall perception of the scale’s relevance. Further, the overall mean item CVI was .81, and the mean expert proportion was .87, exceeding the .78 threshold (Lynn, 1986).
We conducted a second content validation with 17 professional counseling and school psychology students. Rater agreement on matching domain name and description was accurately identified by all 17 participants, and scale rank and order ranged from 82% to 100% across participants. Specifically, all raters accurately identified the rank and order of the scale items as superior (5), excellent (4), and not applicable (NA). Scale item acceptable (3) was accurately identified within the domain counseling skills by 94% of raters, within the domain phases of a counseling session by 88% of raters, and within the domain sessions within the counseling relationship by 88% of raters. Scale item substandard (2) was accurately identified within the domain counseling skills by 88% of raters, within the domain phases of a counseling session by 82% of raters, and within the domain sessions within the counseling relationship by 82% of raters. Finally, scale item poor (1) was accurately identified within the domains counseling skills, phases of a counseling session, and sessions of the counseling relationship by 94% of raters. We determined that these percentages indicated raters could effectively identify the domain name, description, and differentiate the six Likert scale points within domains.
CCSR Application and Identification of Skills
To address Research Question 3, participants (n = 4) determined what skills were present during a 15-min mock counseling session. The identification of skills present within the mock digital recording allowed us to determine the number of correct skills participants were able to identify within the interrater reliability aspect of the study (i.e., Phase 4). The four independent reviewers identified 23 of the 30 skills (see Table 3).
Participant Skill Ratings.
Note. NV = nonverbal; Challenge/discrepancies = challenging and pointing out discrepancies; reflect mean/values = reflecting meaning and values. aThese skills were not present in the mock counseling session. bOne participant did not rate this skill. The overall mean correct across skills was 50.48 (SD = 13.19), incorrect 12.48 (SD = 13.23), and correct identification was 81.8% (SD = 8.0%).
To determine interrater reliability, we asked participants (N = 63) to review the mock counseling session and use the CCSR to assess the skills being executed (see Table 3). Part A skills with a 100% rater agreement across the sample included nonverbal communication, encouragers, and paraphrasing, and skills with the lowest agreement included homework (41.3%), goal setting (44.4%), and observation (47.6%). We conducted Fleiss’ κ to control for chance skill endorsement and interrater agreement (i.e., percentage of correct identification) across the 63 raters of the 30 CCSR items. Findings indicated that there was substantial agreement among raters overall (κ = .70; Fleiss, 1971).
We calculated Cronbach’s α to provide points of reference for clinical work and item response consistency (Cronbach, 1951). We calculated Cronbach’s α coefficients for items representing invitational, attending, and influencing skills. The coefficient for all skills (invitational, attending, and influencing skills) was .84. Individual domain areas were also acceptable (i.e., invitational skills: .78, attending skills: .79, influencing skills: .73, and phases of a counseling session: .74). The coefficient for the domain sessions of the counseling relationship was not included due to it not being present in the digital recording.
CCSR Scoring and Purpose
Research Question 4 referred to those who were administered the counseling session and initially validated the CCSR (N = 63). In general, participants reported the following scores for the five CCSR domains: invitational skills (M = 3.50, SD = .66), attending skills (M = 3.72, SD = .67), influencing skills (M = 3.43, SD = .75), counselor awareness of counseling phases within a session (M = 2.96, SD = .83), and counselor attention to sessions of the counseling relationship (M = 2.57, SD = .78). Domain scores indicate that on average, participants reported ratings as moderate, positively skewed (R = −.02. to .28), and leptokurtic (R = −.28 to −.61).
The CCSR provides trainees with both a quantitative and a qualitative evaluation of their counseling ability. Trainees are provided a numerical (Likert-type scale) rating for each item, a domain-related (e.g., sessions of the counseling relationship) score, and an overall score. It is recommended that trainee evaluation take into account the individual item scores, domain totals, and total quantitative scores. Domain-related scores are gleaned by calculating the group score by summing ratings per section and then dividing by the number of items not designated as NA. The rating associated with a total score is calculated by summing all ratings per domain and dividing by the number of items assessed. The qualitative aspect of the CCSR includes providing trainees with a written supportive example regarding a particular item and, if applicable, a recommendation for how the trainee can improve his or her score (i.e., ability) on a particular item. In addition, there is space at the bottom of the CCSR for providing lengthier written feedback. This evaluation is recommended for both formative and summative feedback.
The skills assessed within the CCSR are categorized into five foundational areas of counseling: invitational skills, attending skills, and influencing skills (Part A), phases of a counseling session (Part B), and sessions within the counseling relationship (Part C). We define counseling skills as the acquired verbal and nonverbal skillset that enhance the therapeutic alliance by helping a counselor establish a reflective relationship and/or influence facilitative change. For Part A, invitational skills refer to nonverbal skills and opening skills (e.g., nonverbal communication, encouragers, and vocal tone). These skills allow the counselor to communicate to clients that he or she is listening to them and that the client is invited to participate in a session and/or relationship (Young, 2013). According to Ivey, Ivey, and Zalaquett (2010), attending skills refer to the focused attention that is placed on another individual in an exchange between the individuals. Attending skills include listening carefully and initiating verbal and nonverbal communication to create a shared sense of empathy and understanding (e.g., open-ended questioning, paraphrasing, and reflection of feelings). Influencing skills refer to strategies designed to enact interpersonal influence on another individual. These skills allow counselors to guide clients by influencing their stories, thoughts, and feelings (e.g., immediacy, feedback, and interpretation). Invitational, influencing, and attending skills are conveyed through a multicultural, contextual, and developmentally appropriate lens (Ivey et al., 2010).
Part B, which evaluates phases of a counseling session, includes items related to aspects of each session (e.g., opening, working, and closing). The opening phase of the counseling session occurs at the beginning of the session and is comprised of the following behaviors: offering a greeting, providing a summary of last session (if applicable), and applying theoretically consistent transitions into the working part of session. The working phase of the counseling session follows the opening phase and is comprised of exploring the client story, developing mutual understanding of the concern(s), keeping the client focused on goals of treatment, demonstrating clear evidence of working with interventions, evaluating progress relative to goals, and providing resources and referrals, if necessary. The closing phase of the counseling session is comprised of a timely ending of a session, summarizes work completed during the session, addresses unexpected end of session client behaviors, and, if needed, plans for future sessions. Finally, Part C involves items related to sessions of the counseling relationship. This includes macro aspects of the entire counseling relationship (e.g., intake, assessment, and termination of the counseling relationship; Ivey et al., 2010; Young, 2013).
Discussion
The CCSR is a psychometrically sound transtheoretical instrument designed to evaluate counseling trainees’ ability to execute the counseling experience. We engaged in a 4-phase research process. After initial CCSR item development, we recruited two panels to determine the content validity and to further refine the design of the CCSR. Panel 1 participants reviewed the initial iteration of the CCSR, provided a numerical indication of the strength of the item fit within the domain description, and provided qualitative information on the rubric design (i.e., scale, format, scoring, differentiation of skills, and item definitions). Panel 2 participants matched the order of the response categories (determined in Panel 1) with the name of each domain. During Phase 3, four counselor educators utilized the CCSR to review and score a digitally recorded counseling session. Ratings from consensus were then used in the final phases to determine both the interrater reliability (i.e., skills being utilized during the session) and the quality of skill execution within the digitally recorded counseling session. Results indicate support for using the CCSR during the training of professional counselors.
Implications for Counselor Education and Supervision
The present investigation into the CCSR reveals several implications for counselor educators and supervisors. The CCSR is a developmentally oriented evaluation designed to be administered in a formative and summative manner throughout a trainee’s professional preparation and postgraduate practice. Counselor education programs, supervisors, and graduate students can benefit from a comprehensive evaluation tool aimed at measuring trainees’ counseling ability throughout their graduate experience. Further, Motley, Reese, and Campos (2014) described relevant, timely, and continual feedback as essential to assist trainees in gauging areas of competence and those who need improvement. Thus, the CCSR can serve as a tool for continual reflection on counseling strengths and areas for improvement.
According to Ericksen and McAuliffe (2003), counselor educators must be able to measure their ability to produce effectively trained counselors. Further, to gain support from the public, counselor educators must demonstrate their program’s effectiveness in producing well-trained professionals (Schoener, 1999). Training programs can use the CCSR as a comprehensive assessment tool aimed at evaluating trainees’ ability to execute counseling skills, phases of a counseling session, and sessions within a counseling relationship. Although measuring student outcomes is an essential function for counselor education programs accredited by CACREP (Motley et al., 2014), we were surprised at the dearth of standards for evaluating a trainee’s ability to effectively execute a counseling session (e.g., opening, working, and closing), execute basic counseling skills (e.g., open-ended questions and reflection of feeling), and perform basic counseling sessions (e.g., intake, assessment, and termination). This was particularly troubling, given the evidence indicating that problems (e.g., lowering of self-esteem and worsening of performance) arise when supervision is not delivered in an effective, developmental, and consistent manner (Daniels & Larson, 2001; Green, 2011).
Training programs can use the CCSR to enhance their faculty’s ability to deliver effective supervision. One way in which faculty can gauge the effectiveness of their supervision is by tracking the degree of growth of their trainees over time. In addition, faculty can observe the quantity of abilities the trainees are able to effectively execute at certain points in their professional preparation. The CCSR can also serve as an evaluation of macro-training deficiencies. For example, if trainees are not learning how to reflect feelings and meaning within their professional preparation, this deficiency is likely to show up on many of their internship CCSRs. Finally, core counselor education faculty can monitor the effectiveness of off-site supervision by analyzing the formative and summative CCSRs. For example, if a core faculty member is using the CCSR to analyze off-site internship recordings of multiple school counselors and does not find examples of goal setting, this may be a shortcoming of the off-site supervision.
The CCSR can be used to obtain tangible gatekeeping information. According to Ziomek-Daigle and Christensen (2010), counselor educators and supervisors have an obligation to serve a gatekeeping function as part of their duty to protect the profession and public from trainees who engage in behaviors that could threaten the welfare of those receiving counseling services. Thus, there is a need to enhance professional counseling gatekeeping with performance appraisal (Bhat, 2005). Given that the CCSR thoroughly evaluates a trainee’s ability to apply counseling to a client, low scores can serve as an indication that the trainee may be unfit for the counseling profession. The initial evaluation could take place as early as the admissions process (Swank & Smith-Adcock, 2013). For example, all potential program admission applicants could be asked to provide a brief mock counseling session. The session would be scored using a modified version of the CCSR, and the results would provide tangible evidence supporting a decision to decline admission.
We have six recommendations for future research. First, future research could produce rubrics that provide further depth into other aspects of the counseling relationship. While the development of effective counseling skills is an essential aspect of counseling, there are a variety of areas that should have outcomes. For example, a rubric could be developed to measure a trainee’s ability to apply appropriate diagnosis to clientele. Second, the present study should be replicated with an increased sample size. The sample should be comprised of participants from diverse geographical locations at different phases of their training (e.g., skills course, internship, and fieldwork). Third, future investigations could analyze the construct validity of the CCSR. Fourth, a future investigation could explore how counselor education programs use the rubric for the purposes of program evaluation and addressing program deficiencies. Fifth, future investigations could examine the most common skills assessed within particular counseling specialty areas (e.g., clinical mental health, school, college, marriage, and family counseling). Finally, the overall CVI mean, domain-related CVI means, individual expert proportion relevant scores, and mean expert proportion score all exceeded the recommended minimum indices; however, there were 10 individual CVI items below the suggested CVI minimum of .78. While none of the CVI items were endorsed as being more relevant to an alternative domain, experts rated them lower than other individual items. Future research could qualitatively investigate how a sample of experts would identify the lower rated skills and their relevance.
Limitations
The study was the initial investigation into the psychometric properties of the CCSR. Five limitations should be considered when examining the findings of this research. First, the CCSR is designed to assess trainee counseling ability, and how counseling skills are used within various phases of a counseling session and sessions within the counseling relationship. The CCSR is not designed to measure other closely related phenomena (e.g., multicultural competency and ethical decision making), and thus supervisors should consider augmenting the CCSR with other instruments if necessary. Second, participants within this study were not chosen at random. Future investigations could employ random sampling for a more valid examination of the CCSR’s psychometric properties. Third, the lack of ethnic diversity in the sample is a potential threat to external validity. Future studies could test the CCSR with a more ethnically diverse sample. Fourth, the small sample size constitutes a potential threat to our internal validity. Finally, this is the only known examination of a comprehensive counseling skill instrument, and thus further development of this measure is needed.
Conclusion
The psychometric properties of the CCSR were examined to determine the instrument structure and format, content validity evidence, effectiveness in skill identification, and consistency across participants in ratings of skill levels. The results indicate that there was substantial agreement among raters on all levels of analysis and that the CCSR is a psychometrically sound transtheoretical instrument. The CCSR can be used for a variety of purposes, including conducting formative assessment of trainees between sessions, comparing trainee skill level with a diversity of clientele demonstrating different issues, measuring trainee change in skill level at various points in their program, evaluating supervision interventions, conducting trainee self-assessment, and identifying programmatic training deficiencies. Further, the CCSR offers faculty and programs a comprehensive tool for assessing student learning outcomes, regarding their basic invitational, attending, and influencing skills within the various phases of a counseling session and the counseling relationship.
Footnotes
Acknowledgment
Cynthia Waltman, Emily Russell, and Lauryn Barton.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.