Abstract
School counselors increasingly are being asked to develop and implement data-based interventions for academic and social/emotional problems. The purpose of this article is to provide a demonstration of goal attainment scaling (GAS) as an intervention outcome assessment tool for school counselor’s work with students. The strengths and limitations of the GAS method are described and an application is provided on the use of the method with school counseling consultation cases. We conclude that GAS is a useful tool for school counselors to evaluate individual or group intervention outcomes.
As school counselors increasingly adopt the standards of practice that are promoted and defined in the American School Counseling Association’s (ASCA, 2012) national model of school counseling, the field has moved toward a practice that is data based (Dimmitt, 2009). This is especially evident in practices used to select interventions for students that are based on best practice, evidence-based research (Hoagwood et al., 2007), and in the regular monitoring of intervention progress and outcome data (ASCA, 2012). In addition to the emphasis on data-based decision making in the national model, school counselors are increasingly employed at schools that are implementing the response to intervention (RTI) approach to student support (Gruman & Hoelzen, 2011). This sequential three-tiered approach to providing academic and behavioral support for students relies on data-based progress monitoring at each tier to determine a student’s improvement and the need for further intervention. To accomplish these changes, it may be necessary for school counselors to overcome a tendency to perceive assessment as less important than other aspects of the profession (Hatch & Chen-Hayes, 2008).
Data collection is now considered a foundation for effective school counseling because it allows school counselors to identify areas of need and evaluate the effectiveness of interventions (Poynton & Carey, 2006). The current framework for data-based decision making and accountability typically is limited to data available in school information systems (ASCA, 2012). These databases often include information such as graduation rates, attendance rates, and discipline referrals that are helpful in monitoring student progress. They are not, however, designed to reflect the outcome data of direct and indirect interventions provided by school counselors such as individual counseling and consultation. School counselors need intervention outcome assessment tools that are sensitive to change and that are reliable, valid, and easy to use in a school setting.
In this article, we describe our experience with monitoring intervention outcomes in our graduate training program in professional school counseling. When developing the program, we wanted to employ a tool for students to use in a school consulting class to monitor the progress of our students’ work with students at their internship sites. The method we chose had to be both useful at the formative and summative assessment levels in determining progress and overall outcomes on our students cases in their internships. Because our student interns worked in a collaborative manner with a schoolteacher who implemented the intervention, they also needed a method that could be taught easily to the school teacher, took little time to complete, and was accurate in determining the level of progress or lack of progress on their cases.
We chose one such method—goal attainment scaling (GAS; Kiresuk & Sherman, 1968). The GAS method has been used for many years in the school psychology field and in clinical and community psychology settings to monitor client progress (Coffee & Ray-Subramanian, 2009; Kiresuk, Smith, & Cardillo, 1994; Roach & Elliott, 2005), but apparently it is not as well known in the school counseling field (Yarbrough & Thompson, 2002). We believe the GAS method is well suited for school counselor’s intervention progress assessment in that it is an evaluative tool that meets the criteria of being easy to implement, easy to teach, and the method can be reliably and validly used to measure intervention progress and outcomes.
Sites and Participants
Chapman University is a small, private university located in Orange County, CA. The student population is approximately 7,000. Our Professional School Counseling graduate program typically accepts between 10 and 20 students each year. It is a 2-year master’s program, with the consulting class occurring during the second semester of the second year while students are in their internship. Our practicum and internship sites are varied, ranging from urban, low socioeconomic status areas to affluent, suburban schools. For the available case data, 15 school counseling interns provided services to a total of 20 students in middle and high school.
GAS
GAS is a criterion-referenced rating scale approach that can be readily applied in school settings for monitoring both academic and behavioral counseling outcomes. Kiresuk and Sherman (1968; see also Kiresuk et al., 1994) developed the GAS method to evaluate the effectiveness of mental health services. When applied in school counseling settings, multiple sources (e.g., teachers, aides, parents, school counselors, or students themselves) can complete GAS ratings on academic or social behavior intervention outcomes at individual or group levels (Coffee & Ray-Subramanian, 2009).
The GAS method involves the following steps: (a) selecting a specific behavior(s) for intervention, (b) defining the intervention outcomes in objective terms, and (c) ranking goal attainment from negative to positive outcomes (Elliott & Busse, 2004; Roach & Elliott, 2005). The outcomes are operationally defined in successive levels of progress on a 5-point scale (i.e., −2 to +2, wherein −2 indicates that a problem is much worse; −1 indicates a problem is somewhat worse; 0 is the baseline or current level of performance; +1 indicates some improvement; and +2 indicates an intervention goal is attained).
Depending on the behavior, raters can provide hourly, daily, or weekly reports of student progress that are derived from direct indicators of progress (e.g., homework completion, and classroom test scores) and/or from the raters’ perceptions of progress (see Figure 1 for an example of a GAS worksheet). For example, if math homework completion is the target behavior, the school counselor may choose to gather data on a weekly basis, so there are enough data compiled to monitor progress. Alternatively, if the target behavior is more sensitive to daily data gathering, such as on-task behavior, then daily GAS ratings may be more useful than weekly ratings to monitor ongoing progress.
Goal attainment scaling summary worksheet.
Counseling goals may be best determined in a team format, wherein the teacher and school counselor are involved in the process and, when appropriate, parents and student. Goal definitions are specific to the child, such that the initial rating of zero is based on baseline data (current level of performance), and the subsequent goal ratings of negative and positive 1 or 2 are based on an individual student’s needs. Once the baseline is determined, the team or the school counselor uses their professional judgment to set goals that are appropriate for a given student. For many behaviors, the goals are best defined as ranges. For example, the student depicted in Figure 1 has a baseline of 45–55% work completion in math. A GAS rating of 1 was set at 56–89% to indicate progress toward the goal, and a GAS rating of 2 was set at 90% or greater to indicate goal attainment. The negative ratings mirror these goal ratings to indicate that the percentage of work completion had worsened, with a rating of −1 for 21–44% work completion and a rating of −2 for
The GAS method possesses several strengths that make it an ideal tool for school counselors. The method: (a) is time efficient; (b) can be used with individuals or groups; (c) can be used across settings and sources; (d) can be used as a self-monitoring tool; (e) can be used to repeatedly monitor progress; (f) is easy to graph and interpret; (g) provides a method for evaluating progress toward outcome goals; and (h) is easy to use and readily understood by consumers. The limitations of the GAS method are that ratings for certain behaviors necessarily are based on summary observations and perceptions, such as goals for behaviors that may not lend themselves to accurate appraisal (e.g., teacher ratings of class participation), and subjectivity is involved in the determination of the level of goal attainment (Elliott & Busse, 2004).
The GAS method has been shown to possess solid psychometric properties. Cardillo and Smith (1994) used the method in clinical settings and found interrater reliabilities ranging from .87 to .93, and a test–retest reliability of .84. Coffee and Ray-Subramanian (2009) and Roach and Elliott (2005) summarized the psychometric properties and found that GAS is a reliable and valid method for evaluating intervention outcomes in school settings. We have used the GAS method for measuring academic and behavioral RTI outcomes with school psychology interns in a study similar to the one we describe here (see Brady, Busse, Carriere, Hass, & Kennedy, 2013).
Academic Intervention Outcomes for Consulting
The intervention outcome data were gathered by our students during their internship as a class assignment in the school counseling consultation class. The academic target behaviors included on-task behavior, work completion, and assignment notebook completion. The interventions included homework completion contracts, weekly counseling, classroom contingency management, and other reinforcement contingencies.
Table 1 provides the outcomes from the academic consulting cases the interns completed in 2011 and 2012 for the students they counseled. The GAS ratings were gathered daily or weekly, depending on the case. We used the last four daily or weekly ratings to compute a mean GAS rating outcome for each case. We chose the last four ratings to provide consistency in the outcome computations and to reflect the most recent level of goal attainment. According to guidelines proffered by Busse, Elliott, and Kratochwill (2010), mean GAS ratings
Consulting Academic Intervention Outcomes.
Note. GAS = goal attainment scaling. GAS ratings are on a 5-point scale from +2 [goal attained] to −2 [strong negative effect]. Outcome ratings are the means of the last four ratings for each case.
Strengths and Limitations
One positive outcome of using GAS ratings has been to increase the data-based decision making and accountability component of our program. Thus, we have implemented a more systematic method for evaluating our interns’ impact on the students we serve. A second strength of the GAS method is that its versatility allows for the documentation of school counseling interventions such as individual and small group counseling, crisis interventions, and consultation that may not otherwise be captured through more traditional measures of data collection.
A limitation is that the interventions were directed at academic behaviors such as homework completion—we do not know whether the interventions resulted in skill attainment or whether the students’ grades were affected. Another limitation is that the interns did not assess whether the teachers who implemented the interventions actually conducted the interventions as prescribed, nor whether the GAS ratings were accurately reported by the teachers. These two issues refer to the concepts of treatment integrity and evaluation integrity. If the intervention is not implemented as intended, it is a threat to internal validity, and if the data are not accurately collected, it is a threat to external validity. As such, school counselors should be monitoring the integrity with which interventions are applied and evaluated. To address these limitations, research should be conducted on training school counselors in the use of GAS ratings for a variety of academic and behavioral/emotional problems, including foci on treatment and evaluation integrity.
Discussion and Conclusion
Taken together, the GAS data indicated that the overall consulting outcomes were moderately positive. This finding is not surprising and may be reflective of typical school-based outcomes. In practice, sometimes our interventions have a strong impact on the students we serve, and sometimes the outcomes may be more moderate or nonexistent. Furthermore, the interventions were implemented by teachers rather than the interns. For some students/behaviors, more intensive, direct intervention probably is necessary to affect change. Finally, the consultations were conducted by novice counseling interns; the intervention outcomes may have been stronger if the consultations were conducted by more experienced school counselors.
As the field of school counseling moves toward a greater focus on data-based decision making, it is incumbent upon practitioners to expand their roles and skills to include tools to systematically evaluate intervention outcomes and to incorporate treatment and evaluation integrity monitoring into their interventions. It is also incumbent upon school counseling programs to train students in the use of these methods and for students to apply them in practicum and internship settings. School counseling trainers also should provide continuing education for practitioners through venues such as conferences, online web-based training, and extension classes.
Although we focused on academic intervention outcomes in this article, GAS can be readily applied to behavioral/emotional interventions and can be applied at the individual or group level. Whether the interventions are delivered through consulting or are directly delivered, GAS is a useful method for evaluating school counseling intervention outcomes.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.