The Assessment Skills and Knowledge Standards

Crafting the Standards

IDENTIFYING the areas of need for professional development in assessment was the initial reason for the work that became the ASK Standards. How can professional educators articulate student learning outcomes, assess those outcomes, and use the results to further inform attention to student learning if they lack the knowledge and skills with which to do so? How can students' actions, interactions, and commitments indicate the knowledge and skills they have gained, measured against what we intended them to learn? With these challenges in mind, the directorate (board) of the ACPA Commission for Assessment and Evaluation conducted a Web-based survey of the approximately 800 members of the commission to determine professional development topics. Directorate members clustered the topics from the survey results into broad areas for professional development. Many directorate members are full-time assessment professionals at institutions that range from small private liberal arts institutions to large comprehensive public universities. Those doing the clustering had daily contact with professionals on their own campuses who had similar development needs.

Over several months and countless e-mail discussions, the initial clustering went through several iterations. Additional topics were added to the clusters, clusters were rearranged, and additional topics were discussed and either added or dropped. Feedback on the emerging draft was sought from new and seasoned student affairs educators in functional areas such as residence life and career services as well as diversity groups chosen for variety in race and ethnicity, disability, sexual orientation, and gen-der. Along the way, someone wondered, “Why not call it ASK? You know, ‘assessment skills and knowledge.’”

Important Endorsements

THE RESULTING ASK STANDARDS can be applied in academic affairs and in student affairs. The standards have been endorsed by the Higher Learning Commission of the North Central Association, the Western Association of Schools and Colleges, and the Association of American Colleges and Universities, as well as by one of the authors of this article, Peggy Maki, an assessment professional whose publications and presentations are internationally known. Clustered into three subgroups, the thirteen standards identify the skills and knowledge that professionals need to document and improve student learning.

The Standards in Brief

THE FIRST SUBGROUP of ASK standards, Foun-dational Issues, encompasses two standards, Assessment Design (Standard 1) and Articulating Learning and Development Outcomes (Standard 2). The first standard identifies tasks in designing an assessment plan based on conceptual connections from broad areas outlined in an institutional mission and translated to context-specific learning outcomes for students. In academic affairs, the specific context might be a major or, even more specifically, a course. Likewise, in student affairs, the context might be a functional area such as student activities, student health, or another programmatic area. Regardless of the context, the first ASK standard focuses on the ability to translate broad areas of learning into more specific learning outcome statements about what students should achieve through educational opportunities inside and outside of the classroom. Professionals with the skills outlined in this standard are well positioned to respond to public concern that the mission of an institution is an empty promise. As a response to such a challenge, professionals skilled in assessment design can offer specific examples of the courses, programs, and services through which students are encouraged to learn and develop in a manner consistent with the promises of an institution's mission statement.

The second standard focuses even more on the ability to articulate student learning and development goals and outcomes. What do we wish students to demonstrate, represent, or produce through participation in our courses, programs, and services? As educators, when we know what we wish students to achieve in terms of their learning, we are in a better position to design instructional approaches in the curriculum and the cocurriculum that are likely to foster that learning. In short, Standard 2 instructs educators how to articulate learning and development targets as a necessary prerequisite for positioning students to reach them.

The next group of ASK Standards, Tools and Techniques (Standards 3–9), covers data collection and management (Standard 3), assessment instruments and surveys (Standards 4 and 5), interviews and focus groups (Standard 6), analysis (Standard 7), benchmarking (Standard 8), and program review and evaluation (Standard 9). Many practitioners have at least rudimentary knowledge of some of these tools. The ASK Standards identify more of the knowledge and skills needed in each area so that educators can develop their ability to measure or in some way obtain feedback on the manner and degree to which students are reaching intended outcomes. As educators, we must be able to determine whether the courses and activities we design are helping students to reach our intended outcomes. Information gleaned through these tools helps us hone experiences that are effective and redesign those that fall short of the mark.

The third group of skills—Advanced Issues—focuses on the following topics: assessment ethics (Standard 10); effective reporting and use of results (Standard 11); the politics of assessment (Standard 12); and assessment education (Standard 13). We may be able to translate our institution's mission to specific learning outcomes; design courses, programs, and services likely to encourage those outcomes; and measure our success. But if we do not anchor our efforts in a strong sense of ethics and communicate the results clearly and strategically, our work likely will be of diminished use. The knowledge and skills described in the Advanced Issues group increase the likelihood that our work will leverage improved teaching and learning, an outcome of the utmost importance to audiences both inside and outside the academy.

Application of ASK Standards

PRESENTATIONS about the ASK Standards to groups such as institutional researchers, faculty at historically black colleges and universities, and liberal arts faculty from institutions across the country support the assumption that the standards are applicable in both academic affairs and student affairs. When applied, the standards will improve the policy and practice that guide effective inquiry into student learning and the use of results to improve student learning. For example, a faculty member in the audience at one of the ASK presentations observed that the standards could help faculty members identify skills and knowledge that could be used to determine how well students learned the outcomes identified on a course syllabus. Another faculty member, who also had an appointment in student affairs, discussed how he could use the language in the ASK Standards to develop a common language about assessment so that faculty in academic affairs and staff in students affairs could use the same language and begin developing ways that both divisions could evaluate the learning outcomes that students were or were not achieving at his college, both in and out of the class-room. A vice president for student affairs envisioned how she would use the standards to determine the assessment strengths and weaknesses of her staff and then create professional development opportunities to address identified weaknesses. She considered creating professional development opportunities that focused on weak areas of professional skill so that student affairs staff could improve their skills and thereby document the learning and development outcomes from participation in cocur-ricular programs.

The ASK Standards can assist in building what Peggy Maki, in Assessing for Learning, calls “a culture of inquiry.” In such a culture, educators use assessment to “[pursue] questions about teaching and learning” (p. 1). The ASK Standards can be used as a framework for developing that culture. Using this road map to guide what they need to know and be able to do in assessment, faculty and student affairs educators can set goals for areas of competency in which they want to develop greater proficiency; identify professional development options that will help them reach those goals; and measure their progress. These steps are similar to what we ask of our students: understand what we wish you to learn, identify ways that we can assist you in learning, and measure the degree to which you are progressing in meeting those goals. The ASK Standards put faculty, student affairs educators, and students on the same page: a focus on learning.

In addition to developing a culture of inquiry, the ASK Standards can be used to help develop and implement an assessment plan. If someone wanted to demonstrate the learning outcomes that students achieve by participating in a semester-long leadership development workshop, the person would begin by developing an assessment plan (Standard 1). Using the assessment cycle in Peggy Maki's Assessing for Learning as the basis for the assessment plan, the first step would be to develop learning outcomes that are specific to the leadership development workshop and that support the institution's or department's mission and educational objectives. Often, essential staff buy-in is achieved by discussing a draft of those outcomes and honing them through clarifying discussions. Once the outcomes have been finalized, the next step would be to decide on the manner and type of evidence to be gathered in order to determine whether the outcomes have been achieved. The last step is the most important and the most often overlooked: using the evidence to implement any needed programmatic changes to increase student learning of the intended outcomes.

Standard 2 focuses on developing learning outcomes and on the opportunities afforded to students— courses, workshops, programs, and services—that are most likely to help students reach those outcomes. For example, after a review of the literature and conversations with other staff members responsible for leadership development, the staff member formulating the assessment plan could articulate a set of learning and development outcomes that students should be able to demonstrate after completing a semester course. These outcomes might include meeting facilitation, conflict resolution, group dynamics management, or other leadership abilities. Once those outcomes are identified, activities should be developed or reviewed to ensure that they intentionally promote learning in those areas.

The next step in the assessment design is to gather evidence to determine whether the outcomes have been achieved. With that in mind, the staff member would select a data collection method after considering the variety of options available (Standard 3). In determining the most appropriate types of evidence relevant to the intended learning outcomes, the staff member might consider using an existing leadership assessment (Standard 4), surveys of students in the course (Standard 5), interviews or focus groups (Standard 6), and a direct method of assessment, such as videotaping student leaders as they lead organizations or meetings and scoring their abilities against criteria and standards of judgment for leadership.

Once the data collection has been implemented, the results would be analyzed (Standard 7). For example, if an established leadership inventory were used to assess outcomes of the leadership course, univariate analysis could be used to measure how many students learned particular skills such as meeting facilitation. The analysis would ideally become more complex, including multi-variate statistics to determine relationships between variables and possibly predict which activities or individual characteristic would be most likely to be associated with particular leadership learning outcomes. For example, multivariate analysis could be used to determine the correlation between learning a particular outcome such as resolving conflict and students' class year. Or if two different strategies are used to teach conflict resolution— for example, students' observation of conflict resolution versus their engagement in role playing in a conflict resolution scenario—then a comparison of students' conflict resolution strategies might reveal that the role-playing activity is more effective in developing conflict resolution strategies than mere observation.

It also might be useful to benchmark a leadership development class on one campus against a similar course on another campus in the same institutional system or on another campus at a different institution (Standard 8). Perhaps the same curriculum is being used on two or more campuses. Comparing the achievement of outcomes at each of the campuses may provide insight into more effective implementation strategies that could be adopted by all campuses that are using the leadership curriculum.

Educators want to know which activities, courses, programs, and forms of pedagogy are most effective in fostering the qualities, traits, or behaviors we desire. If students are not achieving expectations through current programs, methods, and teaching strategies, then we need to ask ourselves what can be done to help more students reach our intended outcomes (Standard 9). Analysis helps us make good choices and use limited resources carefully and in a targeted fashion.

Throughout the assessment process, an educator must pay particular attention to the ethics of an assessment project (Standard 10) so that the rights or safety of the assessment participants are not compromised. After analyzing the data, the educator would sit down to write a report for an intended audience, keeping in mind the unique needs of that audience (Standard 11). For example, the report might be a brief summary that is shared with the staff directly involved in leadership course implementation; a PowerPoint presentation for the staff of the student center whose members oversee leadership development on campus; a podcast interview intended to boost participation by students; or part of a more comprehensive program evaluation of the leadership development program of which this course is one element. As Standard 11 states, the manner in which report information is conveyed is as important as the information itself. The manner must be appropriate for its audience.

The political implications of assessment pepper the entire process (Standard 12). In a time of budget constraints and a need to demonstrate the learning outcomes of cocurricular programs, leadership development may be a set of skills and knowledge that many stakeholders in and outside of the classroom value and can easily comprehend. Hence, different reports may be tailored for multiple audiences. One report for parents may not use any statistics, while another report for faculty in the sciences may include fine-grain statistical details. A report for the legislature may be written to tout the benefits to the state economy of leadership outcomes such as multicultural competence, while another report for student affairs administrators may focus on psychological benefits such as self-confidence.

Impact of the ASK Standards

THE ASK STANDARDS are poised to affect professional development at national, institutional, and individual levels. At the national level, the ASK Standards provide a framework for professional organizations in both student affairs and academic affairs that wish to develop educational opportunities such as webinars and e-learning courses centered on the individual standards. The ASK Standards could also be used as guidelines in developing pre-conference workshops, an assessment institute within a national convention, or an assessment discussion group or in selecting conference and convention programs. Finally, the ASK Standards could be used to develop prepackaged half-day or daylong assessment workshops for state or regional drive-in work-shops. ACPA has used the ASK standards in all of these ways. So, too, could others. While assessment education can be delivered in a variety of ways, the ASK Standards provide a consistent framework that outlines the skills and knowledge that should be advanced in educational sessions, regardless of delivery method.

At institutions, staff groups or departments can refer to the ASK Standards to help them identify areas of strength as well as areas that need development among their members. The ASK Standards Needs Assessment is one tool that can help with this type of skills inventory. Does someone in your department have special skills and knowledge in program review and evaluation? Might that person assist other faculty or staff members who must tackle an academic program or student affairs unit review conducted, perhaps, as part of a regular five-year rotational review? If so, ASK Standard 9 provides guidance on how to identify the types of skills needed for program review and evaluation. Do staff need additional assistance, perhaps from outside your institution, in understanding assessment instruments? If so, ASK Standard 4 lists options for consideration. Might a professional development webinar or assessment workshop offered locally to your staff help develop greater knowledge of and skills in assessment practices and principles? Such a group learning experience could help develop a common language of assessment among members of the department or staff, a common language that can under-gird a common focus on student learning.

An individual staff or faculty member could use the ASK Standards Needs Assessment to help determine where her strengths and weaknesses lie and then seek educational opportunities to address identified weak-nesses. These opportunities might include reading books that support one of the standards, attending a regional or national assessment conference based on the standards, or attending sessions at a national convention that addresses a particular standard or group of standards. Using the standards in this way helps an individual staff or faculty member first prioritize standards to focus on and then be intentional in the search for educational opportunities.

The ASK Standards can provide a valuable framework for building assessment into the professional preparation of current and future faculty and staff. The standards can help faculty infuse assessment into an already packed curriculum. A focus on assessment is a focus on student learning and the manner and degree to which students are able to acquire, use, and integrate the learning we intend. So a focus on assessment helps all of us to determine just how well students are achieving our expectations, through examining a range of their work or performances. Given the importance of experiential education such as assistantships, internships, and practicums in the professional preparation of faculty and student affairs staff, educators can use the ASK Standards to determine the extent to which assessment is included in those experiences or to identify ways to assess student learning and development associated with these kinds of educational opportunities. Courses on student development might include a module on how to assess the type of development discussed. Faculty might also draw on the standards in working with administrators to provide workshops for their colleagues or graduate students who will eventually be responsible for assessment in their professional teaching careers. The ASK Standards could help graduate students determine how to assess the extent to which they affect individual student learning.

Conclusion

RESEARCHERS have successfully made the case that learning is a chronological process of making meaning. Those whose work has contributed to this understanding include Marcia Baxter Magolda and Patricia King in Learning Partnerships: Theory and Models of Practice to Educate for Self-Authorship, as well as the collaborations among national organizations that have led to the widely accepted principles of learning and development represented in, for example, Learning Reconsidered 1 and Learning Reconsidered 2. Multiple and varied kinds of educational opportunities and experiences inside and outside of the classroom represent the context within which students learn and develop. Through the process of assessment presented in the ASK Standards, student affairs educators and faculty now have the systematic means to demonstrate the efficacy of their practices in fostering students' holistic process of learning and development.

The ASK Standards represent a landmark development in ACPA's history that recognizes the critical importance of assessment in the roles and responsibilities of that organization's chief constituency, student affairs professionals, as well as the principles and practices that guide professional assessment of student learning and development. Specifically, the standards enable student affairs educators to

In addition, faculty and other educators can draw from the ASK Standards to professionalize their work in assessment, because the standards represent sound professional principles of and practices in conducting robust assessment that leads to actionable results. Further, faculty and student affairs professionals can choose to work in tandem with each other, following the ASK Standards, to collaboratively assess one or more shared outcomes, which will provide two lenses through which to assess students' levels of achievement—one from the curricu-lar side and another from the cocurricular side.

Since 2002, ACPA has conducted six student affairs assessment conferences designed to help education professionals learn about and engage in assessment of student learning. Those six years contributed to the organization's ability to draw together assessment research, writings, and experiences that led to professionalizing, systematizing, and relating theory to practice in the new ASK Standards. The ASK Standards describe and prescribe how student affairs professionals will now take responsibility for assessment, provide evidence of learning to the public, and continually use results to improve educational practices in order to promote student learning.