Enhancing Teaching With Effective Data Mining Protocols

Abstract

Various data systems have been long and pervasively used in schools to collect student data. However, very few educators are able to apply their collected data to improve their teaching. The purpose of this article is to investigate how middle school teachers adapt data mining protocols to enhance their teaching and to improve their students’ learning. Results of the research indicates that most of the middle school teachers use data systems to enter their students’ data under the school requirements and only 20% of the teachers actually retrieve collected data from the system and analyze the data for intervention purpose.

Keywords

data mining data-driven decision-making teaching learning

Introduction

Data-driven decision-making (DDDM), a systematic process to gather, store, analyze, and apply data to make decisions for increasing performance, has been proved to be successful in business, yet not in education. DDDM, as it applies to education, still reminds to be a challenge due to the confusing to read and not easily accessible data for teachers to use in their daily instruction. Despite all the challenges, educators believe that data have the capacity to improve student learning. Thus, it is essential to know how to effectively dig the data deeper to enhance teacher’s teaching.

Data systems are prerequisites for DDDM (Means, Padilla, C, & Gallagher, & SRI International, 2010). In the United States, there are hundreds of educational data systems that include Aeries, Destiny, Orasi Advanced Reporting Solution (OARS), Cruncher, and so forth. The data collected from these systems range from grades, attendance, to state mandated testing scores. The quantitative data have very limited instructional use because these numerical data only tell how well student did on a test, rather than what and why students did well on.

The purpose of this article is to investigate effective data mining protocols to enhance teacher’s teaching. The article will benefit policymakers who need to assess and select effective data systems to use for their schools, districts, or educational systems. The article will also help school administrators in implementing student information systems in their schools. Most importantly, teachers will receive benefits of enhancing their teaching with effective DDDM strategies. To effectively apply DDDM, a review of literature is conducted first. Current popular educational data systems used by school districts are investigated. Effective DDDM protocols are introduced. This article will synthesize what 10 secondary math and science teachers find as useful data, what they use with these data, and how it affects student’s academic and behavioral improvement. Finally, recommendations and implications are provided.

Literature Review

The literature review will provide an overview of the types of educational data systems and current popular educational data systems used by the school districts in the United States first. Then, the advantages, disadvantages, and issues of DDDM mentioned in the literature will be addressed and synthesized.

Types of Electronic Student Data Systems

Currently, there are hundreds and thousands of educational data systems available with different data collection capability in the market. Wayman (2005) classified that there are four main types of electronic student data systems.

Student information systems

This type of data systems collects student data such as attendance, demographics, test scores, grades and schedules. They provide real-time access to the information. Examples include Aeries,

Data warehouses

This type of data system collects and stores data. Users of such system can access to current and historical data on students, personnel, finances and so on.

Instructional or curriculum management systems

Users of such system can access to instructional and curriculum resources, such as planning tools, model lesson plans, creation of benchmark assessments, linkage to state content standards, communication and collaboration tools. One of the examples for the collaboration tools is threaded discussion boards. The system provides a unifying framework for access.

Assessment systems

This type of data system focuses on collecting and analyzing assessment scores, such as state assessment. It supports fast organization and analysis of benchmark assessment data. Examples include Cruncher, OARS, and so forth.

Means et al. (2010), in their national survey of 427 school districts in spring 2007 and site visits during school years 2006–2008, found that almost all school districts have an electronic data system to provide real-time access to information such as student enrollment and attendance. In their study, they found that only 10 out of 427 school districts in the survey sample do not have any electronic data system. Seventy percent of school districts have had electronic student information system for six or more years. Districts continue building their data system technology capacity by acquiring other types of electronic data systems. Specifically speaking, 79% of districts report having an assessment system that organizes and analyzes state assessment data. Seventy percent of districts report having a data warehouse that allows their users to access to current and historical data on students. Sixty-four percent of districts report that they have an instructional or curriculum management system to support teacher’s access to curriculum and instructional resources.

Popular Educational Data Systems in U.S. School Districts

With the advancement in technology, mountains of data can be collected at a fingertip in education. There are hundreds and thousands of electronic data systems available in the market. The data system itself ranges from collecting student’s performance grades, to number and type of detentions, suspensions, state mandated testing scores, and so on. It is also found that the data systems collect both quantitative and qualitative data (Hershkovitz, 2015; Kaufman, Graham, Picciano, Popham, & Wiley, 2014).

Means et al. (2010) found that U.S. school districts prefer electronic data systems with the ability to collect student demographics and test scores to the data systems with the ability to link instructional resources to student achievement data. In addition, school districts also do not like to use data systems which can combine data from different types of systems. In their study, they found that over 90% of the U.S. school districts have and use electronically stored data system to collect data, such as student demographics, attendance, student academic grades, student’s statewide assessment scores, and student course enrollment histories.

Means et al. (2010) further reported that only 42% of districts use educational data systems that offer links to student’s learning process and outcomes. To enhance student’s learning, teachers must have access to the data regarding student’s learning process and outcomes. In addition, less than 50% of the school districts use educational data system that generate data reports showing student performance linked to participation in specific instructional programs and to teacher characteristics.

Advantages of DDDM for Teaching

The main goal and advantage of DDDM is to improve student learning. Literature has validated this advantage of DDDM (Kerns, 2013; Means et al., 2010). Means et al. (2010), in their national study, found that school districts involved in implementing a student performance assessment system show an increase in teacher’s use of data from year to year than other school districts which do not have an assessment system. Other advantages of DDDM include assessing student level of proficiency in certain subjects and skills, preventing potential risks of a student with attendance tracking, chronicle demographic change in a school site, and so on.

Issues of DDDM in Education

Although hundreds of data have been collected in schools and school districts, there are lots of issues involving DDDM. A review of literature suggested that the major barriers for teachers to use collected data to improve instruction is time constraint in analyzing the data (Hershkovitz, 2015; Means et al., 2010). Teachers often reported that they do not have time to analyze data, especially qualitative data, such as video logs. Teacher’s work time already fully loaded with lesson planning, professional development, giving students appropriate feedback, communicating with colleagues and their students’ families, committee work at their site, and a variety of other things. Therefore, when an additional tool is overly time consuming, then they are less likely to use that tool even when it can provide so much benefit.

Another barrier in applying DDDM is the time delay in receiving data. Teachers often do not receive student’s assessment data soon after the assessment. Some data even take several months or years to receive or access. For example, students’ state standardized test results usually are received or released after three months or more. The worst thing is that when teachers receive their students’ test results, the students have moved to another grade level and no longer in the teachers’ classes. In such a situation, the collected data are not so helpful for teachers to reflect and improve their teaching. Teachers cannot reteach the contents and students cannot relearn to master the contents, either.

Another issue which prevents teacher’s effective use of the data is regarding the collected data. Since before the No Child Left Behind era, quantitative data have been most prevalent in the state or federal levels of the education system. But because these data only tell how well student did on a test, rather than what students did well on, or why students did well on it, these data have very limited instructional use (Kaufman et al., 2014). In addition, any data systems that districts and schools have are often underutilized by teachers because they find it difficult to access reports that are straightforward to interpret and facilitate quick decisions (Kaufman et al., 2014). These factors result in very limited, and very shallow, data use for instructional purposes.

Effective Strategies of Data Mining in Education

At the heart of the purpose for DDDM are instructional changes to improve student’s performance. Specifically speaking, it includes certain results for students, such as improving test scores, assessing skills and knowledge, documenting the level of proficiency of academic tasks, and tracking behavior trends. In addition, at the heart of the instructional changes themselves is the teacher’s belief about and capacity for data use. Teachers must understand the purposes and uses of the range of available assessment options and must be skilled in translating them into improved instructional strategies (Datnow & Hubbard, 2015). Yet, in the real-world educational setting, both teachers and students are often at a loss of what to do when their data becomes accessible. Teachers and schools cannot make significant improvements when the details of their underperformance are unknown, and when possible remedies for underperformance are not known as well (Staman, Visscher, & Luyten, 2014). Therefore, it is essential to learn effective data mining strategies for all educators including principals, staff, and teachers. In this session, three effective data mining strategies will be presented and discussed. They are capacity-building interventions, Datnow and Hubbard’s 11-step protocol, and data gallery walk protocol.

Capacity-building Interventions

Capacity-building interventions (CBI) is a strategy to help teachers build their capacity in interpreting and using data to enhance their teaching. Marsh and Farrell (2015) emphasized that like dividing learning communities into small groups, the use of CBI assesses teacher needs as well as student needs, provides feedback and shares expertise, and dialogue and questioning. The strategy assists teachers to overcome their negative beliefs of either the usefulness of data itself, or of their personal belief in their own capacity through training, time with colleagues to collaborate, and a nurturing school culture. CBI helps teachers to get ready to implement practical methods for data analysis, planning, reevaluation, and replanning.

To implement CBI in a school, it requires school or school district leaders to facilitate and to encourage an open sharing culture. Specifically, it is to create a learning community to support teachers and staff to analyze data, plan instructional strategies, and implement in teaching. Marsh and Farrell (2015) indicated that

building capacity in teachers to utilize DDDM-based instruction increases autonomy, creative problem-solving, and boosts overall morale and inclusiveness. The school becomes a community with shared interest who, through their regular joint work, improve upon the art and science of teaching.

Brown (2016) also emphasized the important role of principals in implementing CBI. He argued that the principal’s job is to “involve establishing a common vision with their staff, nurturing an environment that everyone can support and believe in, and respect their staff’s expertise and rely on them to be strong players” (p. 105).

Once teachers overcome their negative beliefs of either the usefulness of data itself, or of their personal belief in their own capacity through training, time with colleagues to collaborate, and a nurturing school culture, they may be ready to implement practical methods for data analysis, planning, reevaluation, and replanning.

Datnow and Hubbard’s 11-Step Protocol

Datnow and Hubbards (2015) outlines a specific 11-step protocol for teachers to use that involves obtaining performance feedback the student monitoring system can provide, diagnosing the cause for underperformance on the learning progression, having conversations with individual students, drawing up plans that specify how each student currently performs, how each student will hopefully perform at the next assessment, which approach will be followed for each student, implementing these plans, and then evaluating whether the approach taken has worked. If the plan is successful, then new goals and plans are developed. If not successful, then the plan is adapted to increase the probability of success. Figure 1 shows the protocol taken from Datnow and Hubbards (2015).

Figure 1.

DDDM protocol from Datnow and Hubbards (2015).

The process is not simple. Many stakeholders must be involved, including the student themselves, and thus, the task seems daunting. It is no wonder that teachers dread staff meetings where they are assigned to pour over document after document of data.

Data gallery walk protocol

Another practical and less daunting protocol is for staff to complete a data gallery walk which is proposed by Kennedy, Mimmack, and Flannery (2012). In the gallery walk, staff is assigned into small groups, each group is given multiyear data on a particular topic, such as school suspension rates or a particular test score. Then each group makes a summary poster for that topic (Kennedy et al., 2012). The poster includes three main strands of thoughts: (a) Here’s What—a summary of the data and changes in that data over time, (b) So What?—An explanation of why these data matter, and (c) Now What?—A description of immediate next steps to address the problem. Kennedy et al. (2012) says that this process can be done in about one and half hour session, including the final walk through of all the posters, as if in an art gallery viewing pieces of art on exhibition.

Theoretical Framework

The theoretical framework that used to support and guide the study is technology, pedagogy, and content knowledge (TPACK; Koehler & Mishra, 2009). Although TPACK focuses on the integration of instructional technologies used for daily teaching, the theory can be applied to the utilization of the technology, student information system, in schools. Data mining is a complicated process that requires an interweaving of various specialized knowledge, especially with an aim to improve teaching. It requires teachers to apply knowledge of the student information system. Thus, an effective data mining approach depends on sufficient access to the system, to retrieve the data, and interpret the data for pedagogical reflection and improvement.

Significance of the Study

Although data systems have been long and pervasively used in school districts and educational institutions to collect student information such as student enrollment, attendance, grades, and so on, not many teachers can apply the collected data to improve their teaching and student’s learning. Furthermore, few studies have focused on investigating how teachers perceive the collected data and how they can use the data to enhance their teaching and improve student’s learning. Results of this study will provide educators with effective advice and suggestions on how to use the collected data to enhance teaching and offer researchers insights and value of DDDM in education.

Research Questions

The researcher has found that the existing research studies on data mining mostly focus on what data collection systems are used and what protocols can be used to help teachers interpret the collected data. It will be interesting to find out teachers’ perspectives on the student information systems, collected data, and data mining for their teaching. Thus, in this research study, the researcher will investigate the following two research questions:

How do math and science teachers perceive the data collected from the student information system that their schools use?

How do math and science teachers’ data mining knowledge and skills help in making decision to improve their teaching?

Methodology

This study adopted a mixed method research approach and occurred in the winter quarter of 2018, specifically from January 2018 to March 2018. Participants were five science teachers and five math teachers in three public secondary schools including one high school and two middle schools located at northern California. Eight of them teach in two middle schools, and two of them teach in a high school.

The three schools were selected because the schools have implemented a student information system to collect their students’ data for 5 years or more. One middle school has been using Aeries, a student information system, to collect their students’ data since 2003 and the other middle school has also been using Aeries since 2006. The high school has been using two student information systems, SchoolLoop for student’s grade reports for 5 years and Illuminate for attendance for 4 years.

All the 10 teachers have been in their positions for 3 years or more. This is to ensure that the teachers have some consistency and longevity accessing to the student information systems in their positions. The ages of the 10 teachers range from 25 to 45. Six of them are White Americans and four of them are Asian Americans. They all have received a teaching credential to teach either math and science in a public secondary school in California and they also have received a master degree in educational technology.

Data were collected from survey and interviews. Interviews aimed to capture the perspectives of the 10 participants regarding their perception, knowledge, skills, and utilization of the collected data from the student information systems (Patton, 2002). Ten face-to-face individual interview lasted time ranging from 20 minutes to 35 minutes were conducted. The interviews assisted in understanding participants’ knowledge and skills as it related to the collected data from student information system. A google forms survey consisted 10 questions with several subquestions under the 10 questions was sent out to the 10 participants in early January in 2018, and all the 10 participants completed the survey by the end of January 31, 2018. Examples of survey questions include: Have you received any training on the use of the student data system? If you do, please describe what type and length of the training you have received? How effective is the training? Can you apply what you learn in the data system training to your daily teaching? Are the training mandatory or voluntary with incentives?

Results

Using the TPACK theory as a framework, this study explores math and science teachers’ perceptions of student data collected from school district’s student information system. TPACK framework was used to examine themes in this study. The teachers’ knowledge themes that were examined in the study included the participants’ knowledge of the student information system used by their school district, types of student data collected with the system, and the capability and functions of the system. The teachers’ skill themes that were examined include using their school’s student information system to enter their students’ data, retrieving their students’ data from the system, and interpreting the data. The teachers’ pedagogy themes that were examined were the teachers’ reflection of their pedagogical change due to the collected data.

Teachers’ Perceptions of Student Information System

Participants were presented with the interview question, “How do you perceive the use of student information system in your school?” Participants appeared to have experiences of using the system to enter their students’ data and some knowledge of what the system could do. Interviews of the math and science teachers who participated in the study indicated that they all used one or two student information systems to enter data. Eight of 10 participants reported that they felt the use of student data system was helpful for both schools and students, especially the collected data on student attendance because school administrators could use the attendance record to monitor some students’ behavior and had an early prevention to avoid problems.

Two participants who worked in the same school indicated that their school used two different student information systems. One was Schoolloop which was for teachers to enter student’s grades. The other one was Illuminate which was used to enter students’ attendance. Participants in the school received mandatory training for both student information systems with incentives and they were able to use both systems. However, they reported that they hoped their school could have only used one data system instead of two.

Teachers’ Skills of Using Student Data Systems

All participants in this study received training from the schools they served. The training lengths varied depending on the school districts. In this study, one middle school and the high school provided 8 hours of training for their teachers before the beginning of the school year. Another middle school provided half day training for all their new teaching employees during the academic year. All participants indicated that they were required to enter information, such as student attendance and grade reports, into the systems.

When the participants were asked whether they knew how to retrieve their students’ data from the system, their responses varied. All 10 participants indicated that they should be able to retrieve data. However, only five of them had the actual experiences of retrieving data from the system. The other five participants indicated that they did not see the need to do so.

In regard to the interpretation of the retrieved data from the student information system, 5 of the 10 participants who had experiences of retrieving data indicated that the data, such as student state standards test results, were good indicators for teachers to reflect how well their students had performed in a certain subject. Two of them actually retrieved data from the student information system implemented in their school regularly and were able to interpret the data because they had to use student’s math tests scores to identify students who needed to be placed in a math intervention program. Three other participants also had experiences of retrieving data from their school information system, yet they have not really used the data for any academic needs.

Teachers’ Pedagogy Change With Effective DDDM

The teachers’ pedagogy themes examined the teachers’ reflection of their pedagogical change. Interview and survey data showed that 8 out of 10 participants did not think that their pedagogy had made any changes due to the implementation of student information system. Only two middle school math teachers reported that they felt a change in their pedagogy because of the need of a math intervention program in their school. The two middle school math teachers used students’ seventh grade state standardized tests scores in math as well as their math test scores in the two final exams in fall and spring semesters to identify students who need to be placed in the math intervention program. The two middle school math teachers indicated that they were able to analyze the data and prepared remedial math lesson plans for the students.

Three teachers reported that student’s demographic data help them to prepare early for English as a second language (ESL) learners. They were able to identify appropriate language resources, such as online translator or translation apps to assist their ESL learners.

Discussion

This research study revealed two common concerns about implementation DDDM in K-12 schools. The common concerns by most of the teachers included time constraint and the need to have easy to read and up-to-date data that were readily accessible. For the first concern, all participants reported that they were very busy during the day and really could not find time to study the data. To solve the time constraint issue, schools could assign time for teachers to analyze the data, to set goals, to plan strategies, to implement the plan, and to reevaluate the goal and plan.

For the concern of data being not readily accessible, it could be helped with an effective data mining protocols, such as data gallery walk strategy mentioned in the literature review. According to the study, participants usually would not go to retrieve data from the student information data system implemented in their school if there was no need to do so. The motives for two participants who were able to use the collected data to plan lessons for the math intervention program were the need and demands. By adapting data gallery walk strategy, school leaders and administrators could help teachers in their schools to have readily accessible data of their students.

Implications and Conclusions

The first practical contribution of this study is that it provides data on perception, knowledge, and application of K-12 public school science teachers regarding the DDDM. The study is significant because studies on the use of student information systems focus on student information data system selection and implementation. The researchers cannot find any research studies investigating teachers’ perception, knowledge, and use of the collected data. In addition, this study has the potential to encourage school district leaders consider teachers’ need when selecting a student information data system for their school districts. It is suggested to consolidate data system use with only one system.

If schools are to utilize DDDM effectively, they must have easy to read and up-to-date data that are readily accessible. Then, they must assign time for teachers to analyze the data, to set goals, to plan strategies, to implement the plan, and to reevaluate their instructional goal and teaching plans. The three data mining protocols presented in this article offer teachers, educators, and school administrators strategies to interpret and analyze students’ data collected from the electronic school systems. It is with hope that teachers will be able to apply what they have learned from the presented strategies to analyze the collected data they have. Then, they can replan their teaching lessons and reevaluate their pedagogy to improve student’s academic performances as well as students’ behaviors.

Footnotes

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

Author Biography

Li-Ling Chen is a full professor at California State University, East Bay. She received her PhD from the University of Texas at Austin, specializing in instructional technology. Her research interests focus on the impact of Web-based instruction and technology in education. She is interested in investigating the relationship of learners' learning styles and online instruction, and how to effectively integrate technology for teacher education students. She has been a speaker in numerous conferences and has been published many papers in various refereed journals.

References

Brown

I. G.

(2016). Leadership’s influence: A case study of an elementary principal's indirect impact on student achievement. Education, 137(1), 101–115. Retrieved from https://login.proxylib.csueastbay.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=ehh&AN=118494321&site=ehost-live&scope=site

Datnow

Hubbard

(2015). Teacher capacity for & beliefs about data-driven decision making: A literature review of international research. Journal of Educational Change, 17(1), 7–28. doi:10.1007/s10833-015-9264-2

Hershkovitz

(2015). Data-driven instruction in the classroom. Technology, Instruction, Cognition & Learning, 10(2), 81–84. Retrieved from https://login.proxylib.csueastbay.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=ehh&AN=110435099&site=ehost-live&scope=site

Kennedy

M. J.

Mimmack

Flannery

K. B.

(2012). Innovation in data-driven decision making within SWPBIS systems: Welcome to the gallery walk. Beyond Behavior, 21(3), 8–14. Retrieved from https://login.proxylib.csueastbay.edu/login?url=http://search.ebscohost.com/login.aspx?diect=true&db=ehh&AN=83304854&site=ehost-live&scope=site

Kaufman

T. E.

Graham

C. R.

Picciano

A. G.

Popham

J. A.

Wiley

(2014). Data-driven decision making in the K-12 classroom. In J. M. Spector, M. D. Merrill, J. Elen, M. J. Bishop (Eds.), Handbook of research on educational communication and technology (4), (pp. 337–346). New York, NY: Springer Science + Business Media. doi:10.1007/978-1-4614- 3185-5_27

Kerns

(2013). Data-driven decision making can improve student learning. Dream Box Learning, Retrieved from http://www.dreambox.com/blog/adaptive-learning-enables-data-driven-decision-making

Koehler

M. J.

Mishra

(2009). What is technological pedagogical content knowledge. Contemporary Issues in Technology and Teacher Education Journal, 9(1). Retrieved from http://www.citejournal.org/volume-9/issue-1-09/general/what-is-technological-pedagogicalcontent-knowledge

Marsh

J. A.

Farrell

C. C.

(2015). How leaders can support teachers with data-driven decision making. A Framework for Understanding Capacity Building. Educational Management Administration & Leadership, 43(2), 269–289. doi:10.1177/1741143214537229

Means

Padilla

Gallagher

, & SRI International. (2010). Use of education data at the local level: From accountability to instructional improvement. U.S. Department of Education Office of Planning, Evaluation and Policy Development. Retrieved from http://www2.ed.gov/rschstat/eval/tech/use-of-education-data/index.html

10.

Patton

M. Q.

(2002). In qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: SAGE Publications.

11.

Staman

Visscher

A. J.

Luyten

(2014). The effects of professional development on the attitudes, knowledge & skills for data-driven decision making. Studies in Educational Evaluation, 42, 79–90. doi:10.1016/j.stueduc.2013.11.002

12.

Wayman

J. C.

(2005). Involving teachers in data-driven decision making: Using computer data systems to support teacher inquiry and reflection. Journal of Education for Students Placed at Risk, 10(3), 295–308.