Arab Israeli EFL teachers’ perceptions and practices vis-à-vis teaching higher-order thinking skills: A complicated relationship

Abstract

Higher-order thinking skills (HOTS) have been an integral part of the English language curriculum in the Israeli school system since 2013, when questions that needed HOTS were added to the matriculation exams in all modules. Teachers subsequently have been responsible for integrating such questions into their teaching. The study aims to investigate the cognitive levels of questions posed by 13 Arab Israeli EFL teachers while teaching reading comprehension in heterogeneous tenth-grade classrooms. It also seeks to reveal the teachers’ perceptions with respect to teaching HOTS, and the challenges they face while doing so. Data were gathered by means of a classroom observation checklist and semi-structured interviews. The results show that the teachers tended to emphasize lower-order rather than higher-order questions. The findings of the thematic analysis of interviewees’ answers indicate that while the teachers have positive perceptions about teaching HOTS, they nevertheless face significant challenges. Factors hindering HOTS implementation include the teachers, the students, the system, and certain social norms. This implies the need for training courses to develop teachers’ knowledge about HOTS and to ensure successful implementation in the English language classroom.

Keywords

Arab Israeli EFL teachers cognitive level of questions higher-order thinking skills reading comprehension teachers’ perceptions

I Introduction

Teaching higher-order thinking skills (HOTS) to students of all ages is fundamental. Not only are HOTS essential to student success in educational settings but they also represent a skill set that translates into long-term achievement throughout a person’s life (Zohar, 2008). In acknowledgment of this, in 2001, Israel’s Ministry of Education began to establish HOTS as a centerpiece of the educational framework by formulating a policy that specifies teaching HOTS as an explicit pedagogical objective. The curriculum, accordingly, was altered, launching a race to create meaningful paths for students to acquire HOTS in all disciplines across the K-12 educational system. The Ministry also appointed experienced educators in each field to oversee implementation of the changes.

Recognizing the role of English language as a discipline that enables students to function successfully in technologically advanced societies, officials in the Ministry of Education revised the English curriculum in 2013 to integrate HOTS at all levels. Since then, English teachers have been required to teach HOTS explicitly through focused instruction, and to generate and implement assessment plans, which include instructional questions meant to enhance students’ thinking skills.

The importance of these skills was further reinforced when HOTS questions were added to the reading comprehension sections of English standardized exams at all levels. This resulted in major educational challenges for teachers who suddenly found themselves navigating new expectations and curricular focuses (Gallagher, Hipkins, & Zohar, 2012; Zohar, 2013).

Previous studies show that teachers tend to have positive perceptions about teaching HOTS (e.g. Ardini, 2017; Hashim, 2003; Seif, 2017; Seman, Yusoff, & Embong, 2017; Ulmer & Torres, 2007). Likewise, they acknowledge the importance of infusing thinking strategies into their teaching, not only for the students’ benefit but also for their own personal growth. However, whether teachers are able to teach HOTS and generate questions that promote students’ HOTS is a matter in need of further investigation.

Studies conducted by Hashim (2003) and Ulmer and Torres (2007) note a disconnect between teachers’ acknowledgement of the importance of HOTS and their teaching practices. Findings emerging from more recent work, on the other hand, reveal that teachers with more positive perceptions vis-à-vis HOTS have a greater tendency to use methods that encourage students to think critically (e.g. Mustika, 2019; Utami, Nurkamto, Marmanto, & Taopan, 2019).

To the authors’ knowledge, no previous studies have been conducted hitherto in the Arab sector in Israel that investigate English language teachers’ perceptions and practices following the new reform. Hence, it was decided that a study was needed that would delve deeply into teachers’ perceptions about teaching HOTS in English, and explore their practices in reading comprehension lessons specifically. The present study, accordingly, aims to answer the following questions:

What are the cognitive levels of the questions posed by Arab Israeli high school English teachers during reading comprehension lessons?

What are the teachers’ perceptions vis-à-vis teaching HOTS?

What are the challenges the teachers face while integrating HOTS in the English as a foreign language (EFL) classroom?

II Literature review

1 Reading comprehension

Scholars in educational linguistics use cognition and linguistics to identify strategies and interventions that might be considered ideal practices for teaching languages (Hult, 2010; Spolsky, 2010). Teachers, according to Spolsky (2010), are responsible for providing students with literacy development opportunities. And to become literate, as Kucer (2009) opines, students must learn how to control effectively, efficiently, and simultaneously not only the linguistic dimensions of written language but also the cognitive, sociocultural, and developmental dimensions.

In contemporary research on literacy, a focus on reading comprehension development undoubtedly predominates. Ballou (2012), for instance, asserts that students ought to know how to approach a reading exercise with critical skills, and that when students acquire such skills, noticeable gains in comprehension can be achieved. Snow (2002) also claims that when students reach the level at which they are able to use different reading skills simultaneously, they know when to apply each strategy ‘and read with comprehension even when the material is neither easy to understand nor intrinsically interesting’ (p. xiii). Sidek (2010), similarly, affirms that ‘[w]ithout reading comprehension skills that lead to information literacy, students’ academic performance, particularly at higher education institutions, could be severely handicapped due to their inability to acquire the required content’ (p. 4). This comports with Enabulele’s (2011) finding that proficient readers tend to be more successful in their lives. Indeed, all in all, the consensus among scholars today seems to be that HOTS play a role in student’s achievement and in learners’ long-term ability to make meaning from written texts (Lateef, Dahar, & Latif, 2016; Nourdad, Masoudi, & Rahimali, 2018; Zohar & Dori, 2003).

In this context, teachers are viewed as essential guides leading students through the developmental stages toward an aptitude in making meaning and extracting knowledge from texts (Celce-Murcia, 1991; Durkin, 1993; Grellet, 1981; Mayer, 2003; William, 1984). Accordingly, they are expected to nurture a classroom culture where students have meaningful reading experiences that enable them to succeed (Freahat & Smadi, 2014). Ballou (2012), for example, insists that teachers must emphasize reading comprehension strategies and provide students with sufficient and explicit reading instructions in order to help them read critically. Others, however, argue against such a teacher-centered approach to student development. Aloqaili (2012), for instance, contends that it is the student reader’s responsibility to interact with the text and apply their experience and previous knowledge to establish meaning, learn, and develop understanding. While the present article focuses mainly on the role of teachers, what we ultimately come to understand out of the tension between the two aforementioned positions is that reading is an activity that involves multiple players all of whom rely on perceptual, linguistic, and cognitive abilities.

2 The role of questions in pedagogy

One of the most salient components of the teaching/learning process is questioning. Questioning is a type of dialogue acknowledged to be uniquely beneficial to teachers and students alike. Teachers can use well-designed questions to engage students in lessons, develop their thinking skills, and assess their understanding. These, in turn, help students to find meaning in what they learn and make connections to prior learning.

In reading comprehension lessons, each text the teacher assigns is accompanied by both oral and written questions, which teachers tend to derive from textbooks, while also supplementing them with questions especially tailored to the lesson objectives and their own particular needs. Prior studies show that through simple use of thoughtfully constructed questions; teachers can enhance students’ achievement, retention, and engagement (Remark & Ewing, 2015; Larson & Lovelace, 2013; Tofade, Elsner, & Haines, 2013). Larson and Lovelace (2013), for example, affirm the valuable role that questions play in pedagogy as a technique for encouraging critical thinking among students, stimulating their interest, and checking their understanding. Seth (2013) adds in this regard that teachers have the capacity to meaningfully enhance students’ cognitive levels by means of thought-provoking questions. Thompson (1997), similarly, confirms that a question’s complexity can have a broad effect on students’ learning. Indeed, as foundational research in support of examining questions as tools for learning grows, it paints an increasingly compelling picture of the important role teacher-developed questions play.

In teacher education for the EFL classroom, teachers learn that reading comprehension questions are divided into two general categories. The first includes questions that operate on a lower cognitive level, namely, those that inquire about the information in the text (that is, the students must read the text to find the information required). Questions of the second type, on the other hand, ask students to do something with the information they derive from reading the text. Lightbown and Spada (2006) term these two types ‘display’ and ‘genuine/referential’ questions, respectively, while Ur (1996) refers to them as ‘closed-ended’ and ‘open-ended’. This conception of questions falling into two distinct categories invariably calls for a discussion of teacher’s expectations.

Each question posed by a teacher involves certain expectations of outcomes. Display or closed-ended questions lead to short and simple responses that are not cognitively demanding. Typically, they ask students to search a text for a specific answer. Genuine or open-ended questions, on the other hand, require higher cognitive processing, and deeper understanding. Questions of this type are designed to invite students to think critically by providing them with insight into how to process new information and assess it from a variety of different perspectives (Lee & Lai, 2017). By incorporating both closed-ended/display questions and open-ended/ referential questions teachers can stimulate active participation of all students regardless of their individual cognitive level (Lightbown & Spada, 2006; Ur, 1996). While both categories of questions result in expression of the learner’s reading comprehension, Lewis (2007) claims that higher-level thinking questions elicit more information about students’ understanding. It is clear then, that a need exists for further investigation of the categories and cognitive levels of questions used by teachers.

3 Cognitive levels of teachers’ questions

Several taxonomies have been proposed in the area of questions and objectives. The cognitive taxonomy created by Bloom et al. (1956) and revised by Krathwohl (2002) is used in this study as a tool for analysis. The Revised Bloom’s Taxonomy consists of six levels divided into lower-order thinking skills (LOTS) and HOTS: remembering, understanding, and applying are considered LOTS, while analysing, evaluating, and creating are considered HOTS. Surjosuseno and Watts (1999) suggest that Bloom’s Taxonomy can be used to effectively plan objectives, form questions, and assess items while teaching critical reading in EFL classes.

Studies that investigated the use of HOT questions in teaching reading comprehension (Fahim & Sa’eepour, 2011; Fakeye & Ayede, 2013; Nourdad et al., 2018; Remark & Ewing, 2015; Teemant, Hausman, & Kigamwa, 2016) indicate that not only do HOT questions help train students to express their ideas but they can also substantially improve students’ learning processes, engagement, and achievements. Fahim and Sa’eepour (2011) and Teemant et al. (2016), for example, in examining the effects of teaching HOTS on reading comprehension, find that teaching critical thinking skills has significant beneficial effects on students’ reading comprehension and language proficiency. Likewise, Remark and Ewing (2015) and Nourdad et al. (2018) in analysing the ramifications of using HOTS questions for students’ reading comprehension and engagement, conclude that by asking HOTS questions as a scaffolding strategy, classroom interaction and students’ reading comprehension achievements can be significantly improved. A common recommendation that emerges from these studies is that English teachers ought to use higher-order thinking (HOT) questions to guide students in reading comprehension.

Despite the widely observed positive effects of HOTS instruction, studies that examined the cognitive levels of teacher’s questions indicate that teachers nevertheless have a powerful tendency to fall back precisely on closed-ended, lower-order thinking (LOT) questions when leading classroom discussions. Tofade et al. (2013), for instance, show that teachers’ questions tend to be concentrated on the lower cognitive levels of remembering and understanding, while Shafeei, Hassan, Ismail, and Azian (2017) observed that out of 51 questions posed by teachers in their study, an overwhelming 49 (96.08%) were display questions. Likewise, teachers in a recent study by Rachmawaty and Ariani (2019) employed display questions far more often than referential questions.

Several additional studies yielded similar results (Ball & Garton, 2005; Ewing & Whittington, 2007; Fischer, Bol, & Pribesh, 2011; Larson & Lovelace, 2013; Marzano, 2003; Zulkpli, Abdullah, & Atan, 2017): seldom did teachers’ questions compel students to analyse, evaluate, or create information. Moreover, studies that analysed reading comprehension questions in textbooks (e.g. Assaly & Smadi, 2015; Freahat & Smadi, 2014) found that low-level questions were dominant, comprising 61% and 83%, respectively, of the questions posed in the textbooks’ reading comprehension sections.

4 Teachers’ perceptions about teaching HOTS

A positive correlation appears to exist between teachers’ perceptions and knowledge about teaching HOTS and their practices (Hashim, 2003; Rachmawaty & Ariani, 2019). In other words, teachers’ tendency to regard teaching HOTS as important depends, to a large extent, on how strongly they believe that such thinking skills positively affect students’ academic achievements, help them to make the right decisions in their daily lives, and encourage them to think out of the box (Seif, 2017). He further emphasizes that when teachers are convinced of the importance of HOTS to their students and learn how to use them, they demonstrate a greater readiness to teach to higher cognitive levels, and to use diverse teaching strategies to promote students’ HOTS.

Studies investigating teachers’ perceptions about teaching at higher levels of cognition (Ansori & Nurkamto, 2019; Ardini, 2017; Enabulele, 2011; Ulmer & Torres, 2007; Wilson & Narasuman, 2020; Seman et al., 2017) reveal that teachers not only are aware of the importance of implementing HOTS in their lessons but are generally in favor of doing so . Thus, they express not infrequently a readiness to be creative, to work as facilitators to promote critical thinking (Ansori et al., 2019), and even to cooperate in planning HOTS teaching and assessment (Seif, 2017; Utami et al., 2019).

Considering that Seif (2017) is the only study hitherto that has sought to examine the effects of HOTS implementation among Arab comprehensive high school teachers in Israel, the present study, we are convinced, holds a unique value in that it is likely to add significantly to the field by focusing on the perceptions and practices of Arab Israeli EFL teachers vis-à-vis HOTS.

5 Challenges involved in teaching HOTS

It has been widely observed cross-disciplinarily that a gap frequently exists between teachers’ positive attitudes toward HOTS, on the one hand, and their actual in-class practices, on the other. For instance, several studies that have investigated challenges shaping the types of questions teachers use in teaching HOTS (Ardini, 2017; Dwee et al., 2016; Seman et al., 2017; Shafeei et al., 2017) have found that the teachers themselves often comprise an obstacle. That is, the dearth of knowledge and experience they possess with respect to HOTS can have a less than auspicious impact on the cognitive level of the questions they formulate. In this context, recent work by Wilson and Narasuman (2020) and Veloo, Ramli, and Khalid (2016) shows that teachers’ questions often depend on prior knowledge and training. While a number of other studies cite teachers’ awareness of the importance of HOTS as a factor influencing the cognitive level of the questions and questioning strategies used (Rachmawaty & Ariani, 2019; Utami et al., 2019). Finally, from a slightly different perspective, Seif (2017) and Seman et al. (2017) argue that teachers’ persistent reliance on traditional teacher-centered instructional methods combined with deep reservations towards new ones comprise significant barriers to HOTS implementation.

Students comprise the second major obstacle noted by teachers. In this regard, teachers note differences in students’ English proficiency levels and diverse learning abilities as major challenges, which force them to concentrate more on content using LOT questions (Ardini, 2017; Seman et al., 2017; Shafeei et al., 2017). Likewise, Zohar and Dori (2003) show that while HOT questions can be beneficial to low and high achievers alike, teachers tend to prefer using them with high achievers exclusively. Dwee et al. (2016), for their part, cite second language anxiety as a related problem that hinders the creation of a thinking classroom, asserting that teachers cannot gauge a student’s critical thinking level when they are too embarrassed to talk, and that teachers tend to resort to LOT questions when faced with passive learning attitudes among students. Finally, Ellis (1993), Fakeye and Ayede (2013), and Rachmawaty and Ariani (2019) all have attested to English language teachers’ use of display questions to enhance student interaction, motivation, and attention, while simultaneously maintaining control of the classroom.

A third challenge often noted by teachers is time restraints. In three separate studies (Ardini, 2017; Ellis, 1993; Seman et al., 2017), teachers claimed that they deliberately relied on lower-order rather than higher-order thinking questions due to insufficient classroom time. Indeed, teachers express a preference for approaches and techniques that require less time and preparation (Dwee et al., 2016), while citing tight schedules and an overload of responsibilities, content to cover, and skills to teach as factors precluding or limiting their ability to devote attention to HOTS (Seif, 2017; Seman et al., 2017; Wilson & Narasuman, 2020; Zulkpli et al., 2017).

Additional impediments to HOTS teaching include the learning environment, cultural norms, and lack of resources. Among learning environment challenges, classroom size (Hashim, 2003), atmosphere, and seating arrangements are noted. With respect to challenges of a cultural nature, Seman et al. (2017) refer to the culture of ‘spoon feeding’ that prevails in schools today. Finally, examples of resource-related obstacles include lack of professional development courses and appropriate materials. About the need for HOTS-related training and support for teachers, scholars are adamant (Ansori et al., 2019; Zohar, 2013). As for learning materials, suitable textbooks that incorporate HOT questions similar to the ones that appear on standardized tests can be profoundly helpful (Ansori et al., 2019; Seif, 2017; Wilson & Narasuman, 2020). The present study, as aforesaid, seeks to investigate the specific challenges encountered by Arab Israeli EFL teachers and their effects on the cognitive level of the questions the teachers pose to students in the course of reading comprehension lessons.

III Methodology

1 Context of the study

Although English is an international language used worldwide, few Arab students in Israel feel that it is essential for them in their daily lives. English is a third language for them, after Arabic, their native language and the language of communication and instruction in the Arab school, and Hebrew, the official language of the State. Hebrew is the language that predominates in the media, government, workplace and higher education. It is, accordingly, considered more important than English, and generally easier to assimilate (Amara, 2014).

While English is a second (rather than third) language for most Hebrew speakers, the English curriculum and matriculation examinations are nevertheless uniform in Israel’s school system. Accordingly, students in both the Jewish sector and the Arab sector are assigned the same number of English hours a week, and teachers in both sectors are obliged to use one of the textbooks recommended by the Ministry of Education for each grade depending on the students’ proficiency level.

Master Class, the textbook used by the participants in this study, is one of the textbooks recommended by the Ministry of Education for students at a higher proficiency level. Cognitive levels of reading comprehension questions in the Master Class textbook were evaluated in a previous study (Assaly & Smadi, 2015), which showed that 52% of the questions/activities emphasize the cognitive level of comprehension, while higher-order thinking questions/activities comprise 40% of the questions.

2 Study design

The study was conducted using a mixed-methods approach. Both quantitative and qualitative data were gathered by the researchers as a means ‘to offset the weaknesses inherent within one method with the strengths of the other’ (Creswell, 2014, p. 213). This design was selected to enable the researchers to better understand and interpret the results, and to ensure that rich qualitative data would be effectively obtained from participants to explain the quantitative data.

3 Participants

The study included a convenience sample of 13 Arab high school EFL teachers who taught in nine comprehensive schools in the Northern and Southern Triangle regions of Israel, and used the Master Class English textbook. Participants included three male and ten female teachers, whose high school teaching experience ranged from 2 to 20 years. All the teachers were cognizant of the 2013 reform and their obligation to teach in accordance with it. The researchers observed two to three reading comprehension lessons taught by each participant in heterogeneous tenth-grade classrooms. Class sizes ranged between 25 and 35 students. Each participant was observed while teaching one of three texts that related to two units entitled ‘A Matter of Taste’ and ‘In the Frame’.

4 Instruments

a Observation checklist

An observation checklist based on Bloom’s Revised Taxonomy (Krathwohl, 2002) was used to record and categorize the cognitive levels of the questions posed by the participants. The checklist was composed of a table with seven columns, whose items included transcripts of the questions, and the six cognitive levels of Bloom’s Taxonomy.

b Semi-structured interviews

To answer the second and third research questions, the researchers used semi-structured interviews with the aim of drawing out in-depth information from the teachers. A structured interview guide was developed based on the objectives of the study, which revolved around teachers’ understanding of HOTS, their beliefs about teaching HOTS, and the perceived importance of HOTS for students. The researchers also sought to learn about the teachers’ teaching practices, and the potential challenges they face that hinder teaching HOTS. To validate the interview questions, a panel of five experts in the field of English language instruction and curriculum design evaluated the questions provided in the interview guide and offered their critiques and suggestions before they were used.

5 Data collection

The researchers observed two to three lessons per teacher depending on the time required to cover a specific text. Most teachers covered a text in two lessons. Each teacher was observed from the moment they began the lesson until they deemed the text fully covered. All lessons were audiotaped. Once the observations were completed, each teacher was contacted by the researchers to arrange an interview. Most interviews took place in the school setting at a time convenient for the teacher. Permission was obtained from each participant to audio-record the interview. Interviews were generally no longer than 30 minutes in duration.

6 Data analysis

a Analysis of the cognitive levels of teachers’ questions

The cognitive level of the teachers’ questions was measured using the data collected from the observations for each of the 13 teachers. All observations were audio-taped and transcribed. Each teacher’s questions were then coded, entered into the checklist in the order they were posed, and categorized across the six cognitive levels of Bloom’s Taxonomy (Remembering, Understanding, Applying, Analysing, Evaluating, and Creating).

Afterwards, the researchers calculated the frequency of each cognitive level for every teacher observation. The six cognitive levels were then combined into a grand total of all the questions posed by the 13 teachers to determine the frequency of the teachers’ questions. Finally, the number of questions at each cognitive level was divided by the grand total of teachers’ questions, resulting in a percentage of the teachers’ questions at each cognitive level.

b Analysis of semi-structured interviews

All interviews were recorded and transcribed. The responses were then read, and a combination of inductive and deductive approaches to data coding and thematic analysis were applied. While some of the codes and themes emerged from the content of the data itself, most were derived from the concepts and ideas the researchers brought to the data. Codes that related to each other were grouped together into themes (Braun & Clarke, 2006). The themes were then classified. Some themes were divided into subthemes. Finally, the themes and subthemes were counted for frequency.

IV Findings

1 What are the cognitive levels of the questions posed by Arab Israeli high school English teachers during reading comprehension lessons?

Table 1 presents the number of questions posed by EFL teachers during the reading comprehension lessons observed. It shows the number of questions at each cognitive level, arranged in descending order according to the frequencies of each cognitive level. It is noted that the total number of teachers’ questions was 483. The vast majority were aimed at the cognitive levels of understanding (206) and remembering (189). By contrast, participants only seldom asked questions demanding the higher cognitive skills of analysing (41), evaluating (29), creating (17) and applying (11).

Table 1.

Frequencies and percentages of cognitive levels of teachers’ questions during reading comprehension lessons.

Cognitive level of question	Frequency	Percentage (%)
Understanding	206	42.7
Remembering	179	37.1
Analysing	41	8.5
Evaluating	29	6.0
Creating	17	3.5
Applying	11	2.3
Total	483	100.0

2 What are the teachers’ perceptions vis-à-vis teaching HOTS?

Analysis of the findings on this question revealed two main themes:

teachers’ beliefs, namely, the teachers’ own views and interpretations of HOTS; and

teachers’ attitudes, which refers to the effects that teachers’ beliefs have on their behavior.

The first theme was divided into three sub-themes, while the second was divided into two.

a Teachers’ beliefs

Definition of HOTS

Nine out of the 13 teachers defined HOTS as a teaching tool. However, they were neither specific nor able to provide a conceptual definition of HOTS. In most cases, they noted functional applications or what was expected of students. TM, for instance, defined HOTS as ‘the ability to analyse and compare two things’. He claimed that ‘it is a skill that helps students to think deeply.’ ‘The students need to process information in order to answer the question,’ he added. Similarly, TD conceived HOTS as a ‘way to absorb aspects of the texts . . . the ability to analyse, to criticize, to compare and contrast.’ TF said, ‘I taught thinking skills without naming them before, but now I emphasize the name of each skill by asking students to compare and contrast or to recognize different perspectives.’ TC, however, related to HOTS mainly as a strategy to pass the Bagrut (matriculation) exam.

Only four teachers out of the 13 defined HOTS in conceptual terms, using phrases such as ‘metacognitive thinking’, ‘cognitive processing of the material read’, ‘using the cognitive taxonomy for learning’ and ‘thinking beyond the text and using one’s own judgment’. None of them, however, were able to offer a clear-cut definition of HOTS.

Benefits of teaching HOTS

Most teachers conveyed a positive outlook on teaching HOTS, citing its various benefits. TD, for example, claimed that ‘HOTS help students to better understand and analyse literary and factual texts.’ TG, likewise, expressed a belief that ‘HOTS expand students’ horizons and develop their thinking skills’ while in TK’s view, ‘students are more active in the learning process when taught HOTS.’ Others such as TM held that HOTS ‘help students manage their learning’, while TD pointed out that ‘HOTS make students care more about education than grades and help them acquire values and ethics.’ Finally, TJ opined that ‘by using HOTS lessons are more interesting and challenging.’

HOTS as innate/teachable

Twelve out of the 13 teachers regarded HOTS as teachable/learnable skills. On this score, TA suggested that ‘thinking skills can be taught for those who do not have them by simplification’, while TK felt that ‘teaching these skills should begin from an early age at school’. ‘Thinking skills are teachable,’ TM remarked, ‘but they need efforts, money, good intentions and will.’ Two teachers, however, tended more to the view that students were born ‘either with or without’ such skills, submitting that teachers ought to deal with each student on an individual basis according to their own level.

b Teachers’ attitudes

Teaching HOTS as the teacher’s responsibility

All the participants accepted the notion that cultivating HOTS was part of their responsibility as teachers. TC, for example, stated in this regard, ‘It is our role to teach HOTS; we have to teach for life not for exams,’ while TF’s philosophy was that ‘HOTS are important skills for the future; we have to work on them in class.’ Similarly, TB commented that ‘it is important to raise students’ awareness of HOTS, to teach critical thinking to improve their thinking skills.’ Lastly, TE opined, ‘Aims justify means; we should aim at educating a generation that thinks critically, not one that cares more about grades.’

How to develop students’ HOTS

All the participants expressed a sense of obligation to work on developing students’ thinking skills, and even proposed some ideas for doing so. TA suggested, for instance, that ‘English teachers should integrate thinking skills while teaching listening, reading, writing, and speaking,’ while TC proposed that ‘students should be given tasks to practice using thinking skills.’ TG went even further, espousing the view that ‘teachers should not be traditional. They should teach by questioning and encourage students to ask. Students should not learn the material by heart.’ Introducing students to different genres was suggested by TF as a way to ‘develop students’ thinking skills’. Finally, on the relationship between teaching HOTS and assessment methods, TL added that ‘we should change our formative and summative assessment methods according to the requirements of the new reform.’

In sum, from the findings on teachers’ perceptions it is clear that the participants were aware of the new requirements embedded in the revised curriculum, and that this had a positive effect on their perceptions and attitudes toward HOTS. However, the enthusiasm they voiced for HOTS was scarcely reflected in the cognitive level of the questions they posed during the lessons observed.

3 What are the challenges the teachers face while integrating HOTS in the EFL classroom?

Teachers’ responses to the third research question reveal key challenges faced by the teachers in their attempts to teach HOTS, which were attributed to both internal and external factors. The internal factors relate to the teachers themselves, while the external factors have to do with the students, the system, and cultural norms.

a Internal factors

Participants admitted that some of the barriers to HOTS implementation lie within themselves. Five teachers confessed, for instance, that ‘preparing lessons in which HOTS are infused requires a great amount of effort in terms of preparation and assessment. Teachers prefer the easy way, which is not too time consuming.’ Four teachers confided that lack of knowledge and experience impede their ability to plan and teach critical thinking. TF, for example said, ‘Teaching HOTS necessitates tremendous efforts and experience, and a lot of preparation. Designing questions and tasks to assess students is also not easy. Many teachers are not ready for that.’ The teachers added that they needed more training to improve their understanding of HOTS, and to cultivate their teaching and assessment strategies. Two participants also cited the use of traditional methods as a factor that stands in the way of HOTS development, noting that because some teachers were not aware of the importance of HOTS, they found themselves unprepared for the policy changes, and thus persisted in their reliance on outmoded approaches.

b External factors

The students

Three out of 13 teachers considered students’ English proficiency level to be a key external factor. Three as well emphasized the effects that students’ attitudes toward learning English and/or HOTS have on the teaching/learning process. An overwhelming ten teachers, however, associated the problem of teaching HOTS with students’ cognitive levels, while seven related it to prior learning habits: ‘Our students are used to taking everything from the teachers,’ claimed TG. Of the participants, TB articulated the challenges students present for teachers in perhaps the most candid terms: ‘HOTS are suitable for good students, especially when we are talking about teaching them in English. With weak students we face two problems: low cognitive level and inadequate language level.’

The system

Participants also highlighted system-related barriers. Three teachers, for instance, expressed a belief that age of initial exposure to HOTS at school has a direct effect on students’ cognitive level, with TK alleging in this regard that ‘teachers in elementary schools do not expose kids to HOTS’. Two participants attributed difficulties in HOTS implementation to heterogeneous classrooms, one of them explaining: ‘It is difficult to teach HOTS in a class where the thinking levels of the students are not the same.’ Lack of resources was also considered a challenge. TF, for example, argued that insufficient resources and budgets allocated to schools impair educators’ ability to teach HOTS: ‘Schools do not have enough money to buy computers or teaching aids, pay for extra hours for the teachers to teach extra lessons or to adapt to new materials.’

Likewise, teachers cited a dearth of suitable learning materials, and in particular textbooks, as a factor undercutting HOTS instruction. Among them were TC and TJ, who complained that the textbooks recommended by the Ministry of Education could not be relied upon, with TJ adding that ‘most of the reading comprehension questions that follow the texts in the textbook are unlike the ones students encounter in the matriculation exam.’ TH, similarly, maintained that ‘changing the curriculum without offering suitable resources to depend on, the teachers will not be able to apply what is required efficiently’.

Five teachers pointed to the dramatic transformations the curriculum had undergone in the last two decades, claiming that the rapid changes, combined with little training, sowed confusion among teachers: ‘The Ministry of Education adds and changes parts of the curriculum every few years and the assessment methods are changed accordingly,’ said TE.

Finally, four teachers attributed difficulties to time constraints, insisting that the Ministry of Education must allocate more than a mere four English lessons per week to allow teachers to cover what is required in the new curriculum. TD, for example, wondered, ‘How are we supposed to prepare the students for the Bagrut exam and apply all the changes when we only have four lessons a week?’ TF put it in even franker terms: ‘We can’t teach the four skills, literature, and infuse HOTS in four lessons a week.’

Cultural norms

Eight teachers claimed factors related to cultural norms pose obstacles to teaching HOTS. Three pointed out that in the Arab community there is little awareness among parents of the importance of HOTS, and that even where there is a degree of cognizance few have practical experience applying HOTS with their children. TA, for example, asserted that ‘the students’ first encounter with HOTS is at school, or even high school,’ while TK bemoaned the fact that ‘students were not raised to think critically. Parents concentrate on their kids’ grades more than how they think.’ TC, on the other hand, looked at the matter from a slightly different perspective: ‘I feel that using HOTS affects teachers’ authority in the classroom; that is why they prefer not to apply them.’

V Discussion

1 Cognitive levels of teachers’ questions

Questioning plays a critical role in helping teachers to structure the classroom environment and organize course content. Scholars recommend using both closed-ended/display questions and open-ended/referential questions, as each is effective in achieving certain instructional goals (Ellis, 1993; Rachmawaty & Ariani, 2019). Incorporation of the two types of questions, it has been shown, can enhance active participation of all students regardless of cognitive level diversity (Lightbown & Spada, 2006; Ur, 1996). Marzano (2003) has argued that a careful balance of both declarative knowledge (the what) and procedural knowledge (the how) is key to effective instruction.

The findings of the present study, however, demonstrate that teachers often fail to establish a balance, and that they disproportionately favor questions of the former type (the what) over those of the latter (the how). The results, notably, are congruent with prior studies that investigated the cognitive levels of EFL teachers’ questions (e.g. Ansori et al., 2019; Ewing & Whittington, 2007; Fischer et al., 2011; Larson & Lovelace, 2013; Shafeei et al., 2017).

The teachers’ questions in the current study tended to be ones requiring short and simple answers, often comprised of one word, like yes or no. And in those cases where a longer answer was expected, it was usually a phrase or sentence to be drawn verbatim from the text. Some typical questions were: ‘Which technique are they talking about?’, ‘What is the topic sentence of this paragraph?’, ‘Look at the picture and describe what you see. What is the main idea of the first paragraph?’ and ‘What do we mean by encounter?’ Even when the teachers tried to activate the cognitive level of applying, none of the questions they asked actually required that the students apply what they learned from the text read in a new context. Instead, their questions revolved around (rather mechanical) assignments like identifying grammatical forms in a text, transforming sentences from passive to active, underlining adjectives and adverbs, etc.

On the other hand, the teachers did, to a certain extent, pose reading comprehension questions of the sort that, according to Larson and Lovelace (2013), stimulate students’ interest, encourage critical thinking, and aid in the evaluation of student comprehension. Comprising 18% of the total questions, two noteworthy examples were as follows: ‘Can you anticipate what we will have in paragraph three?’ and ‘Would you like to be a professional photographer? Why or why not? If so, what type of photographer would you like to be and why?’

As non-participant observers, the researchers noted that only rarely did the teachers formulate HOT questions. Instead, they took most questions virtually word for word from the textbook, a finding that comports with prior studies (e.g. Rachmawaty & Ariani, 2019; Shafeei et al., 2017; Tofade et al., 2013; Veloo et al., 2016). Moreover, a majority of these questions demanded only short answers, that is, they tended to be multiple-choice or yes-or-no questions. Thus, the teachers neglected precisely the open-ended textbook questions, which asked students to express their opinions, as well as activities calling for work in pairs or groups to discuss, for example, the author’s perspective – in other words, those questions and follow-up activities meant to promote HOTS, as asserted by Assaly and Smadi (2015).

2 Teachers’ perceptions regarding teaching HOTS

The findings show that all the teachers who participated in the study had positive perceptions about teaching HOTS and acknowledged HOTS as important and beneficial to students’ academic performance and future lives. Of these, most articulated, in one fashion or another, the conviction that they must prepare students not only for exams but also for life, and that teaching HOTS was an essential part of fulfilling this obligation. These results, again, are consistent with findings from previous studies (e.g. Ansori et al., 2019; Enabulele, 2011; Hashim, 2003; Seman et al., 2017; Wilson & Narasuman, 2020).

Most of the teachers, likewise, asserted a belief that HOTS are skills that are teachable/learnable, albeit with some effort, and that, regardless, it is part of their duty as teachers to nurture them. Seif (2017) and Utami et al. (2019) also found that teachers tend to take full responsibility for developing not only their students’ HOTS but also their own professionalism.

Yet, when the participants in the current study were asked what HOTS meant, their responses revealed that most actually lacked sufficient knowledge about HOTS. Indeed, only a few were able to provide a conceptual definition, while the rest struggled to articulate a meaning in anything more than general terms. Thus, similar observations recorded in previous studies (e.g. Ardini, 2017; Seman et al., 2017; Shafeei et al., 2017) where teachers’ responses and practices reflected misconceptions about the meaning of HOTS and how to conduct a lesson using critical thinking, were further corroborated. On this score, TB admitted, ‘We are not fully prepared for the program,’ while TK confessed, ‘Sometimes I feel I can integrate HOTS in my lessons, while other times I find it difficult.’

A glance at Table 1 above reveals the vast chasm observed between the teachers’ alleged attitudes toward teaching at higher cognitive levels, on the one hand, and how they actually taught in class, on the other. Reasons for this discrepancy between teachers’ positive attitudes and their in-class practices will be discussed in the following section.

3 Explanations for the divergence between teachers’ positive perceptions about HOTS and their classroom practices

Analysis of the findings pertaining to the third research question shed light on the numerous challenges teachers face when it comes to teaching HOTS. These challenges were divided into two categories: the first, internal factors, which have to do with the teachers themselves, and the second, external factors, which included the students, the system and cultural norms.

Teachers’ insufficient understanding of the concept of HOTS and lack of experience in applying such skills in the classroom, according to the participants, comprise key obstacles to teaching HOTS, which were observed primarily in two situations. First, as aforementioned, while all of the teachers attempted to define what HOTS meant to them, only a few succeeded in articulating a conceptual definition. In an earlier study, Seman et al. (2017) attributed teachers’ misconceptions about certain key components of HOTS to their limited understanding and knowledge of the subject.

Second, during the reading comprehension lessons, the researchers observed that the teachers seldom posed higher-order thinking questions, and that most of the questions they asked to assess students’ understanding of the reading material were lifted verbatim from the textbook. Teachers admitted that they felt they did not possess the necessary competencies to formulate HOT questions, that they were, as Hashim (2003) puts it, ‘not adept with the teaching strategies’ (p. 44). Wilson and Narasuman (2020) and Veloo et al. (2016) also found that the teachers who participated in their studies depended largely on reference books for higher-order thinking questions, either because they lacked confidence in their ability to devise such questions, or because of time constraints. Seman et al. (2017), as well, holds that teachers who do not have a basic understanding of HOTS are unable to design and implement effective methods of developing these skills.

An additional internal factor noted by one of the teachers in the present study was an inability to let go of traditional teaching methods that depend on teacher-centered instruction. Seif (2017), in this context, observes that some teachers prefer quiet and orderly classrooms, and that they tend to rely more on traditional methods based on memorization. Accordingly, they find it difficult to accept and apply the new changes in the curriculum. Unprepared for the reform, they would rather remain in their ‘comfort zone’, as Seman et al. (2017, p. 536) suggest. In sum, these three internal factors doubtlessly influenced the cognitive level of the participants’ questions in this study; they also may partially explain their tendency to avoid HOT questions while teaching reading comprehension.

Concerning the external factors, most of the teachers cited students’ low cognitive and proficiency levels, as well as their negative attitudes toward English, as obstacles to teaching HOTS, alleging that nearly half of their students were low achievers and unmotivated. Some, such as TL, also felt that language anxiety was partially to blame: ‘The English language is not easy for the students to acquire, not to mention teaching HOTS in English, which is a third language for Arab students.’ Dwee et al. (2016) corroborate the prevalence of such attitudes among students, noting that they ‘are not keen to share their thoughts and ideas . . . because of their low confidence in their English language ability and fear of being ridiculed by their peers’ (p. 637).

The problem, however, goes beyond low proficiency levels and negative attitudes toward learning English, as some of the students also suffer from low critical thinking levels. Thus, TC said, ‘I don’t usually relate to HOTS. But I will concentrate on them more with students who can internalize HOTS.’ This finding is consistent with prior work by Zohar and Dori (2003), and Hashim (2003) in which one of teachers was quoted as saying, ‘It is not easy to teach these students to think if they cannot even understand conversation and express themselves in class in the National Language’ (p. 43). Given impressions such as these, it is not surprising that some teachers have chosen to disregard HOTS in their classes.

The second external factor cited by participants in the present study was the dramatic changes in the curriculum and assessment methods initiated by the Ministry of Education over the last two decades. The teachers argued that introducing reforms that place additional burdens and responsibilities on their shoulders without providing them with adequate training, learning materials, and budgets results in inefficient implementation.

The third and final external factor mentioned was cultural norms. According to Abed and Dori (2013) and Seif (2017), Arab society in Israel is fundamentally conservative and the wisdom of elders is usually respected. As such, teachers persist in their traditional pedagogical role in which they ask the questions and the students answer with deference and respect. Students rarely ask questions during class time. At any rate, as virtually no work is done in terms of developing higher-order thinking skills at the elementary and middle school level, high school teachers and students alike find that they lack a solid foundation to start from in this regard. At the same time, parents in the Arab community, as some teachers emphasized, also have little prior knowledge of HOTS and thus find themselves at a loss in terms of how develop these skills with their children at home. In sum, all these cultural factors and more influence teachers’ educational awareness and attitudes towards HOTS.

Taking all the aforementioned factors into account, it can be concluded that the teachers in this study tended to use LOT questions more often for several reasons. First, LOT questions – as Hashim (2003), Shafeei et al. (2017) and Zohar and Dori (2003) claim – better suit the level of most of their students. This is because the answers to such questions can be more readily retrieved by all students, regardless of their cognitive level, from the reading text. Second, LOT questions are usually ‘display’ questions, which have one correct answer. Using such questions makes it easier for the students to answer and for the teacher to rapidly assess students’ comprehension. Third, as findings of previous studies affirm, LOT questions motivate students and encourage participation (Ellis, 1993; Fakeye & Ayede, 2013; Rachmawaty & Ariani, 2019).

Time constraints are another factor the participants cited with some frequency. The teachers complained that given a mere four English lessons a week, they can barely cover the basics, let alone devote sufficient time to HOTS. Seif (2017), as well, found that ‘tasks involving divergent thinking were not frequently used due to the curriculum pressure’ (p. 86). Ellis (1993), Fakeye and Ayede (2013) and Seman et al. (2017), likewise, observed that teachers deliberately relied on lower-level cognitive questions to keep lessons moving at a pace rapid enough to ensure that they had sufficient time to go over other things related to the curriculum.

A fifth reason noted for teachers’ reluctance to use higher-order thinking questions was their belief that most of the reading comprehension questions on the Bagrut exam are in fact LOT questions. Teaching classes with mixed cognitive levels, in which about half the students are low achievers, the teachers were more interested in helping all their students to pass the matriculation exam, and practicing LOT questions was regarded as more efficacious in this regard. Thus, it could be said that the teachers were more focused on bottom-line academic achievement than cultivating advanced thinking skills. Likewise, Hashim (2003) found that both teachers and school administrators were more concerned with examination results than with teaching students to be critical thinkers.

In conclusion, the findings summarized above highlight the participants’ perceptions and practices while teaching reading comprehension. Regrettably, however, they cannot be generalized due to certain limitations inherent to the study. First, the research sample included only 13 teachers, a convenience group that may or may not adequately represent the larger population of Arab high school EFL teachers in Israel. Second, the time researchers spent conducting observations in the classroom was extremely limited, meaning that their presence could very well have influenced both the teachers’ and the students’ behaviors during the reading comprehension lessons observed. Thus, the present findings only offer insights, albeit significant ones, into Arab Israeli EFL teachers’ perceptions with respect to HOTS’ infusion into the new English curriculum and their practices while teaching reading comprehension.

VI Conclusions and recommendations

Overall, the findings show that while the Arab Israeli EFL teachers who took part in the study have positive perceptions about HOTS, and acknowledge the importance of imbuing them while teaching English, their in-class practices with respect to the cognitive levels of their questions are out of synch with these perceptions. Several factors were found to affect the cognitive level of the teachers’ questions. These were divided into two categories: internal factors, which relate to the teachers’ readiness and qualification to teach HOTS, and external factors, which include the students, the educational system, and certain cultural norms.

In congruence with earlier studies (e.g. Ansori et al., 2019; Seif, 2017; Shafeei et al., 2017; Teemant et al., 2016), the findings also show that one of the most important reasons for the gap between teachers’ positive attitudes toward HOTS and the cognitive levels of their questions is a lack of knowledge and training with respect to HOTS. One way of addressing this issue is by offering teachers professional development courses that introduce them to HOTS. In this regard, Ansori et al. (2019), Seif (2017) and Teemant et al. (2016) have noted the positive changes in teachers’ HOTS-related perceptions and practices effected by in-service training courses.

Accordingly, it is clear that the Ministry of Education must assume greater responsibility by providing not only appropriate guidance but also professional development courses that reduce teachers’ fears and reluctances about HOTS while improving their teaching methods. Teachers as well must acknowledge that they, above all, are the ones responsible for promoting their students’ HOTS. Hence, they should be proactive, and make every effort to raise their awareness and develop their teaching methods, as only teachers who regard HOTS as important, and feel equipped to integrate them into their teaching, will be able to effectively improve their students’ thinking skills (Mustika, 2019; Utami et al., 2019) and their reading comprehension as a result. Likewise, teachers ought to consider a thoughtful balance of the cognitive level of the questions they ask by using a combination of lower-order and higher-order thinking questions in teaching not only reading comprehension but other skills as well. Moreover, as Ballou (2012) emphasizes, teachers should be ‘willing to devote planning and instructional time to the cause in order to see changes in their students’ standardized assessment scores’ (p. 68) and thinking skills in general.

Finally, further research on HOTS-related perceptions and practices among a wider population of Arab EFL teachers in Israel is recommended to confirm the reliability of the current findings. Conducting a comparative study in the Jewish sector may also be advisable. This undoubtedly would yield valuable new insights relevant to the integration of HOTS in English language education across the school system in Israel. Implications and recommendations, subsequently, would be submitted to the Ministry of Education concerning the kinds of professional development courses that might be offered to teachers.

Footnotes

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Ibtihal Assaly

References

Abed

Dori

(2013). Inquiry, chemistry understanding levels, and bilingual learning. Educación Química, 24(1), 37–43.

Aloqaili

A.S.

(2012). The relationship between reading comprehension and critical thinking: A theoretical study. Journal of King Saud University – Languages and Translation, 24, 35–41.

Amara

(2014). Policy and teaching English to Palestinian students in Israel: An ecological perspective to language education policies. In Gorter D., Zenotz V., & J. Cenoz (Eds.), Minority language and multilingual education (pp. 105–118). Springer.

Ansori

Nurkamto

(2019). Teacher’s beliefs and practices in the integration of higher order thinking skills in teaching reading. ELS Journal on Interdisciplinary Studies in Humanities, 2, 541–555.

Ardini

(2017). Teachers’ perception, knowledge and behavior of higher order thinking skills (HOTS). English Teaching Journal, 8, 21–33.

Assaly

I. R.

Smadi

(2015). Using Bloom’s taxonomy to evaluate the cognitive levels of Master Class textbook’s questions. English Language Teaching, 8, 100–110.

Ball

Garton

(2005). Modeling higher order thinking: The alignment between objectives, classroom discourse, and assessment. Journal of Agricultural Education, 46, 58–69.

Ballou

A.K.

(2012). Using explicit strategy instruction to improve reading comprehension. Master dissertation. Paper 221. Available at: https://fisherpub.sjfc.edu/education_ETD_masters/221 (accessed July 2021).

Bloom

Englehart

Furst

Hill

Krathwohl

(1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. Longman.

10.

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101.

11.

Celce-Murcia

(1991). Teaching English as a second or foreign language. Heinle & Heinle.

12.

Creswell

J.W.

(2014). Research design: Qualitative, quantitative, and mixed methods approaches. 4th edition. Sage.

13.

Durkin

(1993). Teaching them to read. Allyn and Bacon.

14.

Dwee

C.Y.

Anthonya

E.M.

Salleha

B.M.

Kamarulzamana

Kadir

Z.A.

(2016). Creating thinking classrooms perceptions and teaching practices of ESP practitioners. Social and Behavioral Sciences, 232, 631–639.

15.

Ellis

(1993). Teacher questioning behavior and student learning: What research says to teachers? Paper presented at 64th annual meeting of the Western States Communication Association, Albuquerque, New Mexico, USA. Available at: https://eric.ed.gov/?id=ED359572 (accessed July 2021).

16.

Enabulele

(2011). A Critical thinking in secondary language arts: Teacher’s perception and relevant strategies. Research report presented at the School of Education, Indiana University South Bend, USA. Available at: https://eric.ed.gov/?id=ED521706 (accessed July 2021).

17.

Ewing

Whittington

(2007). Types and cognitive levels of questions asked by professors during College of Agriculture class sessions. Journal of Agricultural Education, 48, 91–99. Available at: https://eric.ed.gov/?id=EJ840135 (accessed July 2021).

18.

Fahim

Sa’eepour

(2011). The impact of teaching critical thinking skills on reading comprehension of Iranian EFL learners. Journal of Language Teaching and Research, 2, 867–874.

19.

Fakeye

D.O.

Ayede

(2013). Teachers’ questioning behaviour and instructional organization as correlates of students’ achievement in English language. Global Journal of Human Social Science, 13, 13–19.

20.

Fischer

Bol

Pribesh

(2011). An investigation of higher-order thinking skills in smaller learning community social studies classrooms. American Secondary Education, 39, 5–26. Available at: http://www.jstor.org/stable/23100399 (accessed July 2021).

21.

Freahat

Smadi

(2014). Lower-order and higher-order reading questions in secondary and university level EFL textbooks in Jordan. Theory and Practice in Language Studies, 4, 1804–1813.

22.

Gallagher

Hipkins

Zohar

(2012). Positioning thinking within the national curriculum and assessment systems: Perspectives from Israel, New Zealand and Northern Ireland. Thinking Skills and Creativity, 7, 134–143.

23.

Grellet

(1981). Developing reading skills. Cambridge University Press.

24.

Hashim

(2003). Malaysian teachers’ attitudes, competency and practices in the teaching of thinking. Intellectual Discourse, 11, 27–50. Available at: https://journals.iium.edu.my/intdiscourse/index.php/id/article/view/245 (accessed July 2021).

25.

Hult

(2010). The history and development of educational linguistics. In Spolsky

Hult

(Eds.), The handbook of educational linguistics (pp. 10–24). John Wiley & Sons.

26.

Krathwohl

(2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41, 212–218.

27.

Kucer

(2009). Dimensions of literacy: A conceptual base for teaching reading and writing in school settings. 3rd edition. Erlbaum.

28.

Larson

Lovelace

(2013). Evaluating the efficacy of questioning strategies in lecture-based classroom environments: Are we asking the right questions? Journal of Excellence in College Teaching, 24, 105–122.

29.

Lateef

Dahar

Latif

Kh.

(2016). Impact of higher order thinking skills of university students on their academic performance. Pakistan Association of Anthropology, 28, 2031–2035.

30.

Lee

Lai

(2017). Facilitating higher-order thinking with the flipped classroom model: A student teacher’s experience in a Hong Kong secondary school. Research and Practice in Technology Enhanced Learning, 12, 8.

31.

Lewis

(2007). Developing questioning skills. The University of Texas at Austin. Available at: https://www1.udel.edu/chem/white/U460/Devel-question-skills-UTx.pdf (accessed July 2021).

32.

Lightbown

Spada

(2006). How languages are learned. 3rd edition. Oxford University Press.

33.

Marzano

(2003). Language, the language arts, and thinking. In Flood

Lapp

Squire

J.R.

Jensen

J.M.

(Eds.), Handbook of research on teaching the English language arts (pp. 687–716). 2nd edition. Lawrence Erlbaum.

34.

Mayer

(2003). Learning and instruction. Pearson Education.

35.

Mustika

Nurkamto

Azizah

(2019). Exploring English teachers’ perception towards higher order thinking skills (HOTS) in the 21st century learning. Proceedings of the International Conference on Future of Education, 2, 46–51.

36.

Nourdad

Masoudi

Rahimali

(2018). The effect of higher order thinking skill instruction on EFL reading ability. International Journal of Applied Linguistics and English Literature, 7, 231–237.

37.

Rachmawaty

Ariani

(2019). Investigating the types of teacher questions in EFL secondary classroom. Advances in Social Science, Education and Humanities Research, 276, 40–49.

38.

Remark

Ewing

(2015). Use of high-level questioning to increase student achievement in reading. Masters action research paper. Available at: https://sophia.stkate.edu/maed/127 (accessed July 2021).

39.

Seif

(2017). Developing higher order thinking skills of Arab high school students in Israel. Doctoral thesis, University of Derby, Derby, UK. Available at: http://derby.openrepository.com/derby/handle/10545/621587 (accessed July 2021).

40.

Seman

Yusoff

Embong

(2017). Teachers challenges in teaching and learning for higher order thinking skills (HOTS) in Primary School. International Journal of Asian Social Science, 7, 534–545. Available at: https://EconPapers.repec.org/RePEc:asi:ijoass:2017:p:534-545 (accessed July 2021).

41.

Seth

(2013). Enabling adolescents to build life skills: Part I: Understanding concepts . . . Evolving strategies. United Nations Population Fund (UNFPA).

42.

Shafeei

Hassan

Ismail

Azian

(2017). Incorporating higher order thinking skills (HOTS) questions in ESL classroom contexts. LSP International Journal, 4, 101–116. Available at: https://journals.utm.my/lsp/article/view/15215 (accessed July 2021).

43.

Sidek

(2010). An analysis of the EFL secondary reading curriculum in Malaysia: Approaches to reading preparation for higher education. Unpublished doctoral thesis, University of Pittsburgh, PA, USA. Available at: http://d-scholarship.pitt.edu/id/eprint/9837 (accessed July 2021).

44.

Snow

(2002). Reading for understanding: Toward an R & D program in reading comprehension. RAND.

45.

Spolsky

(2010). What is educational linguistics? In Spolsky

Hult

(Eds.), The handbook of educational linguistics (pp. 1–9). John Wiley & Sons.

46.

Surjosuseno

Watts

(1999). Using Bloom’s taxonomy to teach critical reading in English as a foreign language classes. Queensland Journal of Educational Research, 15, 227–244. Available at: https://www.iier.org.au/qjer/qjer15/surjosuseno.html (accessed July 2021).

47.

Teemant

Hausman

Kigamwa

(2016). The effects of higher order thinking on student achievement and English proficiency. INTESOL Journal, 13.

48.

Tofade

Elsner

Haines

S.T.

(2013). Best practice strategies for effective use of questions as a teaching tool. American Journal of Pharmaceutical Education, 77, 155.

49.

Ulmer

Torres

(2007). A comparison of the cognitive behaviors exhibited by secondary agriculture and science teachers. Journal of Agricultural Education, 48, 106–116.

50.

(1996). A course in language teaching practice and theory. Cambridge University Press.

51.

Utami

Nurkamto

Marmanto

Taopan

(2019). The implementation of higher-order thinking skills in EFL classroom: Teachers’ perceptions. Proceeding of the 2nd International Conference on Future of Education, 2, 64–72.

52.

Veloo

Ramli

Khalid

(2016). Assessment practices among English teachers in Malaysian secondary schools. International Journal for Infonomics, 9, 1220–1227.

53.

William

(1984). Reading in the language classroom. MacMillan.

54.

Wilson

D.M.

Narasuman

(2020). Investigating teachers’ implementation and strategies on higher order thinking skills (HOTS) in school-based assessment instruments. Asian Journal of University Education, 16, 70–84.

55.

Zohar

(2008). Teaching thinking on a national scale: Israel’s pedagogical horizons. Thinking Skills and Creativity, 3, 77–81.

56.

Zohar

(2013). Challenges in wide scale implementation efforts to foster higher order thinking (HOT) in science education across a whole school system. Thinking Skills and Creativity, 10, 233–249.

57.

Zohar

Dori

(2003). Higher-order thinking skills and low-achieving students: Are they mutually exclusive? Journal of the Learning Sciences, 12, 145–181.

58.

Zulkpli

Abdullah

A.H.

Atan

N.A.

(2017). The Rasch analysis of mathematics teachers’ practices on teaching thinking skills. Man in India, 97, 141–152.