Abstract
Higher education has been transformed by technology over the past two decades, and teaching and learning online has become considerably more prominent. We examine the extent to which journals in the discipline mirror the rising use of technology in the classroom in the frequency of articles they publish on teaching and learning with technology, with a major focus on online teaching. We find a relatively small share of articles focus on pedagogical technologies, including online teaching, and their impact on student learning. Our findings indicate that in light of the rapid and broad adoption of technology by college teachers, there is much opportunity to expand research on how large numbers of political science students learn in technology-infused classrooms, including online instruction.
Introduction
In the last two decades Technology Enhanced Learning (TEL) has transformed higher education in the United States, the United Kingdom, and in many other countries most notably through the rise of online modalities: completely online, mixed mode, 1 flipped, 2 and Massive Open Online Courses (MOOCs). It has also transformed the traditional face-to-face classroom, where contemporary students frequently use personal response systems (‘clickers’) to provide instant feedback, consult online textbooks and other reading materials, participate in computer-enhanced simulations, and use social media to communicate with their peers. Technology has also expanded student access to academic staff beyond the limits of class time and office hours to include email, Skype, WhatsApp, and other communication tools such as Twitter, Instagram, and Facebook. Electronic course management tools have helped change how academic staff members administer their courses and grade books.
While some of these technologies are enhancements to traditional face-to-face classroom instruction and are additions to, or perhaps replacements for, overhead projectors, note cards, and email listservs, other types of technology have fundamentally affected how college staffs teach, and students learn. The online classroom stands out as an example. To illustrate, in the United States, the steady growth in the institutional adoption of classroom technologies – particularly those based on online modalities – in higher education is well documented (Allen and Seaman, 2013; 2014). Here, in the 10-year period between 2002 and 2012, the percentage of students enrolled in at least one online course more than tripled, from 9.6% to 33.5% (Allen and Seaman, 2014: 33). Popular perceptions aside, this growth has not been fuelled by the rise of for-profit online colleges. To be sure, 64% of private for-profit institutions have online courses, but this number pales when compared with private non-profit (82%) and public (99%) colleges and universities (Allen and Seaman, 2014: 32). Beyond the raw enrolment numbers lies a pattern of institutional investment in online education; the percentage of U.S. higher education institutions offering online courses plus complete degree programmes increased from 34.5% in 2002 to 62.4% in 2012 (Allen and Seaman, 2013: 20). Distance learning is also widespread in the United Kingdom, and popular degrees are listed on websites such as ‘10 Top UK Universities for Distance Learning’ (Distancelearningportal, 2012).
Perhaps unsurprisingly, American university administrators like the online enrolment model and are unusually sanguine about the pedagogical value of Internet-based teaching. In 2013, over 80% of 2831 chief academic officers assessed online instruction as ‘the same as’ or ‘superior to’ face-to-face instruction, and nearly 70% agreed that ‘online education is critical to the long-term strategy of my institution’, compared with about 50% in 2002 (Allen and Seaman, 2014: 8, 11). A U.S. Department of Education meta-study concludes that ‘classes with online learning (whether taught completely online or blended) on average produce stronger student learning outcomes than do classes with solely face-to-face instruction’ (Means et al., 2010: 18). Clearly, online learning is here to stay. This trend likely extends to political science. A recent symposium entitled ‘The Troubled Future of Colleges and Universities’ in PS: Political Science & Politics (PS) (King and Sen, 2013a: 81) identifies online learning as one of the ‘possibly threatening’ innovations higher education has to face; King and Sen (2013b: 85) identify the Internet and distance learning as two of the ‘attacks’ of the current business model of higher education in the United States. Political scientists are aware of the changing nature of higher education, which also affects political science. To add to the complexity of online education, the proliferation of MOOCs has extended to political science as well, as the long list of available MOOCs in the discipline demonstrates. 3 Several colleges have begun to offer college credit for students completing MOOCs. 4
Given the rapid rise in the use of technology in classrooms in institutions in higher learning, particularly in online education, the question becomes, what do we know about the efficacy of this pedagogy on learning outcomes, compared with traditional face-to-face instruction? In other words, do we know how teaching and learning in political science is affected by the digital revolution, reflected by the widespread use of technology in the classroom in general, and online instruction in particular? More generally, how does the scholarship of teaching and learning (SoTL) in political science compare with SoTL within the larger multi-disciplinary universe of higher education? When it comes to assessing the impact of technology in the classroom, is political science keeping pace with other fields? For example, are political scientists studying and evaluating the use of the same technologies in the classroom as other disciplines?
Previously we looked at the extent to which the rise of technology in political science is reflected in teaching-oriented articles in the early stages of online education (Hamann et al., 2014). Our analysis of over 400 articles, all of which were published between 2005–2013 in three major political science outlets that publish regularly, or even solely, on teaching, showed that only about 15% addressed emerging and established technologies – indicating that the large majority of pedagogical studies are not concerned with the increased presence of technology.
This study extends and contextualizes our earlier work in three ways. First, we return to the previous dataset (updated through 2014) and determine whether technology-focused articles only describe the technology (technology-focused/non-SoTL) or also assess the effect of the technology on student outcomes (technology-focused/SoTL). In other words, are political scientists bringing social science methodologies to bear on the question of technological efficacy? Second, we develop a parallel dataset, based on articles published in two major multi-disciplinary, teaching-oriented journals for the years between 2005 and 2014. To what extent has college teaching, writ large, been concerned with technology – with an evaluation of its effects on student learning? Lastly, we compare the trends we identify in our study against a broader set of journals in the discipline.
The study
To determine the extent to which political science teaching research has kept pace with – and has assessed the impacts of – new teaching technologies, we examined the abstracts (and, when necessary, the articles themselves) of 481 entries in three journals in the discipline that regularly – and in the case of Journal of Political Science Education (JPSE), exclusively – publish articles on teaching and learning: International Studies Perspectives (ISP), PS, and the JPSE. While we conduct an in-depth study of these journals, we also provide additional data on a wider variety of journals; we find that the trends uncovered in these three journals are quite typical for the political science/international relations and related disciplines on both sides of the Atlantic. We included the years between 2005, the year that JPSE was first published, and 2014. We identified all articles that discussed, in any way, the use of technologies in teaching political science. We also coded each technology article as non-SoTL or SoTL. Additionally, we recorded the type of technology in question as follows: online teaching, mixed mode instruction, clickers (student response systems), online simulations (including games), the use of Internet sources, teaching with multimedia, software, or podcasting. Data from the three journals were merged.
To see where research on the use of technology in political science fits in the larger context of technology in higher education, we coded a total of 400 articles from two of the most influential (as measured by impact factor) international higher education teaching journals with a technology focus: Computers and Education (CE) and the International Journal of Computer-supported Collaborate Learning (IJCSCL). According to its website, CE (5-year impact factor, 3.24) welcomes papers on ‘cognition, educational or training systems development using techniques from and applications in any technical knowledge domain’. IJCSCL (2.61), ‘promote[s] a deeper understanding of the nature, theory, and practice of the uses of computer-supported collaborative learning’.
CE is a truly prodigious resource for teaching scholarship with a focus on instructional technology. During the period under study (2005–2014), CE published 1929 articles, with yearly increases from around 50–100 per year in 2005–2007 to over 200 per year in 2008–2014. We coded every article in the 2005–2007 issues, and then drew and coded random samples (n = 40) for each even-numbered year, 2008–2014. (The data were weighted to correct for the over-representation of the 2005–2007 issues.) For IJCSCL, we coded all articles in each even-numbered year between 2006 and 2014. (IJCSCL’s inaugural year was 2006.) The combined CE/IJCSCL dataset has a weighted-n of 400, 278 from CE and 122 from IJCSCL. As with the political science journals, we coded for technological content and SoTL methodology. Our main goal here is to compare the combined political science dataset with the combined CE/IJCSCL dataset. (Note: Hereafter, in the text we refer to the CE/IJCSCL dataset as the ‘CE articles’ or ‘CE dataset’.) In the analyses that follow, we first describe differences in technology orientation and (more interestingly) trends in the focus on technology. Next, we consider the application of SoTL methodologies. We then turn to the types of technologies scholars have studied, and we try to discern trends.
Findings
Considering our selection of higher education journals that focus on computers, it comes as no surprise that the CE articles are more likely to address technology than articles in the political science journals (Table 1). In the CE dataset, 301 articles (75.3%) deal in some fashion with teaching technology compared with 70 articles (14.6%) in the political science journals. Yet, in our sample, none of the articles discussed politics courses. The trend data (Table 2) are a bit more interesting, if not intriguing. Notice the way those 301 CE technology articles are distributed over the data stream: from a high of 87.5% in 2005 (35 tech articles out of 40 total cases) to a low of 65.5% in 2014 (36 of 55 cases) – a 20 percentage-point decline. As depicted graphically (Figure 1), one can see that the saw-tooth pattern of 2005–2008 becomes a steady erosion during 2010–2014. Political science has seen the opposite trend: a general increase, at least post-2008, in a focus on technology. Of course, we are dealing with only 70 cases here, so inferences can be treacherous. Even so, it bears noting that political science bested its overall mean of 14.6% in four of the five most recent years, 2010–2014. (The exception is 2011.)
Are journal articles focused on teaching with technology?
CE: Computers and Education; IJCSCL: International Journal of Computer-supported Collaborate Learning.
Chi-square = 330.043.
p = .000.
Are journal articles focused on teaching with technology?: by journal group and year.
CE: Computers and Education; IJCSCL: International Journal of Computer-supported Collaborate Learning.
Technology focus, by journal group and year.
Table 3 shows similar usage of SoTL techniques between tech articles in the two datasets. Comparing the 300 CE articles with the 69 political science cases (each dataset lost one case to missing data), we see rough parity: 68% of the CE cases and 58% of the political science articles used SoTL standards for assessing the efficacy of teaching technologies. Although substantively noteworthy, this difference is not statistically meaningful (chi-square = 2.519, p = .112). Further analysis revealed no overtime trends (analysis not shown). Thus, political science’s research interest in technology is relatively weak but growing; and its rigor in assessing technologies for their instructional efficacy is relatively strong.
Do tech articles use SoTL methodologies?
CE: Computers and Education; IJCSCL: International Journal of Computer-supported Collaborate Learning.
Chi-square = 2.519.
p = .112.
Which technologies are favoured subjects for investigation? Table 4 reveals commonalities and differences between political science and higher education in general. In both datasets, online instruction received the most attention. Combining online and mixed-mode modalities, about one-third of political science articles and over 40% of CE articles reside in these cells of Table 4. Yet, there are reasons that the two columns of numbers are significantly different (chi-square = 31.923, p = .000). Note especially the comparatively large percentage of political science articles focused on simulations (21.4% vs 8.4% of the CE cases). Political science academic staff use simulations to demonstrate the complexity of political decision making in areas such as international relations, budget processes, and so on, and increasingly these simulations are completed in the online environment or using simulation software. (See, for instance, the many articles on simulations in ISP or the special issue on simulations in JPSE, volume 10, issue 1, 2014). On the other side of the ledger, political researchers have been less interested in multimedia (7.1% vs 14.8%) or teaching-related software (7.1% vs 16.8%).
Journal group and technology focus.
CE: Computers and Education; IJCSCL: International Journal of Computer-supported Collaborate Learning.
Chi-square = 31.923.
p = .000.
Attempts to investigate trends are a bit of a challenge: with multiple time points and multiple tech-types, the numbers of cases thin out rapidly. To gain some leverage on this interesting question – has the technology du jour shifted over time? – we divided the datasets into two periods, ‘earlier’ (2005–2007) and ‘later’ (2012–2014). We then reran the Table 4 analysis separately for each time period. (No statistical tests were conducted in this analysis.) Table 5 reports the results. Not surprisingly, as online instruction evolved from the novel to the mundane, interest in studying it waned, dropping from 45.6% to 28.7% among CE articles and from 31.6% to 19.4% in political science. Personal response systems (‘clickers’), a technology virtually unheard of before 2005, gained increased attention, as did simulations. (As we saw in Table 4, political science has become especially enamoured of this pedagogical tool, which seems to be particularly prominent in international relations classrooms.) Predictably, studies of social media increased in both datasets. Other apparent trends are less easily tied to changing technological tastes; multimedia, a staple of face-to-face and online instruction, gained among CE scholars (14.4% to 21.3% percent of articles) but stagnated in political science (10.5% to 6.5%). And instruction software, commonly featured in the CE data (although dropping between periods), began to appear in the latter day literature of political science.
Technology focus by journal group, 2005–2007 and 2012–2014.
CE: Computers and Education; IJCSCL: International Journal of Computer-supported Collaborate Learning.
To place our findings into a wider context, we also examined patterns in other political science journals as well as in a related discipline, sociology. Analysing articles published in the U.S. flagship pedagogical journal in sociology, Teaching Sociology, we identified similar trends as in political science. Between 2005 and 2014, we counted 20 articles that addressed questions related to instructional technology; about half of those focused on online teaching, such as discussion postings, video streaming, and so on. Other political journals that regularly publish articles on teaching follow roughly the same pattern. The U.K. journal Politics, for example, featured some articles on the use of video and of blogging/social media between 2005 and 2014 but no articles that were centrally focused on online teaching. Another journal, European Political Science, also regularly includes articles on teaching and learning. The journal published eight articles that focused on online teaching (or aspects of online teaching) between 2005 and 2014. We also checked the 2015 international political education database (IPED) database for ‘Technology Enhance Learning Articles’ for additional articles on online teaching and learning. This database lists a sizeable body of literature on teaching with technology, but when we excluded the journals that are already included in our study and focused on online instruction in politics, we found only occasional additional articles across the journals. The journals carrying these articles are as varied as Enhanced Learning in Social Sciences (taking the lead with six articles), Distance Education, Learning and Teaching in the Social Sciences (LATISS), Behavioral and Social Science Librarian, Critical Studies on Terrorism, Simulation and Gaming, Teaching Public Administration, Social Science Computer Review, Politics & Policy, Zeitschrift für Internationale Beziehungen, and Journal of Chinese Politics. These findings are echoed by Blake and Morse (2015), who conducted a content analysis of four journals (in addition to JPSE, they looked at Teaching of Psychology, Teaching Sociology, and the Journal of Information Technology Education (JITE)) to identify articles on teaching with technology that contain some aspect of assessment. Their findings indicate that 17% of articles published in Teaching of Psychology, 5.5% of articles in Teaching Sociology, and 62% of articles that appeared in JITE meet these criteria. The trends we found in the journals that formed the core of our analysis are thus echoed by a review of the publications in a broader set of journals.
Discussion and conclusion
Instructional technology has become pervasive in political science classrooms and has transformed the way many college students are taught and learn. SoTL has begun to assess the effects of technology on student learning. However, despite the rapid increase in the use of technology in general, and online teaching in particular, over the last decade or so, articles on teaching and learning published in the prominent journals on teaching and learning in politics and international relations continue to focus overwhelmingly on traditional, face-to-face classroom settings. Less than 15% focus on the use of classroom technology. However, about one-third of those articles are concerned with teaching in the online environment (including mixed mode), which has arguably had a bigger impact on higher education than other technologies integrated into face-to-face settings. It is also notable that the share of articles published in political science journals concerned with instructional technology has increased over time as the use of technology in the classroom has become more prominent. Thus, political scientists interested in learning about the effects of the use of instructional technologies in their classrooms can turn to research that may be helpful in designing courses and assignments to enhance student learning. Interestingly, prominent multi-disciplinary journals focusing on instructional technology contain few articles analysing the use of technology in political science. While political science academic staff can no doubt learn much from the experiences in other disciplines, they are nonetheless well advised to turn to disciplinary journals that publish pedagogical articles if they are interested in the use of instructional technology. Politics teachers appear to publish primarily in discipline-specific journals rather than in journals focused on instructional technology, perhaps because it is easier to get credit for publications in one’s substantive field. Or perhaps politics instructors publish rarely in technology-focused pedagogical journals because overall, they are not as interested in conducting research on the effects of the use of technology in their classrooms.
We can only speculate why political scientists appear to be less interested in conducting research on teaching and learning with technology than on other pedagogical issues, such as simulations, for example. Hamann et al. (2009) show that much of the SoTL research in Political Science, in the three journals that form the core of the analysis here (PS, JPSE, ISP), is conducted by tenure-earning teaching academics; oftentimes, those starting out their careers are younger, are socialized to the online environment and therefore perhaps find teaching and learning online less of an innovation that merits analysis. Certainly, future research should explore the reasons why online education is a relatively understudied subject in the discipline-specific pedagogical literature; a survey study, for instance, might provide answers that go beyond some initial speculation.
Our findings indicate a gap between the higher education experience of a growing number of college students, who increasingly learn in technology-infused classrooms including entirely online, and what the SoTL in the discipline focuses on. However, as these technologies have become more pervasive in higher education, we increasingly know more about how to use these technologies to enhance student learning and improve our teaching. As this area of research continues to grow, we learn more about best practices in utilizing technologies in Political Science classrooms and about how student learning in the discipline is affected by the introduction of these technologies. As the research on these technologies develops, existing studies can help frame future research agendas.
Footnotes
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.