An Evaluation of Production Versus Error-Recognition Techniques for Teaching APA-Style Citations and References

Abstract

American Psychological Association (APA)-style writing is an essential part of the undergraduate psychology curriculum. However, little is known about the effectiveness of specific techniques for instructing students on how to format APA-style citations and references. The current research compared the effectiveness of having students (N = 76) produce versus recognize errors in APA-style citations and references. Production of materials led to significantly better performance on both an immediate quiz and a later test. Student evaluations (N = 43) of the activities showed that they elicited similar enjoyment and effort, but most students preferred the error-recognition activity despite indicating that the production activity would lead to more learning. These results indicate that initial instruction of APA style should emphasize the production of correctly formatted APA-style materials.

Keywords

college teaching writing skills psychology education APA style

Professional standards in psychology dictate that undergraduates should develop both general writing skills and the ability to write using American Psychological Association (APA, 2011, 2013) style. However, APA style is challenging because it requires students to scrupulously follow an expansive set of rules that are not inherently interesting, that frequently lack intuitive logic, and that often contradict style guidelines learned in composition courses. Research shows that teachers perceive students’ APA-style performance as falling significantly below their expectations (Landrum, 2013). Despite teachers’ instructional efforts, students frequently fail to recognize basic mistakes in APA style, and this is especially true for the formatting of citations and references (Mandernach, Zafonte, & Taylor, 2016; Van Note Chism & Weerakoon, 2012). Although APA style can be challenging to teach and to learn, one cause of difficulty may be the use of ineffective instructional methods. Thus, the purpose of this study was to examine the relative effectiveness of requiring students to produce versus recognize errors in citations and references when they are initially learning APA style.

Students may struggle to perfect their writing, but there is no doubt that they significantly benefit from APA-style instruction. Research has consistently shown that courses and stand-alone modules on APA style improve students’ performance both over time and in comparison to control groups (Fallahi, Wooed, Austad, & Fallahi, 2006; Froese, Boswell, Gracia, Koehn, & Nelson, 1995; Jorgensen & Marek, 2013; Luttrell, Bufkin, Eastman, & Miller, 2010; Smith & Eggleston, 2001; Zafonte & Parks-Stamm, 2016). However, APA-style courses and modules typically involve the use of many instructional techniques, and it is not clear what methods are most beneficial. A few researchers have examined the effects of specific teaching methods (Franz & Spitzer, 2006; Slezak & Faas, 2017), but strong evidence for the superiority of one method has not emerged. As such, further research is needed to evaluate the effectiveness of specific techniques for instructing students to format citations and references in APA style.

One technique that should be evaluated is having students identify APA-style mistakes. The technique of providing students with improperly formatted materials so that they can search for errors has made its way into recommendations for teaching APA style (Mandernach et al., 2016), and teachers frequently implement the technique in APA-style courses and modules (Jorgensen & Marek, 2013; Smith & Eggleston, 2001; Van Note Chism & Weerakoon, 2012; Zafonte & Parks-Stamm, 2016). There is sound logic behind asking students to identify errors. Students cannot spot an error unless they know the rules. Also, proofreading for format errors is an essential skill that students need when editing their own writing. Nonetheless, there may be more effective methods for the initial instruction of APA style.

Basic research on learning and memory has direct implications for how college students learn (Putnam, Sungkhasettee, & Roediger, 2016), and there are several research-based reasons to believe that production of APA-style citations and references should be more effective than recognition of APA-style errors. According to research on the generation effect, people remember more when they produce the answer to a question than when the answer is provided to them (Bertsch, Pesta, Wiscott, & McDaniel, 2007; Foos, Mora, & Tkacz, 1994). A similar concept is the testing effect in which practice recalling information leads to better retention than simply reviewing the information (Dunlosky & Rawson, 2015; Roediger & Karpicke, 2006). Both the generation and testing effect imply that having students produce APA-style citations and references could lead to better memory for formatting rules than having them identify errors.

Fluency may be another factor in student learning of APA style. The subjective ease of encoding material into memory does not predict later recall; sometimes difficult tasks lead to more learning (Bjork, Dunlosky, & Kornell, 2013). Producing APA-style citations and references is hard, and recognizing APA-style errors is easy, especially considering that students may be unaware of the high proportion of errors that they miss (Van Note Chism & Weerakoon, 2012). Thus, the fluency of recognizing errors in APA-style materials, relative to producing APA-style materials, may lead students to initially learn less about APA style and subsequently believe that they do not have to work hard to implement APA-style rules. Despite the implications of basic research for the instruction of APA style, classroom research is also needed, and the purpose of the current study was to compare production versus error-recognition as methods for learning to format APA-style citation and references.

This study evaluated the effects of an APA-style course module on students’ quiz and test performance. A randomly assigned activity in one class period required students to either produce APA-style citations and references or recognize errors in APA-style citations and references. In the subsequent class period, the groups switched and completed the other activity. A quiz assessed students’ ability to produce APA-style citations and references after each activity, and a test assessed their ability to recognize and produce correctly formatted citations and references at the end of the module. In addition, students reported their perceptions of the production and recognition activities. These methods allowed for the investigation of the following research questions:

How does producing APA-style materials versus recognizing errors in APA style affect students’ ability to apply APA style?

How do students perceive activities that emphasize production of APA-style materials versus recognizing errors in APA style?

Answers to these questions will help teachers in the difficult task of instructing students to write using APA style.

Method

Participants

Participants consisted of students enrolled in three consecutive semesters of a one-credit course designed as an introduction to the psychology major for students who had completed introduction to psychology. Although there were 130 students enrolled across the three course sections, absences and incomplete data led to a final usable sample of 119 students. Part 1 of the study examined the effectiveness of the instructional methods and included 76 students across two course sections offered in consecutive semesters. Part 2 of the study included 43 students from the next semester of the course who reported their perceptions of the instructional methods. The participants were primarily females (75%) of traditional college age in their first or second year of study at a medium-sized, private university in the Midwest. Students provided informed consent at the start of the semester, but the procedures occurred as part of regular classroom activities with no inducement for participation.

Materials and Procedure

Part 1: Effectiveness of the instructional methods

The purpose of Part 1 of the study was to examine the effectiveness of instructional methods emphasizing production of APA-style materials versus recognition of APA-style errors. The APA-style module occurred over two 50-min class periods. On the first day, random assignment placed students into either the production-first group or the recognition-first group. After a 10-min lecture on the basics of APA-style citations and references, students received worksheets containing images of six sources (i.e., books and journal articles) that contained the information necessary for an APA-style reference entry (see Figure 1). In the production-first group, the worksheets contained no other information, and the instructions asked students to produce an APA-style citation and reference for each of the six sources. They could use the APA manual or any other tool, but they could not collaborate.

Figure 1.

The production activity (A) required students to create full American Psychological Association (APA)-style citations and references using an image of a journal article. The quiz and test questions use the same format. The recognition activity (B) required students to circle all APA-style mistakes in examples of citations and references.

The procedure for the recognition-first group was identical except that, rather than images of sources, the worksheet contained APA-style citations and references with formatting mistakes, and the instructions asked students to circle the errors (see Figure 1). The citations and references were for the same six sources included in the production-first worksheet. Each source contained five to seven errors in APA style. After 15 min of individual work on the task, students collaborated for 5 min on producing the correct answers with a partner in the same experimental group. Then, there was 5 min of large-group discussion where students could ask questions about the correct answers or other formatting issues. Because all worksheets contained the same six sources, discussion of correct answers was equally relevant to both groups. Finally, students individually completed a practice quiz.

On the second day of the module, the initial 10-min lecture emphasized common citation and reference mistakes. Then, the same procedures occurred except students who were in the production-first group during the first class switched to the recognition worksheet and students who were in the recognition-first group switched to the production worksheet. All worksheets were matched for type of source (e.g., books vs. articles), order of sources, and complexity of source characteristics (e.g., number of authors). Thus, at the end of the module, students had completed the same activities in counterbalanced order and taken two quizzes.

The open-book practice quizzes provided students with an image of a journal article and required them to produce an APA-style citation and reference entry using the article. Because the quizzes did not contribute points toward the course grade, scoring occurred by counting APA-style errors. Citations and references that correctly followed every APA-style rule received scores of 0 for having no errors. Examples of errors included missing information (e.g., no volume number, no doi), incorrect information (e.g., wrong journal name, authors out of order), and incorrect formatting (e.g., missing italics, incorrect punctuation). All unique mistakes led to the addition of one point, but multiple instances of the same mistake counted only once. For example, referencing this journal as “Teaching Of Psychology” would count as two errors for lack of italics and incorrect capitalization, and referencing “Scholarship Of Teaching And Learning In Psychology” would also count as two errors because it contained the same two mistakes, albeit multiple instances of the capitalization mistake. Due to the limited information available in the image of the journal article, the content of the sentences did not affect scores. Most students produced simple sentences that paraphrased the titles of articles.

After the APA-style module of the course, students completed an open-book, online test worth 25 points, which represented one fifth of the course grade. The test included 10 multiple-choice items worth one point each that required students to recognize correctly formatted components of APA-style citations and references. The test also included 3 items consisting of images of journal articles that required students to produce either a citation or a reference entry. The 3 items were worth five points each, and scoring occurred such that each unique error in APA style led to a subtraction of one of the five points. I scored all materials blind to experimental condition.

Part 2: Student evaluations of the instructional methods

After investigating the relative effectiveness of the two instructional methods in Part 1 of the study, Part 2 of the study was necessary to determine students’ perceptions of the methods. Students in a third section of the course completed a brief evaluation survey while engaging in the production and recognition activities. Students completed condensed versions of the activities in counterbalanced order over one 50-min class period. After completing each activity, students evaluated it by rating 3 items on a 7-point scale from strongly disagree to strongly agree with the midpoint being neither disagree nor agree. The items asked whether each activity “helped me learn APA style,” “was easy,” and “was enjoyable.” A fourth item asked students to rate their agreement that they “tried their hardest to get the right answers.” After completing both activities, 2 items asked students to indicate which one would lead to more learning and which one they preferred.

Results

The purpose of the first analysis was to examine students’ performances on quizzes that followed activities that required them to produce APA-style materials and recognize APA-style errors. A 2 (quiz) × 2 (experimental group) mixed analysis of variance compared scores on Quiz 1 and Quiz 2 between the two experimental groups: The production-first group completed the production activity before Quiz 1 and then switched to the recognition activity before Quiz 2, and the recognition-first group completed the recognition activity before Quiz 1 and then switched to production activity before Quiz 2. The within-subjectseffect, between-subjects effect, and interaction were all significant, all Fs > 4.39, all ps < .041, all $η_{p}^{2}$ > .07. Post hoc independent-samples t tests showed that on Quiz 1, the production-first group (M = 5.40, SD = 3.00) had significantly fewer errors than the recognition-first group (M = 7.90, SD = 3.68), t(59) = 2.91, p = .005, d = 0.74. These results show that performance was superior among students who produced APA-style citations and references. On Quiz 2, which followed the groups switching activities, no significant difference emerged between the recognition-first (M = 4.23, SD = 2.66) and the production-first groups (M = 5.03, SD = 3.80), t(59) = 0.95, p = .347, d = 0.24. These results show that the difference in performance was reduced after both groups had completed the production activity. Post hoc paired-samples t tests showed significant improvement from Quiz 1 to Quiz 2 for both the production-first group, t(29) = 2.25, p = .032, d = 0.42, and the recognition-first group, t(30) = 4.61, p < .001, d = 0.78. Examination of the means and effect sizes illustrates that there was more improvement in the group that switched from the recognition activity to the production activity than for the group that started off with the production activity.

The next analysis compared the experimental groups’ performance on an APA-style test administered after the completion of both activities. An independent-samples t test showed that the production-first group (M = 75%, SD = 13) scored significantly higher on the test than the recognition-first group (M = 63%, SD = 26), t(66) = 2.45, p = .018, d = 0.57. These results show that the performance advantage shown by the production-first group on the quizzes extended to a graded assignment completed after all students had completed both the production and recognition activities.

The final analysis examined students’ evaluations of the activities. Paired-samples t tests compared students’ ratings of each activity (see Table 1 for means, standard deviations, and t test results). The students slightly to somewhat agreed that they learned from the activities and tried hard to complete them. Students were neutral in their evaluations of the activities’ easiness and enjoyableness. Although it did not reach the conventional cutoff for significance, students did evaluate the recognition activity as easier than the production activity. Students also selected which activities they thought they would learn more from and which they preferred. Examination of the frequencies showed that 67% of students indicated that they would learn more from producing their own citations and references than from recognizing errors, but only 46% indicated that they preferred the production activity over the recognition activity.

Table 1.

Students’ Perceptions of Class Activities Requiring the Production of American Psychological Association (APA)-Style Materials and Recognition of APA-Style Errors.

Item	Production	Recognition	t	p	d
Learning	5.65 (1.13)	5.51 (1.03)	0.78	.445	0.13
Easy	3.86 (1.30)	4.26 (1.33)	1.90	.064	0.30
Enjoyable	4.19 (1.16)	4.37 (1.41)	0.94	.352	0.14
Tried hard	6.07 (1.33)	6.09 (1.57)	0.16	.875	0.16

Note. Ratings of agreement occurred on a scale from 1 (strongly disagree) to 7 (strongly agree) with 4 being neutral.

Discussion

The current study examined two research questions. The first research question was, how does producing APA-style materials versus recognizing errors in APA style affect students’ ability to apply APA style? The results indicated that requiring students to produce APA-style materials lead to an immediate advantage in their ability to correctly format journal citations and references on an ungraded quiz. Furthermore, the advantage extended to a graded APA-style test even after students in the recognition group had an opportunity to produce citations and references. It seems that initially learning APA style by identifying errors leads to a performance deficit that is difficult to overcome. The second research question asked how students perceive activities that emphasize production of APA-style materials versus recognizing errors in APA style. Students rated the two activities similarly in terms of enjoyability and their own effort. There was a trend toward students perceiving the error-recognition task as being easier, and 54% indicated that they preferred that activity even though 67% indicated that they would learn more from the production activity. These results indicate that students find both activities acceptable but would prefer to engage in the easier, less effective recognition activity.

Finding that production of answers leads to better performance than recognition of answers is consistent with findings from cognitive psychology. Basic memory research shows that generating answers to questions and testing knowledge are highly effective methods of learning even though they can feel more challenging and less productive than other methods (Bertsch et al., 2007; Bjork et al., 2013; Roediger & Karpicke, 2006). Furthermore, feelings of familiarity with easily processed material can lead to ineffective study methods that translate to poor long-term retention (Bjork et al., 2013), and this may explain why the initial deficit in performance among the recognition-first group extended to a test conducted after they had the opportunity to complete the production activity. Overall, application of cognitive psychology to classroom learning emphasizes the importance of students engaging in challenging tasks that require them to repeatedly practice the recall of information on their own (Putnam et al., 2016), and that is consistent with the main findings of the current study.

The main educational implication of the current study is clear: Students need to practice formatting APA-style materials. Some teachers may rely on assigned papers to instruct students on APA style, but writing a paper requires only one repetition of APA formatting. If teachers want students to master APA formatting, more repetitions are necessary. As such, teachers should have students produce APA-style materials in class as an instructional technique, but they should also encourage formatting practice in homework assignments and as a study method. A simple way to practice APA-style citation and referencing in class is to share a journal article and then have students write a citation and a reference entry. Collaboration can easily be incorporated into the activity by having students compare their work, identify inconstancies, and make corrections together. Once students have completed the task, the teacher should review the correct answer so that students can get immediate feedback on the accuracy of their work.

Another implication of the current research is that it may take some effort on the part of teachers to motivate students to meaningfully practice APA-style formatting. The results showed a disconnect between students’ belief about what technique lead to the most learning and what technique they would prefer to use. Spotting errors is a relatively quick and easy task compared to the production of full citations and references, and research shows that students often choose easy but ineffective study techniques despite awareness of more effective methods (Kornell & Bjork, 2007; Morehead, Rhodes, & DeLozier, 2016). Thus, the production of materials during initial APA-style instruction should be required, not optional. Also, teachers should attempt to counteract student resistance to the challenging work of producing APA-style materials. Teachers can explain the learning benefits of production and the science behind why it works. In addition, they can emphasize to students that they will eventually need to produce citations and references in their writing, and the initial hard work at mastering APA style will be rewarded with greater efficiency and better grades on subsequent writing assignments.

Although the current research provided clear and internally valid evidence for the superior effectiveness of one instructional technique over another, there were some limitations that should be noted. One limitation was the exclusive focus on citations and references, which are aspects of APA style with many simple and specific, albeit arbitrary, rules. It is not clear whether the results would generalize to more complex and ambiguous aspects of APA style such as precision in language or the content requirements of empirical reports. Another limitation was the ecological validity of the assessments of students’ APA-style ability. Although the quizzes and test required students to produce APA-style citations and references, the goal of learning APA style is to correctly format written assignments. Quiz and test performance may not predict the correct application of APA style to written assignments. To address these limitations, future research should investigate the impact of the instructional techniques in more advanced classes that require students to produce complete APA-style papers. For example, researchers may compare the effects of giving students a sample outline of empirical report content versus having them produce their own outline.

Psychologists have the advantage of teaching a discipline that produces knowledge directly related to its own instruction. Both basic and pedagogical research emphasizes the benefits of actively trying to generate information during the learning process. There are innumerable ways to apply this research in the classroom, but the current study illustrates the usefulness of having students learn APA style by producing APA-style materials. As is often the case, an effective way to teach psychology is by having students actively do psychology.

Footnotes

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

References

American Psychological Association. (2011). Principles for quality undergraduate education in psychology. Retrieved from http://www.apa.org/education/undergrad/principles.aspx

American Psychological Association. (2013). APA guidelines for the undergraduate psychology major 2.0. Washington, DC: Author. Retrieved from www.apa.org/ed/resources.html

Bertsch

Pesta

B. J.

Wiscott

McDaniel

M. A.

(2007). The generation effect: A meta-analytic review. Memory & Cognition, 35, 201–210. doi:10.3758/BF03193441

Bjork

R. A.

Dunlosky

Kornell

(2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64, 417–444. doi:10.1146/annurev-psych-113011-143823

Dunlosky

Rawson

K. A.

(2015). Practice tests, spaced practice, and successive relearning: Tips for classroom use and for guiding students’ learning. Scholarship of Teaching and Learning in Psychology, 1, 72–78. doi:10.1037/stl0000024

Fallahi

C. R.

Wood

R. M.

Austad

C. S.

Fallahi

(2006). A program for improving undergraduate psychology students’ basic writing skills. Teaching of Psychology, 33, 171–175.

Foos

P. W.

Mora

J. J.

Tkacz

(1994). Student study techniques and the generation effect. Journal of Educational Psychology, 86, 567–576. doi:10.1037/0022-0663.86.4.567

Franz

T. M.

Spitzer

T. M.

(2006). Different approaches to teaching the mechanics of American Psychological Association style. Journal of Scholarship of Teaching and Learning, 6, 13–20.

Froese

A. D.K. L.

Garcia

E. D.

Koehn

L. J.

Nelson

J. M

. (1995). Citing secondary sources: Can we correct what students do not know? Teaching of Psychology, 22, 235–238.

10.

Jorgensen

T. D.

Marek

(2013). Workshops increase students’ proficiency at identifying general and APA-style writing errors. Teaching of Psychology, 40, 294–299.

11.

Kornell

Bjork

R. A.

(2007). The promise and perils of self-regulated study. Psychonomic Bulletin & Review, 14, 219–224. doi:10.3758/BF03194055

12.

Landrum

R. E.

(2013). Writing in APA style: Faculty perspectives of competence and importance. Psychology Learning and Teaching, 12, 259–265.

13.

Luttrell

V. R.

Bufkin

J. L.

Eastman

V. J.

Miller

(2010). Teaching scientific writing: Measuring student learning in an intensive APA skills course. Teaching of Psychology, 37, 193–195. doi:10.1080/00986283.2010.488531

14.

Mandernach

B. J.

Zafonte

Taylor

(2016). Instructional strategies to improve college students’ APA style writing. International Journal of Teaching & Learning in Higher Education, 27, 407–412.

15.

Morehead

Rhodes

M. G.

DeLozier

(2016). Instructor and student knowledge of study strategies. Memory, 24, 257–271. doi:10.1080/09658211.2014.1001992

16.

Putnam

A. L.

Sungkhasettee

V. W.

Roediger

I. L.

(2016). Optimizing learning in college: Tips from cognitive psychology. Perspectives on Psychological Science, 11, 652–660. doi:10.1177/1745691616645770

17.

Roediger

H. L. III

Karpicke

J. D

. (2006). The power of testing memory basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181–210. doi:10.1111/j.1745-6916.2006.00012.x

18.

Slezak

J. M.

Faas

(2017). Effects of an interteaching probe on learning and generalization of American Psychological Association (APA) style. Teaching of Psychology, 44, 150–154. doi:10.1177/0098628317692619

19.

Smith

G. E.

Eggleston

T. J.

(2001). Comprehending APA style through manuscript analysis. Teaching of Psychology, 28, 108–110.

20.

Van Note Chism

Weerakoon

(2012). APA, meet Google: Graduate students’ approaches to learning citation style. Journal of the Scholarship of Teaching and Learning, 12, 27–38.

21.

Zafonte

Parks-Stamm

E. J.

(2016). Effective instruction in APA style in blended and face-to-face classrooms. Scholarship of Teaching and Learning in Psychology, 2, 208–218. doi:10.1037/stl0000064