The Roles of Attitudinal Perceptions and Cognitive Achievements in a Serious Game

Abstract

This study investigated a conceptual model that connects game expertise, attitudinal perceptions, and cognitive achievements, and behavioral intention in digital game-based learning. In the study, attitudinal perceptions consisted of three constructs: perceived enjoyment, perceived usefulness, and satisfaction, and cognitive achievements consisted of two constructs: game scores and recall of content. A total of 298 college students played a serious game about transistor recycling, and structural equation modeling was used to analyze data. Our findings showed that two attitudinal constructs (i.e., perceived enjoyment and perceived usefulness) and one cognitive achievement (i.e., game scores) positively influenced satisfaction, and then their satisfaction positively influenced both their recall of content and behavioral intention while game expertise significantly affected on perceived usefulness. This complementary relationship between attitudinal perceptions and cognitive achievements found in this study suggests that when designing a serious game, game designers need to consider the interplay between entertaining factors and instructional strategies.

Keywords

digital game-based learnings serious game attitudinal perceptions cognitive achievements Technology Acceptance Model

Introduction

Digital game-based learning integrates various types of digital games into learning processes to engage learners more deeply (Van Eck, 2006) because gaming attributes (e.g., challenge, competition, control, curiosity, and fantasy) of digital games can facilitate learning (Baek, 2008; Gee, 2003; Hsu, 2012; Malone & Lepper, 1987). For example, challenging tasks embedded in games make players feel a sense of accomplishment when they reach higher levels. Also, the combination of an interesting story, a clear goal, and a competitive mission keep players motivated as well. Moreover, digital games offer highly engaging environments with high-quality images and sounds that keep players entertained (Foster, 2008; Rieber & Noah, 2008). Further, these specific features of digital games could help learners not only improve cognitive skills, such as problem solving and decision making skills (Brom, Šisler, & Slavík, 2010) but also change their attitudes and behaviors (de Freitas & Maharg, 2011; O’Neil & Perez, 2008).

As one type of digital game-based learnings, serious games are designed with specific educational purposes in addition to intrinsic purpose of entertainment (Michael & Chen, 2006; Van Eck, 2006). For example, Immune Attack (http://immuneattack.org) is a serious game that teaches adolescents how an immune system works in human blood vessel. Prior literature illustrated that players who take serious games can increase recall and comprehensive test scores in many areas, such as language (deHaan, Reed, & Kuwada, 2010), electricity (Mayer & Johnson, 2010), history (Mansour & El-Said, 2008), and physics (Squire, Barnett, & Grant, 2004). Serious games are also designed to change learners’ attitude and behavior (O’Neil & Perez, 2008; Prensky, 2001; van Staalduinen & de Freitas, 2011; Wouters, van der Spek, & van Oostendorp, 2009). For example, Escape From Diab (http://www.escapefromdiab.com) was developed to reduce risk for diabetes and obesity among youth by changing their attitude to diet and physical activity (Thompson et al., 2010).

Although attitudinal perceptions and cognitive achievements in serious games have been considered as important variables, few studies have explored those factors as well as their effects on learning performance and learners’ behavior. Thus, this study aims to investigate how learners’ attitudinal perceptions and cognitive achievements affect outcomes of a serious game. With a serious game called transistor recycling that incorporates entertainment and learning components, this study examines the relationships among game expertise, attitudinal perceptions (i.e., perceived enjoyment, perceived usefulness, and satisfaction), cognitive achievements (i.e., game scores and recall of content), and learning outcomes (i.e., intention to recycling behavior).

Research Model and Hypothesis Development

Our research model includes four categories of variables: external, attitudinal, cognitive, and purpose variables (see Figure 1). An external variable referring to individual differences in technology expertise (Burton-Jones & Hubona, 2006; i.e., game expertise), and three attitudinal perception variables (i.e., perceived enjoyment, perceived usefulness, and satisfaction) were adopted from the extended Technology Acceptance Model (TAM; Davis, 1989; Davis, Bagozzi, & Warshaw, 1989; Lee, Cheung, & Chen, 2005) from a motivational perspective (Deci, 1975). In the previous studies, perceived enjoyment and perceived usefulness have been considered as intrinsic and extrinsic motivations, respectively. Also, our conceptual model contains cognitive achievement variables (i.e., game scores and recall of content) that represent to what extent players understand learning contents during and after a game. Since the goal of a serious game used in this study is to encourage recycling behavior, behavioral intention is used as the final dependent variable as a learning outcome. Our conceptual model is shown in Figure 1. In the following sections, related studies regarding the above-mentioned variables and our hypotheses are presented.

Attitudinal Perceptions

This study adopted three attitudinal perceptions including perceived enjoyment, perceived usefulness, and satisfaction from the modified TAM, which is based on a motivational theory. Several researchers extended the TAM by finding the positive relationships between perceived enjoyment of using a particular technology (defined as intrinsic motivation) and satisfaction toward the technology, and also between perceived usefulness of the technology and the satisfaction (Bruner & Kumar, 2005; Moon & Kim, 2001; van der Heijden, 2004). The TAM originated from information system studies and explores how users adopt and use a particular information technology (Davis, 1989; Davis et al., 1989). The original TAM suggested the important role of two major factors: (a) perceived usefulness (i.e., the degree of the user’s belief that the technology will improve one’s performance) and (b) perceived ease of use (i.e., the degree of the user’s belief that the technology is easy to learn without one’s intensive effort) in determining behavioral intention (i.e., acceptance of a technology). Recent studies have modified the factors influencing adoption decision based on a motivational framework (i.e., intrinsic and extrinsic motivations), and the modified attitudinal factors have been used to predict the technology adoption or acceptance in various learning technology arenas, such as computer-based instruction and serious games (Lee et al., 2005; Liu, Chen, Sun, Wible, & Kuo, 2010; Shin, 2009; van der Heijden, 2004; Yusoff et al., 2010).

Based on a motivational theory, several studies (Lee et al., 2005; Shin, 2009) identified intrinsic (induced from the task itself but not by any outside pressure) and extrinsic motivations (driven by the effort to acquire desired outcomes, such as rewards) that produce specific outcome behavior. In the modified TAM, intrinsic and extrinsic motivations have been assessed by using perceived enjoyment and perceived usefulness (Lee et al., 2005; Yi & Hwang, 2003). These studies adopted the perceived enjoyment to measure the level of player enjoyment for entertainment purposes, and the perceived usefulness to measure how much the player believes a serious game is a learning tool.

Prior studies measured satisfaction to comprehensively evaluate the potential technology usage (Green & Pearson, 2011; Islam, 2011; Lai, 2009; Wang, Wang, & Shee, 2007). In these studies, users’ higher satisfaction level increased their intention to use the technology. Satisfaction indicates the sum of users’ attitudes (i.e., favor/disfavor and positive/negative feelings) toward a particular technology. Thus, this study employs satisfaction as overall attitudinal perception toward a serious game. Since previous studies demonstrated that the two motivational factors (i.e., perceived enjoyment and perceived usefulness) positively influenced learners’ satisfaction represented as overall attitude (Lee et al., 2005; Shin, 2009; Tao, Cheng, & Sun, 2009; Yi & Hwang, 2003), we assume that more pleasurable game experiences will lead to higher satisfaction levels. Also, as the learners perceive the serious game as a valuable learning tool, they will have an increased level of satisfaction toward the serious game.

Previous studies also suggest that the enjoyment perception (i.e., intrinsic motivation) of a technology, such as e-learning systems (i.e., Blackboard) and online shopping sites, positively influenced the usefulness perception (i.e., extrinsic motivation) (Huang, Yang, & Liaw, 2012; Lim, Lim, & Heinrichs, 2008; Yi & Hwang, 2003). When users are intrinsically motivated with a technology, they become deeply involved with the content contained in the technology. Thus, as players feel enjoyment with serious games, they will have more positive perceived usefulness toward the game delivering useful learning content. Specific hypotheses are presented below:

H1: Perceived enjoyment of a serious game will positively influence satisfaction toward the game.

H2: Perceived usefulness of a serious game will positively influence satisfaction toward the game.

H3: Perceived enjoyment of a serious game will positively influence perceived usefulness of the game.

Cognitive Achievements

Serious games are generally designed to enhance knowledge acquisition through cognitive activities in gaming environments (Kenny & Gunter, 2007). Thus, our conceptual model in this study comprised two cognitive achievement variables (i.e., game scores and recall of content) as well as attitudinal perceptions.

The stream of in-game events reflects a continuous exchange of information between the player and the game (Klimmt & Hartmann, 2006). In general, game scores received in a game can be considered as the game’s instant feedback for the player’s performance. In a serious game, game scores represent the progress of learning during a game (Yusoff et al., 2010).

On the other hand, recall test scores measure how much information the player actually remembers after a game. The scores represent the direct evidence of learning performance. So, learners with higher game scores will perform in a recall test better than learners with lower game scores. In addition, the common goal of playing games is to have higher score, and players often replay games to receive higher scores. When they have higher game scores, they will be more satisfied with a game.

Additionally, several researchers argued that attitudes positively affect recall of information because positive attitudes can function as cues to retrieve the information from memory (Olson & Cal, 1984; Ross, McFarland, & Fletcher, 1981). Positive attitudes (e.g., emotion) can help learners remember information by allocating more cognitive resources (Lang, 2006). Consistent with these notions, greater satisfaction with the serious game may facilitate learning outcomes, such as recall of the game content. Thus, the following hypotheses are given:

H4: Game scores gained in a serious game will positively influence recall of learning contents.

H5: Game scores gained in a serious game will positively influence satisfaction toward the game.

H6: Satisfaction toward a serious game will positively influence recall of content in the game.

Behavioral Intention

According to the theory of reasoned action (Ajzen, 1991; Ajzen & Fishbein, 1980), a human tends to guess consequences that will happen before performing tasks. Thus, human actual behavior can be inferred by behavioral intention. A number of studies have examined behavioral intention to predict actual or continuance use of a technology, including serious games (e.g., Bourgonjon, Valcke, Soetaert, & Schellens, 2010; Yusoff et al., 2010).

Many serious games expect to change players’ actual behaviors after learning from content. For example, a serious game about diabetes improved the players’ eating habits (Brown et al., 1997; Turin, Riva, Galbiati, & Cainelli, 2000). A serious game (The Transistor) chosen in the current study was designed to increase actual recycling behavior. Thus, we adopted the behavioral intention variable to measure the learners’ recycling intentions. Numerous previous studies found that satisfaction with e-learning positively affects behavioral intention (e.g., Lai, 2009; Tang & Chiang, 2010). Because satisfaction reflects a comprehensive attitudinal evaluation of a technology, greater satisfaction increases behavioral intention, such as intention to adopt a new technology or purchase an item (Cheon, Lee, Crooks, & Song, 2012).

Additionally, behavioral intention may also be affected by recall of learning content. A study (Yates, Wagner, & Suprenant, 1997) found that increased recall of healthy or risky products positively influenced players’ intention to avoid such type of the products. In other words, well-structured schema of relevant information can affect learners’ actual behaviors (McVee, Dunsmore, & Gavelek, 2005). Thus, recall of recyclable and unrecyclable items from the game may affect actual recycling behavior. Thus, specific hypotheses are provided below:

H7: Satisfaction toward a serious game will positively influence behavioral intention.

H8: Recall of content in a serious game will positively influence behavioral intention.

Game Expertise

Several studies on the TAM have included external variables, such as characteristics of technology or users, which directly or indirectly impacted on behavioral intention (Davis, 1989; Venkatesh & Davis, 1996; Yousafzai, Foxall, & Pallister, 2007). Studies have found that user expertise with a technology positively influenced his or her attitudes toward and adoption to use specific technology (e.g., Liu et al., 2010; Sahin & Shelley, 2008). Individuals’ levels of game expertise (i.e., experiences) can affect their game score and their attitude toward a serious game. Thus, the current study explores the effects of game expertise on the antecedent variables in our conceptual model (i.e., perceived enjoyment, perceived usefulness, and game scores).

A primary reason of game play is to experience enjoyment. The enjoyment will likely occur when demanding tasks are successfully completed. Experienced game players can predict what, where, and when in-game events will happen, and they are more likely to meet required tasks than novices. Past studies argued that game expertise is positively related with the perceived enjoyment (Tamborini, Bowman, Eden, Grizzard, & Organ, 2010; Trepte & Reinecke, 2011). Moreover, experts tend to have more accurate knowledge (Bhattacherjee & Sanford, 2006) about entertaining and learning attributes of serious games. Thus, game expertise may have a positive relationship with the perception of the serious game as a useful learning tool. Finally, because game scores are received whenever the player completes demanding tasks, there may be a positive effect of game expertise on game scores. Based on these assumptions, we have the following hypotheses:

H9: Game expertise will positively influence perceived enjoyment of the game.

H10: Game expertise will positively influence perceived usefulness of the game.

H11: Game expertise will positively influence game scores.

In summary, this study proposes a conceptual model for a serious game environment based on the extended TAM models. Specifically, our model aims to investigate the simultaneous relationships among game expertise, three constructs of attitudinal perceptions (i.e., perceived enjoyment, perceived usefulness, and satisfaction), two constructs of cognitive achievements (i.e., game scores and recall of content), and behavioral intention.

Methods

Participants

Data (N = 298) were collected through convenience sampling (Creswell, 2012) in a undergraduate course “Computing and Information Technology,” which was a core curriculum course required for all students at a large southwestern university in the United States. The students in the course voluntarily participated in this study and received extra course credits for their participation. There were 111 males and 187 females. The ages of the participants (M = 19.78; SD = 2.56) ranged from 17 to 39. The participants were 112 freshmen, 102 sophomores, 59 juniors, and 25 seniors.

Data Collection

A serious game

The participants in this study played a serious game, The Transistor, which was one of the popular educational games released by the Official Web Site of the Nobel Prize (www.nobelprize.org; see Figure 2). This game teaches the importance of transistor recycling in current everyday life. In the game, transistor and nontransistor items appeared on a conveyer belt and moved toward the recycling machine. Players were asked to correctly remove only nontransistor items from the belt by using a mouse click. They received one point for each item that was correctly removed. They lost two points from their total score if they mistakenly removed a transistor item. As the game level became higher, new items appeared and the belt moved faster. The participants were asked to play repeatedly as many as times for 15 min. The operation and content of the game was simple. One-time game play generally spent 1 or 2 min; therefore, generally the participants could play the game more than 10 times with different items.

Measurements

Using a paper and pencil survey, this study measured the participants’ various perceptions and cognitive achievements. All survey items (game expertise, perceived enjoyment, perceived usefulness, satisfaction, and behavioral intention) were adopted from previous studies (see Appendix). All the items used a 7-point Likert scale, ranging from strongly disagree to strongly agree. Additionally, game scores were self-reported by asking the best game score the participants received. After the attitudinal perceptions and game scores were answered, the participants took a separate section for recall test. Recall test scores were assessed by counting the number of the correctly recalled items among total 21 items (e.g., recyclable items with transistors: TV, phone, and so forth; unrecyclable items without transistors: lamp, speaker, etc.) appeared in the game.

Procedure

This study was conducted in a computer lab with 23 individual desktop computers. Voluntary participants showed up at a predetermined class time within a research module week and received a survey paper. This study consisted of four phases. First, the participants responded to demographic questions (e.g., gender, age, and game expertise). Second, they played a serious game about transistor recycling for 15 min. Third, they stopped the game and wrote down the best game score they received in the game and responded to questions about perceived enjoyment, perceived usefulness, satisfaction, and behavioral intention. Fourth, participants were asked to list all names of transistor and nontransistor items shown in the game.

Data Analysis

This study employed structural equation modeling (SEM) to address the primary research purposes. Because the model comprises five latent variables including game expertise, perceived enjoyment, perceived usefulness, satisfaction, and behavioral intention, paths among the latent variables, measurement errors as well as structural coefficients are main issues. SEM can explicitly consider measurement model and structural model at the same time. A two-stage of modeling approach, recommended by Anderson and Gerbing (1988), was followed such that the convergent and discriminant validity are tested with the confirmatory factor analysis, and the path coefficients are estimated with SEM.

Measurement Model

The descriptive statistics of the measurement items are shown in the first three columns in Table 1. Except for SA4, all means scores of participants’ responses were over 3.5, ranging from 3.72 to 5.20. However, the means of indicators for behavioral intention are about 3.73, which are less than the means of other latent variables. Given that behavioral intention is about the users’ willingness to recycle items with transistors, the result indicates that they may not have high intention to use transistors to recycle items after the transistors recycle game. The high means of other variables indicated an overall positive response to constructs. The standard deviations ranged from 1.58 to 2.16, indicating a reasonable distribution of item scores around each mean.

Table 1.

Results for the Measurement Model.

Construct	Mean	SD	Standardized factor loadings (>0.70)^a	Cronbach’s alpha (>.70)	Composite reliability (>0.70)	Average variance extracted (>0.50)
Game Expertise (GE)				.910	0.913	0.780
GE1	5.20	1.72	0.802
GE2	4.32	2.16	0.877
GE3	4.79	1.93	0.963
Perceived Enjoyment (PE)	.962	0.963	0.867
PE1	4.39	1.55	0.906
PE2	4.19	1.62	0.908
PE3	3.81	1.70	0.932
PE4	4.04	1.64	0.976
Perceived Usefulness (PU)	.977	0.977	0.916
PU1	4.50	1.66	0.934
PU2	4.59	1.65	0.963
PU3	4.69	1.66	0.972
PU4	4.63	1.66	0.958
Satisfaction (SA)	.963	0.943	0.873
SA1	4.51	1.56	0.976
SA2	4.49	1.58	0.985
SA3	4.18	1.66	0.929
SA4	3.37	1.74	0.779
SA5	4.11	1.61	0.891
Behavioral Intention (BI)	.985	0.985	0.946
BI1	3.72	1.71	0.973
BI2	3.79	1.76	0.986
BI3	3.71	1.77	0.977
BI4	3.73	1.77	0.955
Game score^b	130.03	39.29
Recall of content^b	11.88	3.07

Indicates an acceptable level of reliability and validity. Game score and recall of content are formative measures.

Using Mplus 6.11, the measurement model was assessed with individual item loadings, reliability of measures, convergent validity, and discriminant validity. Table 1 presents standardized factor loadings, Cronbach’s alpha, composite reliability, and average variance extracted (AVE). Fornell and Larcker (1981) suggested that the convergent validity is assessed with three measures: (a) item reliability of each measure, (b) composite reliability, and (c) the AVE. The item reliability was assessed by factor loadings for each construct. Gefen, Straub, and Boudreau (2000) suggested that the convergent validity at the item level is adequate if factor loadings are greater than a 0.7 threshold. Factor loadings for five constructs ranged from 0.802 to 0.986, indicating the convergence of each item to each construct. The composite reliability of each construct was assessed using Cronbach’s alpha, and Nunnally and Bernstein (1994) recommended a threshold above 0.7 and DeVellis (2003) recommended a higher threshold between 0.80 and 0.90. All values ranged from 0.910 to 0.977 which far exceeds two criteria, demonstrating a good composite reliability of each construct. Finally, all AVE exceeded a 0.5 threshold that range from 0.780 to 0.946. Given the satisfaction of three criteria, the convergent validity for the proposed constructs of the measurement appears to be adequate.

Discriminant validity is used to assess the extent to which constructs differ. When the variance shared between a set of constructs is less than the variance shared between its measures of a construct, the discriminant validity would be adequate (Kline, 2005). For the discriminant validity, the square root of AVE for a given construct was compared with the correlations between the construct and other constructs. If the square root of the AVE of a construct is greater than the off-diagonal elements, this indicates that a construct is more closely related with its indicators than with the other constructs. The diagonal elements in Table 2 are greater than the off-diagonal elements in the corresponding rows and columns, indicating that the discriminant validity appears to be satisfactory for all constructs.

Table 2.

Discriminant Validity for the Measurement Model.

Construct	GE	PE	PU	SA	BI
GE	.833
PE	.026	.930
PU	.191	.558	.957
SA	.077	.808	.697	.915
BI	.030	.448	.443	.529	.972

Note. The items on the diagonal represent the square roots of the average variance extracted; off-diagonal elements are the correlation estimates. GE = game expertise; PE = perceived enjoyment; PU = perceived usefulness; SA =Satisfaction; BI = behavioral intention.

Results

Structural Model

The proposed structural model was estimated using Mplus 6.11 with the maximum likelihood (ML) method. Although unweight least square, generalized least square (GLS), and maximum likelihood can be used for estimating parameters, ML is most frequently used in estimating the parameters, and the use of other methods require reasonable justification (Hoyle, 2000). Compared to generalized least square, ML provides realistic index of overall fit under conditions of misspecification and less biased parameters (Olsson et al., 2000). SEM was assessed with the fit between the data obtained and the proposed model. Researchers have developed numerous criteria for assessing model fit (Kline, 2005, Schumacker & Lomax, 2010). Table 3 presents model fit statistics as well as recommend guidelines and their references. Except for χ² and χ²/df, all other model fits exceed the recommended guidelines of acceptable level. However, χ² is affected by several factors, among them the sample size (Kline, 2011, p. 204). As sample size increases, the χ² statistic is also likely to have a significant level. This measure may lead to the conclusion of lack of model fit even when the model is appropriate (Bollen, 1989). Thus, the normed chi-square (χ²/df) is proposed to reduce the effect of sample size (Jöreskog, 1970). The normed chi-square statistic is 2.57, which is below 3, indicating an acceptable fit of the proposed model. For the judgment of overall fit of the model, Hu and Bentler (1999) suggested that a combination rule, instead of independent use of model fits, is required to test the adequacy of overall fit of the model to control type I and type II errors at the same time. A set of model fits, shown in Table 3, satisfied the combination rule as well as the independent recommendation level. Thus, the satisfaction of a variety of model fits indicates that the proposed model appears to be adequate.

Table 3.

Model Fit Indices.

Fit indices	Values	Recommended guidelines	References
χ²	567.98	Nonsignificant	Klem (2000), Kline (2005)
χ²/df	2.570	<3 (acceptable) <2 (good fit)	Kline (2005), Tabachnik and Fidell (2007)
CFI	0.962	≥0.90	Hu and Benter (1999)
TLI	0.957	≥0.90	Hu and Benter (1999), Kline (2005)
RMSEA	0.073 (0.065, 0.080)	<0.05 (good fit) <0.08 (fair fit)	Kline (2005), McDonald and Ho (2002)
Standardized RMR	0.047	<0.05 (good fit) <0.08 (fair fit)	Byrne (1998), Hu and Benter (1999); Kline (2005)

Note. CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; RMSEA = Root-Mean-Square Error of Approximation; RMR = Root-Mean-Square Residual.

Hypothesis Testing

Figure 3 shows the results of path coefficients of the proposed research model. Consistent with Hypotheses 1, 2, and 3 of attitudinal perceptions, perceived enjoyment to satisfaction (0.603) and to perceived usefulness (0.553), and perceived usefulness to satisfaction (0.362) are statistically significant. Game scores to satisfaction (0.077) and to recall of content (0.147), and then satisfaction to recall test score (0.116) are also statistically significant although the magnitude of effects are different. Game scores are more likely to influence on recall of content, rather than satisfaction. These results support Hypotheses 4, 5, and 6 of the relationship between attitudinal perceptions and cognitive achievements. For the relationship among satisfaction, recall of content, and behavioral intention, satisfaction affects behavioral intention with 0.526, but recall of content does not influence behavioral intention. Thus, the Hypothesis 7 is supported, but the Hypothesis 8 is not supported. Finally, game expertise is significantly related only to perceived usefulness with 0.176, but it is not related to perceived enjoyment and game scores. Thus, the result supports only Hypothesis 10, but not Hypotheses 9 and 11.

Discussion and Conclusion

This study aimed to determine how attitudinal and cognitive factors contribute to the success of a serious game. The results found that participants’ attitudinal perception and cognitive achievements were significantly related each other, and satisfaction was the key contributor to both recall test score and intention to recycling behavior. However, the cognitive achievement (i.e., recall test scores) did not directly affect the intention to recycling behavior.

Consistent with previous TAM studies, our model demonstrated that perceived enjoyment and usefulness were significant determinants of learners’ satisfaction toward the serious game. In other words, learners who feel that the serious game is enjoyable and useful are more likely to intend to recycle. Thus, learners should be motivated by entertaining gaming components (e.g., challenge, competition, or curiosity) and meaningful contents (Brom et al., 2010; Foster, 2008). Also, enjoyment perception also positively affected usefulness perception. It implies that learners will not perceive a game as a valuable learning tool if the game is not fun. Thus, entertaining components should take precedence over the others in serious game design.

In addition to the attitudinal factors, the findings indicated that game scores positively influenced recall test score and satisfaction as well. Game scores are a typical reward for better performance or progress in a game context (Yusoff et al., 2010). In the serious game in this study, users can receive higher game scores when they correctly identified recyclable or nonrecyclable items. With the gaming activity, they could practice the distinctions, so then they could easily recall the items that appeared in the game. Since the game score is a numerical representation of learners’ learning progress during a game (Kelle, Klemke, & Specht, 2013), the positive effect can be clearly expected. However, if a gaming activity is not associated with a learning goal, a positive connection may disappear. On the other hand, the significant correlation between game score and satisfaction shows that learners’ cognitive performance while playing a game makes a positive impact on their satisfaction with a serious game. Therefore, the results suggest that a scoring system should be carefully designed by considering users’ attitudinal and cognitive perspectives.

Furthermore, this study also affirms that satisfaction positively influenced both recall test score and behavioral intention. For example, motivation by playing a game can facilitate learning (Münz, Schumm, Wiesebrock, & Allgöwer, 2007). Because satisfaction reflects an increase in motivation to learn the content, recall test scores could be facilitated with the enhanced satisfaction (Sung & Mayer, 2012). The results support that attitude is a significant predictor of behavioral intention as illustrated in previous studies (Lai, 2009; Song & Zahedi, 2005; Tang & Chiang, 2010).

However, our study did not find support for the relationship between recall test score and behavioral intention. We conjecture that this finding is due to the type of learning outcome of the serious game. The recycling behavior can be categorized into affective domain rather than cognitive or psychomotor domain. So, the behavioral intention was more closely related with learners’ satisfaction levels indicating overall attitudinal judgment affected by the level of perceptions of the serious game as both entertaining and learning tools in this study.

Additionally, the results found that learner’s game expertise was significantly related to only perceived usefulness. Experienced game players seem to have a better understanding of the benefits of serious games as a learning tool (Bourgonjon et al., 2010). However, their game expertise neither improved game performance nor affected their feeling of enjoyment. This finding does not support our hypothesis in which game expertise could make a serious game more fun or make players achieve higher scores. We assume that it occurred because the serious game was simple to play and contents were mainly learning focused, unlike typical video games. Participants might have conceived commercial games when they were asked to indicate their game experience levels, and the term serious game was potentially new to them. For example, some regular game players might have less experience of playing educational games with specific learning goals. It can be concluded that game expertise does not have strong impact on the success of serious games.

This study proposes a new conceptual model that shows how attitudinal and cognitive constructs influence the goal of a serious game. Since few prior research have explored the simultaneous relationships among the satisfaction toward the serious game, the cognitive outcome, and the behavior goal, the model could be a basis to explore what component is significant in accelerating learning in a serious game. Satisfaction was a critical factor in achieving the learning goal (behavioral intention) as well as enhancing recall of the learning content. Also, the determinants of satisfaction were perceived enjoyment and game score. The interrelated constructs in our model imply that serious game design should balance learning and entertainment. The design principle can be called learning-driven game design (van Staalduinen & de Freitas, 2011). If a game is well designed for enjoyment but does not link to instructional objectives, learners will not learn desired content. On the other hand, if a game is learning oriented but does not incorporate sound game design, learners will not be motivated or engaged (Bergeld, 2010; Kenny & Gunter, 2007). Specifically, we suggest that a scoring system should be designed to enhance learners’ positive attitudes and learning outcomes.

When users play a game, they always try to get higher score or level. This is a key attribute of a game (i.e., challenge) to engage players. On the other hand, since a serious game delivers educational content, game scores represents how well learners understand the content (Kelle et al., 2013). Therefore, a scoring system should accurately measure what learners have learned and effectively motivate learners to achieve higher scores. The gaming strategy and instructional strategy should be integrated into a serious game design. For example, just-in-time feedback (e.g., constructive feedback) could be provided to reflect gained knowledge or correct learners’ mistakes. Personalized learning strategy can be implemented by providing learners with a slightly higher level of the individual’s current competence level (Yusoff et al., 2010).

This study has a number of limitations due to the nature of empirical research. First, our findings may be limited to a serious game to improve players’ attitudes and actual behaviors. Other serious games may focus on providing information learners should comprehend, such as foreign language or scientific knowledge. Thus, future research can explore different types of serious games with learning purposes in a cognitive domain. Second, the game used in this study was simple to play in that learners were supposed to remove goods that did not have a transistor from a belt for recycling. Future research can use more complex games, such as simulation, to examine the relationships among constructs proposed in this study. Third, we collected the game scores from participants after playing the game, and the change of individual’s scores was not archived. The log data of game scores would provide another aspect to investigate the role of prior knowledge in a serious game and the potential to implement personalized learning. Last, the participants’ prior knowledge about recycling items was not collected. The difference of pre- and posttest scores can be used as a cognitive achievement in a further study.

Prior to this study, there was a lack of information about the roles learners’ attitudinal perception and cognitive achievement complementally play in improving learning outcomes in a serious game. To address this deficiency, we developed a new conceptual model to investigate the factors contributing to behavioral intention with a serious game The Transistor. Our study indicated that both learners’ perceptions and game score shape their satisfaction and the satisfaction then influences their recall test scores and recycling intention. The reciprocal relationships between two constructs imply that entertaining and instructional components should be considered together in the systematic complementary way in a serious game design. So, we would not have such serious games with useful information which are boring or not fun; or such fun games which do not deliver learning content effectively.

Footnotes

Appendix: Measurement Items

Variables	Items	References
Game Expertise (GE)	GE1: I think that I am competent for any game I will play. GE2: I think that I am expert in playing any game. GE3: I feel comfortable with any game I will play.	Griffin, Babin, and Attaway (1996), Netemeyer and Bearden (1992), Tripp, Jensen, and Carlson (1994)
Perceived Enjoyment (PE)	PE1: It was pleasant to play this game for fun. PE2: Playing this game was interesting to me. PE3: I like playing this game for entertainment. PE4: Playing this game was enjoyable to me.	Chen and Wells (1999), Pecheux and Derbaix (1999), Roehm and Sternthal (2001)
Perceived Usefulness (PU)	PU1: I think that this game can provide people with useful knowledge. PU2: I think that this game can be an appropriate learning tool. PU3: I think that people can learn through playing this game. PU4: I think that people can gain useful information through playing this game.	Chen and Wells (1999), Pecheux and Derbaix (1999), Roehm and Sternthal (2001)
Satisfaction (SA)	SA1: All things considered, I am very satisfied with this game. SA2: All things considered, I am very pleased with this game. SA3: All things considered, I am delighted with this game. SA4: All things considered, I will definitely recommend this game to my colleagues. SA5: Overall, my interaction with this game is very satisfying.	Arnould and Price (1993), Keaveney and Parthasarathy (2001), Price, Arnould, and Tierney (1995)
Behavioral Intention (BI)	BI1: I intend to recycle items with transistors. BI2: I predict that I would recycle items with transistors. BI3: I plan to recycle items with transistors. BI4: I expect to recycle items with transistors.	Cronin, Brady, and Hult (2000), Jones, Mothersbaugh, and Beatty (2000)

Acknowledgments

The authors wish to acknowledge the helpful comments from anonymous reviewers.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Jongpil Cheon is an associate professor in the Instructional Technology program at Texas Tech University. His research interests involve implementing immersive online learning environments and investigating advanced technologies for interactive learning. Specific research issues include multimedia learning, mobile learning and technology adoption. His research findings have appeared in Computers and Education, Educational Technology Research and Development, Computers in Human Behavior, and other decent journals.

Sungwon Chung is a doctoral candidate in the Instructional Technology program at Texas Tech University. His research interests include digital game-based learning and motivated cognitive processing.

Sangno Lee is an assistant professor of Accounting at Chonbuk National University. His research interests include accounting information systems, business intelligence, and business education.

References

Ajzen

(1991) The theory of planned behavior. Organizational Behavior and Human Decision Processes 50(2): 179–211.

Ajzen

Fishbein

(1980) Understanding attitudes and predicting social behavior, Englewood Cliffs, NJ: Prentice-Hall.

Anderson

J. C.

Gerbing

D. W.

(1988) Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin 103(3): 411–423.

Arnould

E. J.

Price

L. L.

(1993) River magic: Extraordinary experience and the extended service encounter. Journal of Consumer Research 20(1): 24–45.

Baek

(2008) Revealing new hidden curriculum and pedagogy of digital games. In: Ferdig

R. E.

(ed.) Handbook of research on effective electronic gaming in education, Hershey, PA: IGI Global, pp. 1025–1040.

Bergfeld, C. (2010, April 10). 10NTC: Should serious games be fun? (Yes) [Video file]. Retrieved from http://forums.techsoup.org/cs/community/b/tsblog/archive/2010/06/07/10ntc-should-serious-games-be-fun-yes.aspx.

Bhattacherjee

Sanford

(2006) Influence processes for information technology acceptance: An elaboration likelihood model. MIS Quarterly 30(4): 805–825.

Bollen

K. A.

(1989) Structural equations with latent variables, New York, NY: Wiley.

Bourgonjon

Valcke

Soetaert

Schellens

(2010) Students’ perceptions about the use of video games in the classroom. Computers & Education 54(4): 1145–1156.

10.

Brom

Šisler

Slavík

(2010) Implementing digital game-based learning in schools: Augmented learning environment of ‘Europe 2045’. Multimedia Systems 16: 23–41.

11.

Brown

S. J.

Lieberman

D. A.

Germeny

B. A.

Fan

Y. C.

Wilson

D. M.

Pasta

D. J.

(1997) Educational video game for juvenile diabetes: Results of a controlled trial. Medical Informatics 22(1): 77–89.

12.

Bruner

G. C.

II Kumar

(2005) Explaining consumer acceptance of handheld Internet devices. Journal of Business Research 58(5): 553–558.

13.

Byrne, B. M. (1998). Structural equation modeling with LISREL, PRELIS and SIMPLIS: Basic concepts, applications and programming. Mahwah, New Jersey: Lawrence Erlbaum Associates.

14.

Burton-Jones

Hubona

G. S.

(2006) The mediation of external variables in the technology acceptance model. Information & Management 43(6): 706–717.

15.

Chen

Wells

W. D.

(1999) Attitude toward the site. Journal of Advertising Research 39(5): 27–37.

16.

Cheon

Lee

Crooks

S. M.

Song

(2012) An investigation of mobile learning readiness in higher education based on theory of planned behavior. Computers & Education 59(3): 1054–1064.

17.

Creswell

J. W.

(2012) Educational research: Planning, conducting, and evaluating quantitative and qualitative research, 4th ed. Boston, MA: Pearson.

18.

Cronin

J. J.

Jr Brady

M. K.

Hult

T. M.

(2000) Assessing the effects of quality, value, and customer satisfaction on consumer behavioral intentions in service environments. Journal Retailing 79(2): 193–218.

19.

Davis

F. D.

(1989) Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly 13(3): 319–340.

20.

Davis

F. D.

Bagozzi

R. P.

Warshaw

P. R.

(1989) User acceptance of computer technology: A comparison of two theoretical models. Management Science 35(8): 982–1003.

21.

Deci

E. L.

(1975) Intrinsic motivation, New York, NY: Plenum Press.

22.

de Freitas

Maharg

(2011) Digital games and learning, New York, NY: Continuum.

23.

deHaan

Reed

Kuwada

(2010) The effect of interactivity with a music video game on second language vocabulary recall. Language Learning & Technology 14(2): 74–94.

24.

DeVellis

(2003) Scale development: Theory and applications, 2nd ed. Thousand Oaks, CA: Sage.

25.

Fornell

Larcker

D. F.

(1981) Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research 18(1): 39–50.

26.

Foster

(2008) Games and motivation to learning science: Personal identity, applicability, relevance and meaningfulness. Journal of Interactive Learning Research 19(4): 597–614.

27.

Gee

(2003) What video games have to teach us about learning and literacy, New York, NY: Palgrave MacMillan.

28.

Gefen

Straub

D. W.

Boudreau

(2000) Structural equation modeling techniques and regression: Guidelines for research practice. Communications of the Association for Information Systems 7(7): 1–78.

29.

Green

D. T.

Pearson

J. M.

(2011) Integrating website usability with the electronic commerce acceptance model. Behaviour & Information Technology 30(2): 181–199.

30.

Griffin

Babin

B. J.

Attaway

J. S.

(1996) Anticipation of injurious consumption outcomes and its impact on consumer attributions of blame. Academy of Marketing Science Journal 24(4): 314–327.

31.

Hoyle, R. H. (2000). Confirmatory factor analysis. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of applied multivariate statistics and mathematical modeling (pp. 465–497). New York: Academic Press.

32.

Hsu

T.-Y.

(2012) Constructing a gamed-based learning website for children. International Journal of Humanities & Arts Computing 6(1–2): 184–203.

33.

Bentler

P. M.

(1999) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling 6(1): 1–55.

34.

Huang

A. F. M.

Yang

S. J. H.

Liaw

S.-S.

(2012) A study of user’s acceptance on situational mashups in situational language teaching. British Journal of Educational Technology 43(1): 52–61.

35.

Islam

A. K. M. N.

(2011) The determinants of the post-adoption satisfaction of educators with an e-learning system. Journal of Information Systems Education 22(4): 319–330.

36.

Jones

M. A.

Mothersbaugh

D. L.

Beatty

S. E.

(2000) Switching barriers and repurchase intentions in services. Journal of Retaining 79(2): 259–274.

37.

Jöreskong

K. G.

(1970) A general method for analysis of covariance structures. Biometrika 57: 239–251.

38.

Keaveney

S. M.

Parthasarathy

(2001) Customer switching behavior in online services: An exploratory study of the role of selected attitudinal, behavioral, and demographic factors. Journal of Academy of Marketing Science 29(4): 374–390.

39.

Kelle

Klemke

Specht

(2013) Effects of game design patterns on basic life support training content. Educational Technology & Society 16(1): 275–285.

40.

Kenny

R. F.

Gunter

G. A.

(2007) Endogenous fantasy-based serious games: Intrinsic motivation and learning. International Journal of Social Sciences 2(1): 8–13.

41.

Klem, L. (2000). Structural equation modeling. In L. G. Grimm & P. R. Yarnold (Eds.), Reading and understanding more multivariate statistics (pp. 227–260). Washington DC: American Psychology Association.

42.

Klimmt

Hartmann

(2006) ). Effectance, self-efficacy, and the motivation to play video games. In: Vorderer

Bryant

(eds) Playing video games: Motives, responses, and consequences, Mahwah, NJ: Lawrence Erlbaum, pp. 132–145.

43.

Kline

R. B.

(2005) Principles and practices of structural equation modeling, 2nd ed. New York, NY: Guilford.

44.

Lai

J.-Y.

(2009) How reward, computer self-efficacy, and perceived power security affect knowledge management systems success: An empirical investigation in high-tech companies. Journal of the American Society for Information Science and Technology 60(2): 332–347.

45.

Lang

(2006) Using the limited capacity model of motivated mediated message processing (LC4MP) to design effective cancer communication messages. Journal of Communication 56(S1): S57–S80.

46.

Lee

Cheung

Chen

(2005) Acceptance of Internet-based learning medium: The role of extrinsic and intrinsic motivation. Information & Management 42(8): 1095–1104.

47.

Lim

K.-S.

Lim

J.-S.

Heinrichs

J. H.

(2008) Testing an integrated model of e-shopping web site usage. Journal of Internet Commerce 7(3): 291–312.

48.

Liu

I.-F.

Chen

M. C.

Sun

Y. S.

Wible

Kuo

C.-H.

(2010) Extending the TAM model to explore the factors that affect intention to use an online learning community. Computers & Education 54(2): 600–610.

49.

McDonald, R. P., & Ho, M. H. (2002). Principles and practice in reporting structural equation analysis. Psychological Methods, 7(1), 64–82.

50.

Malone, T. W., & Lepper, M. R. (1987). Making learning fun: A taxonomy of intrinsic motivations for learning. In R. E. Snow & M. J. Farr (Ed.), Aptitude, learning and instruction (Vol. 3: Conative and Affective Process Analyses, pp. 223–253). Hillsdale, NJ: Erlbaum.

51.

Mansour

El-Said

(2008) Multi-players role-playing educational serious games: A link between fun and learning. The International Journal of Learning 15(11): 229–240.

52.

Mayer

R. E.

Johnson

C. I.

(2010) Adding instructional features that promote learning in a game-like environment. Journal of Educational Computing Research 42(3): 241–265.

53.

McVee

M. B.

Dunsmore

Gavelek

J. R.

(2005) Schema theory revisited. Review of Educational Research 75(4): 531–566.

54.

Michael

Chen

(2006) Serious games: Games that educate, train, and inform, Boston, MA: Thomson Course Technology.

55.

Moon

J. W.

Kim

Y. G.

(2001) Extending the TAM for a World-Wide-Web context. Information & Management 38(4): 217–230.

56.

Münz

Schumm

Wiesebrock

Allgöwer

(2007) Motivation and learning progress through educational games. IEEE Transactions on Industrial Electronics 54(6): 3141–3144.

57.

Netemeyer

R. G.

Bearden

W. O.

(1992) A comparative analysis of two models of behavioral intention. Journal of the Academy of Marketing Science 20(1): 49–59.

58.

Nunnally

J. C.

Bernstein

I. H.

(1994) Psychometric theory, 3rd ed. New York, NY: McGraw-Hill.

59.

O’Neil

H. F.

Perez

R. S.

(2008) Computer games and team and individual learning, Oxford, UK: Elsevier.

60.

Olson

J. M.

Cal

A. V.

(1984) Source credibility, attitudes, and the recall of past behaviours. European Journal of Social Psychology 14(2): 203–210.

61.

Olsson

U. H.

Foss

Troye

S. V.

Howell

R. D.

(2000) The performance of ML, GLS, and WLS estimation in structural equation modeling under conditions misspedification and nonnormality. Structural Equation Modeling 7(4): 557–595.

62.

Pecheux

Derbaix

(1999) Children and attitude toward the brand: A new measurement scale. Journal of Advertising Research 39(4): 19–27.

63.

Prensky

(2001) Digital game-based learning, New York, NY: McGraw-Hill.

64.

Price

L. L.

Arnould

E. J.

Tierney

(1995) Going to extremes: Managing service encounters and assessing provider performance. Journal of Marketing 59(4): 83–97.

65.

Rieber

Noah

(2008) Games, simulations, and visual metaphors in education: Antagonism between enjoyment and learning. Educational Media International 45(2): 77–92.

66.

Roehm

M. L.

Sternthal

(2001) The moderating effect of knowledge and resources on the persuasive impact of analogies. Journal of Consumer Research 28(2): 257–272.

67.

Ross

McFarland

Fletcher

G. J. O.

(1981) The effect of attitude on the recall of personal histories. Journal of Personality and Social Psychology 40(4): 627–634.

68.

Sahin

Shelley

(2008) Considering students’ perceptions: The distance education student satisfaction model. Educational Technology & Society 11(3): 216–223.

69.

Schumacker

R. E.

Lomax

R. G.

(2010) A beginners guide to structural equation modeling, New York, NY: Routledge.

70.

Shin

D. H.

(2009) An empirical investigation of a modified technology acceptance model of IPTV. Behaviour & Information Technology 28(4): 361–372.

71.

Song

Zahedi

F. M.

(2005) A theoretical approach to Web design in e-commerce: A belief reinforcement model. Management Science 51(8): 1219–1235.

72.

Squire

Barnett

Grant

(2004) Electromagnetism supercharged! Learning physics with digital simulation games. Proceedings of the 2004 International Conference of the Learning Sciences, 513–520.

73.

Sung

Mayer

R. E.

(2012) When graphics improve liking but not learning from online lessons. Computers in Human Behavior 28(5): 1618–1625.

74.

Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Allyn and Bacon.

75.

Tamborini

Bowman

N. D.

Eden

Grizzard

Organ

(2010) Defining media enjoyment as the satisfaction of intrinsic needs. Journal of Communication 60(4): 758–777.

76.

Tang

J.-T. E.

Chiang

C.-H.

(2010) Integrating experiential value of blog use into the expectation-confirmation theory model. Social Behavior and Personality 38(10): 1377–1390.

77.

Tao

Y. H.

Cheng

C.-J.

Sun

S.-Y.

(2009) What influences college students to continue using business simulation games? The Taiwan experience. Computers and Education 53(3): 929–939.

78.

Thompson

Baranowski

Buday

Baranowski

Thompson

Jago

Griffith

M. J.

(2010) Serious video games for health: How behavioral science guided the development of a serious video game. Simulation & Gaming 41(4): 587–606.

79.

Trepte

Reinecke

(2011) The pleasures of success: Game-related efficacy experiences as a mediator between player performance and game enjoyment. Cyberpsychology, Behavior, and Social Networking 14(9): 555–557.

80.

Tripp

Jensen

T. D.

Carlson

(1994) The effects of multiple product endorsements by celebrities on consumers’ attitudes and intentions. Journal of Consumer Research 20(4): 535–547.

81.

Turin

Riva

Galbiati

Cainelli

(2000) Fast, simple and highly sensitive double-rounded polymerase chain reaction assay to detect medically relevant fungi in dermatological specimens. European Journal of Clinical Investigation 30: 511–518.

82.

Van der Heijden

(2004) User acceptance of hedonic information systems. MIS Quarterly 28(4): 695–704.

83.

Van Eck

(2006) Digital game-based learning: It’s not just the digital natives who are restless. EDUCAUSE Review 41(2): 16–30.

84.

van Staalduinen

J. P.

de Freitas

(2011) A game-based learning framework: Linking game design and learning outcomes. In: Khyne

M. S.

(ed.) Learning to play: Exploring the future of education with video games, New York, NY: Peter Lang, pp. 29–45.

85.

Venkatesh

Davis

F. D.

(1996) A model of the antecedents of perceived ease of use: Development and test. Decision Sciences 27(3): 451–481.

86.

Wang

Y. S.

Wang

H. Y.

Shee

D. Y.

(2007) Measuring e-learning systems success in an organizational context: Scale development and validation. Computers in Human Behavior 23(1): 1792–1808.

87.

Wouters

van der Spek

E. D.

van Oostendorp

(2009) Current practices in serious game research: A review from learning outcomes perspective. In: Connolly

Stansfield

Boyle

(eds) Games-based learning advancements for multi-sensory human computer interfaces: Techniques and effective practices, New York, NY: Information Science Reference, pp. 232–250.

88.

Yates

B. T.

Wagner

J. L.

Suprenant

L. M.

(1997) Recall of health-risky behaviors for the prior 2 or 4 weeks via computerized versus printed questionnaire. Computers in Human Behavior 13(1): 83–110.

89.

M. Y.

Hwang

(2003) Predicting the use of web-based information systems: Self-efficacy, enjoyment, learning goal orientation, and the technology acceptance model. International Journal of Human-Computer Studies 59(4): 431–449.

90.

Yousafzai

S. Y.

Foxall

G. R.

Pallister

J. G.

(2007) Technology acceptance: A meta-analysis of the TAM: Part 1. Journal of Modelling in Management 2(3): 251–280.

91.

Yusoff, A., Crowder, R., & Gilbert, L. (2010). Validation of serious games attributes using the technology acceptance model. Proceedings of the 2nd International IEEE Conference on Serious Games and Virtual Worlds for Serious Applications, 45–51.