Parameter adaptive system to learning processes for emerging serious games using cultural algorithms

Abstract

One of the general problems of serious games is their poor ability to adapt to players while they play. One response to that need has been emerging serious games, which allow for real-time adjustment of serious games. To do this, they can use different approaches such as adjusting their strategies, and their plots, among others. In this paper, an adaptive system of parameters for emerging serious games is proposed. This adaptive system enables the emergence of parameters in a serious game to adapt it to an educational environment where it takes place. In particular, this adaptive system uses a cultural algorithm to modify the parameters of the emerging serious games in real-time based on the information of the learning process where it is used. This system is evaluated in a smart classroom for the parameter adaptation of several serious games used in it. Thus. in each of them, it is able to obtain the optimal values of the serious game parameters that allow the players to achieve the objectives of the serious game (a minimum number of points to achieve) in an optimal playing time. The most relevant scientific contribution of the work is the development of an architecture based on Cultural Algorithms to adapt the parameters of a serious game to the characteristics of the players. The results are very encouraging because the serious games are adapted to the behavior of the students in the smart classroom.

Keywords

emerging serious games parameter adaptive system smart classroom cultural algorithm

1 Introduction

Dynamic Game Balance is a domain in a videogame to keep the user interested from the beginning until the end of the game.¹ For that, the parameters, scenarios and behaviors in the videogame are adjusted in real-time, adapting it to the player's ability. In this way, the players are not bored (when the game is too easy) or frustrated (when it is too complex). A game engine must include this capability, but in general, there are currently no game engines that can perform this task. One possibility is to use the emerging serious games (ESG) approach, which has as a premise that the dynamics of serious games (SG) emerge from the behavior of the players. Now, there are currently no engines for ESG.^2,3

Specifically, an ESG combines two areas of study, emerging games and serious games. Aguilar et al.⁴ define an emerging game as a game that can be executed without explicit rules, autonomously, adapting to the players. In addition, a serious game has a motivating entertainment environment, in a way that players “learn by doing” using their errors, and challenges according to the level.^5,6 In an ESG, the story is going to emerge in relation to the environment where the game is used.^2,3 In an ESG its engine must enable different forms of emergence^2,4 with serious game characteristics.³ In this way, the ESG places the player in a context to motivate reaching goals different to only pure fun.

In an ESG, there are several adaptations that can be carried out, at the level of the plots, the strategies used in the game, and the parameters that characterize it, among others. To enable these various settings, in,⁶ a layered ESG engine has been proposed. Now, the specific problem studied in this article has to do with how to optimize its parameters during the game, to adapt them to the characteristics/profiles of its players.

1.1 Related works

Next, we present some articles close to our proposition: Figueira et al.¹ describe a dynamic approach to adjust a game to the current skills of the players. They define different models of players to be used by the game. Rasim et al.⁷ use serious games in a learning process to study theoretic concepts. The serious games are adjustable to the players. With respect to the utilization of intelligent computing techniques in video games, Shafi et al.⁸ analyzed the most used intelligent computing techniques, among which are neural networks, genetic algorithms and learning classifier systems. Singh et al.⁹ adapt a two-level game using a multilevel architecture: in the first level players interact, and in the second level players are divided according to certain rules. The adaptation is defined in terms of the participant number in a game. Singh et al. defined an architecture based on a multi-population cultural algorithm, in which the fitness is determined based on the interactions between the members of a group in an evolutionary game. Yang¹⁰ proposes the generation of new artificial players using learning and imitation operators. A learning operator adjusts behaviors and strategies by studying historical information. An imitation operator can replay the actions and strategies of other players to enhance the performance of a player.

Also, Reynolds et al. have defined a method of knowledge distribution in,¹¹ commented in section 1.1, a new. Jamieson et al.¹² describe games that have combinatorial optimization problems in their dynamics: the first game incorporates resource-gathering aspects in their strategies. The second game uses information from the characters (they are runners), such as jumping power, agility, and speed, to determine the best solutions for the game (e.g., the path to follow). Aguilar et al.³ describe the adaptation of the radius of the “Metropolis” emergent game. The radius determines the effect of a building in others, and is modified in real-time to allow the urban emergence, so that the buildings with a short radius are preferred by the players. Waris et al.¹³ define an approach for the management of social networks based on cultural algorithms for the distribution of knowledge, allowing the emergence of knowledge communities based on distribution mechanisms for competition and cooperation among the members. Khodabakhshian et al.¹⁴ adjust the parameters of the metaheuristic used in the game (the power system stabilizer technique), using a cultural algorithm. Abdolrazzagh-Nezhad et al.¹⁵ model a multiple objective attributes reduction using a cost function that minimizes the number of attributes using the rough set theory and a cultural algorithm.

Equally, maintaining player attention is very crucial in video games. For that, Dynamic Difficulty Adjustments have been proposed to automatically detect the skills of a player, and accordingly, adjust the challenge level in the game. In¹⁶ Demediuk et al. propose four Dynamic Difficulty Adjustments based on Artificially Intelligence (AI), which allow adapting in real-time the difficulty level to the players’ skills. Finally, Shaker et al.¹⁷ define computational models of players’ preferences, skills, or behaviors. They describe different forms to collect and encode the data about the players, and model this data. Arif, et al.¹⁸ study introduces an Adaptive Scenario Selection (ASS) system, utilizing a finite state machine driven by an artificial neural network (ANN). ASS selects scenarios based on player preferences like work, interests, origin, group, and repetition, employing an ANN. ASS accurately chooses game scenarios for players 67.25% of the time. An adaptive serious game for cybersecurity education is proposed by Hodhod et al.¹⁹ This study develops a game measuring improvement within itself to provide meaningful data. Results show positive participant feedback: 79% learned new things, 84% were engaged by the story, 68% had fun, and 84% would recommend the game. Such games can enhance cybersecurity awareness and mitigate cyber threats effectively. Araujo et al.²⁰ provide a framework to guide the creation and assessment of adaptive serious games. The framework emphasizes learning outcomes and adaptation. Triangulation evaluation involving focus groups and expert interviews validates the framework's effectiveness. Hare and Tang²¹ categorize player modeling and game adaptation methods from the past decade, offering comparisons and insights for future research and development. The paper provides a roadmap for researchers and developers to enhance the effectiveness of adaptive SGs, ensuring they continue to evolve as a powerful tool for education and training.

In general, none of the previous works are based on the concept of ESG, which allows a holistic vision of adaptation. Additionally, they are very focused on adjusting a specific aspect of the game (for example, scenarios).

1.2 Research context

This work is based on an ESG engine that follows a hierarchical layered architecture comprising a videogame engine nucleus, a videogame emergence sub-system, and a videogame adaptation sub-system.^4,6,22–24 The engine nucleus contains typical videogame components. The emergence and adaptation sub-systems manage emergence in the ESG using AI techniques and plot sequencing. The emergence sub-system defines the initial ESG, while the adaptation sub-system modifies it during execution. Components include the AI sub-system for smart behaviors and the Emerging Plot Sub-system for plot sequencing. The adaptation sub-system adjusts the ESG during execution, employing strong and weak emergence modules to modify strategies, plots, and objects, among other things. This paper focuses on studying parameter emergence.

In the work²⁵ was proposed a preliminary design of the parameter adaptive subsystem (PAS) for an ESG, with initial ideas about how to use evolutionary algorithms in this system. In this initial work, the need for optimization of the parameters that define a serious game is analyzed, its characteristics, among other things. From there, it is studied how to use evolutionary algorithms in this context. Now, the article does not present any implementation, nor does it incorporate case studies of serious games in real contexts, to evaluate their impact.

1.3 Our contribution and organization of the paper

This article performs a detailed design and describes the implementation of the PAS of this ESG engine. Additionally, the paper presents in detail case studies to analyze the behavior of our system in learning environments. The PAS is one of the components of the adaptive video game subsystem⁴ of the ESG engine, which allows adapting the parameters of an ESG to the players. The PAS has been implemented using cultural algorithms^14,15 because they allow the optimization of parameters using a collective learning process. Finally, the paper presents a case study to demonstrate the ability of the PAS in a smart classroom (called in the paper SaCI, for its acronym in Spanish). The main contributions of this work are:

The proposal of a mechanism to adapt the parameters of an emerging serious game to the characteristics of the players. In particular, the values of the game parameters are adjusted in real-time according to the behavior of the players.

The definition of an approach to use the mechanism in a smart classroom, in such a way as to exploit the knowledge about the profile of the students to adapt the serious game.

The implementation of an adaptive parameter mechanism based on Cultural Algorithms that combines different types of knowledge (historical, current) to make better adjustments to the parameters.

This article is organized as follows: Section 2 defines the theoretical framework, and Section 3 describes our ESG engine architecture. Section 4 presents the design of the PAS using cultural Algorithms. In Section 5 is carried out the analysis of the behavior of the PAS and Section 6 describes a case study in a smart classroom. In Section 7 is carried out a comparison of our proposal with previous works, and finally, Section 8 discusses the conclusions and future works.

2 Theoretical framework

2.1 Parameter emergence

In an ESG, the emergence of parameters modifies its abilities, scenarios, and characteristics, among other things, which can generate new objects, stories, etc..^2,3 The modifications can include new rules in the game; e.g., in Megaman to modify the powers of the virtual players, in dominoes to change the direction of the game (play clockwise), and in Metropolis to modify the radius that defines the impact between the buildings.

The parameter emergence modifies the game (difficulties and complexity, etc.) according to the level of each player (knowledge degree, etc.). It is very important in the learning context where the game must adapt to the students (learning style, emotions, etc.). For example: the parameter to modify in a memory game could be the couples to discover, such that for preschool kids should be 4 couples, first grade kids 8 couples, and so forth. In this way, the ESG complexity is augmented according to the student level.

This adaptive behavior in an ESG must be the product of simple rules used by the ESG engine to manage the ESG. This is known as strong emergence due to it generates deep modifications in the ESG.^26,27

2.2 Cultural algorithms

It is an evolutionary computing algorithm proposed by Reynolds^11,13 that exploits specific knowledge of the problem to solve optimization problems. It can be described as a cultural inheritance process on two layers (see Figure 1): the macro-evolutionary layer that stores the social knowledge built through the generations, and is utilized to define the general behavior of the members of a population; and the micro-evolutionary layer that exploits the genetic material inherited by the population for its improvement.

Especially, a cultural algorithm is composed of^11,13,28:

Population Space: it is a population (set of individuals) and each member has a fitness according to its properties. These individuals can generate descendants using genetic operators, and be replaced by the best ones. This space utilizes: i) A Fitness Function: to determine the individual quality; ii) Genetic Operators: to generate new individuals. The typical operators are mutation and crossover.

Belief Space: it saves the knowledge generated by individuals through generations. This information is used by the population space to improve its quality. There are different knowledge in the belief space^11,13,28,29:

Normative: it defines, for each variable, its best values.

Situational: it defines the knowledge of the context: events, best values for a specific situation, better/worse solutions of the past, etc.

Historical: it defines the knowledge about temporal patterns, historical behaviors, etc.

Domain: it defines the knowledge about how to use the individuals, their relationships, etc.

Topographic: it defines the knowledge about spatial patterns, etc.

Communication Protocols: they define the relationships of the population with the beliefs space. They use two functions: i) The Influence function: it defines the mechanisms that have the population to utilize the knowledge of the belief space; ii) The Acceptance function: it incorporates individual experiences of the best individuals in the belief space to define the social knowledge.

Figure 1.

A general structure of a cultural algorithm.^11,13

The pseudo-code of a cultural algorithm is:

Start

t = 0 // Generate initial population

Start ESG^t // Enhance initial population

Start B ^t // Initialization of the belief space

Repeat Until (reach an end condition)

Assess ESG ^t

Update (B ^t, Accept (ESG ^t)) // Modify belief space

Modify (ESG ^t, influence (B ^t)) // Change the population

// using the operators and the belief space

t = t + 1;

Choose ESG ^t from ESG ^t ^–1 // Select the best individuals

End

3 ESG engine architecture

This work uses the same ESG engine architecture defined in,^4,6,24 which supports an ESG inside a SaCI (more details in^22,23). The ESG engine follows a hierarchical layered architecture (see Figure 2).

Figure 2.

Emerging serious games engine architecture.

This architecture has a first level that is a typical videogame engine (videogame engine nucleus), a second level that defines the initial ESG according to the environment (videogame emergence sub-system), and a third level that adapts the EGS in runtime (videogame adaptation sub-system) considering different emergence types: parameters, sequence, etc. These levels are described in.^4,6 Below, a general description is given.

Videogame engine nucleus: is the main component for the ESG engine and has six components, which are typical for any videogame (e.g., sound, rendering, and video) (see^4,6 for more details).

Videogame emergence and adaptation sub-systems: they are the levels of the architecture to manage the emergence in an ESG using the next components:

AI Sub-system: it defines smart behaviors in the ESG. For that, it uses AI techniques to enable emergencies in an ESG. In this paper will be utilized the cultural algorithm to control the emergence of parameters.

Emerging Plot Sub-system: it brings out the ESG sequences of plots appropriate for the environment.

Videogame emergence sub-system : it defines the initial ESG to be used. When it is invoked by the Videogame emergence sub-system, the Emerging Plot Sub-system has the next elements (see⁴ for details):

(i)

Topic Manager: it defines the theme dealt in the environment. Then, it determines the goals that the ESG must reach.

(ii)

Videogame Manager: it determines the sub-plots or videogames for the previous topic, to be used from videogame repositories (e.g.,: advergame, edugame, etc.^4,24). These videogames/sub-plots are learning resources. It uses the metadata of Learning Resources stored in the repositories with the information built by the Topic Manager for this task. If this component does not obtain the appropriate videogames/plots, then it invokes the ESG Generation Module.

(iii)

ESG Generation Module : it assembles a new ESG based on the videogames/sub-plots supplied by the Videogame Manager. A preliminary work uses an ant colony optimization algorithm for this task,^4,24 which merges the next three elements of the videogame emergence sub-system, to define an initial ESG.

(iv)

Storyboard: it generates the sub-plots for an ESG.

(v)

Scene Manager: it defines the context for an ESG.

(vi)

Event Systems: it generates the events in the ESG to produce its desired behaviors.

Videogame adaptation sub-system : in the case of the Emerging Plot Sub-engine, it adapts the ESG during its execution. This level enables the emergence in the ESG during its utilization, modifying its features. In particular,⁴ proposes 6 emergence types in an ESG, and in this engine are distributed into two modules:

(i)

Strong Emergence Module : it is defined by three sub-levels:

Strategies: it generates new logistics and tactics.

Sequence: it creates new plots, modifying the context of the game.

Property: it modifies the objects.

(ii)

Weak Emergence Module : it is defined by three sub-levels:

Final: it determines the patterns to finish a game.

Business Model: it defines the emerging services around an ESG.

Utility: it defines the utility or interest of the ESG.

Figure 3 shows the Emerging Plot Sub-engine, which is invoked by both subsystems, which enables the emergence in an ESG. The Videogame emergence sub-system carries out the emergence of the initial ESG appropriate to the current environment. The Videogame adaptation sub-system is used to adapt the ESG during its execution, using different emergence mechanisms: sequences, parameters, etc.

Figure 3.

Emerging plot sub-system.

This paper studies the emergence of parameters. The emergence of plots/sequences has been proposed in previous works.⁴

4 Parameter adaptation system

The videogame adaptation sub-system level enables the adequacy of parameters for an ESG, which produces emerging behaviors. For this task, it uses the AI and Emerging Plot sub-engines. The emergence of parameters modifies the rules in the videogame, and features in objects, using an adaptive process of the parameters. It answers to:

What aspects of an ESG could be parameterized, whose values could adapt to the environment?

How can be defined a learning mechanism of these values from the interaction of the players with the ESG?

4.1 Description of PAS behavior

The next step is to define the spaces of beliefs and population. Particularly, the population contains the different values of the serious game parameters that are sought to be optimized. In each generation, the quality of the individuals is evaluated and 20% of the best are selected to update the belief space. Then, Mutation and crossover operators are used to generate new individuals, which use the knowledge obtained in the belief space to generate new individuals. In the end, the best individual represents the optimized parameters. Now, the different components of the culture algorithm (belief space, adaptation function, communication protocol, among others), must be designed specifically for the PAS. The following section details the design of each one (Figure 4).

Figure 4.

PAS Flowchart.

4.2 Characterization of the cultural algorithm used by the PAS

This work uses a cultural algorithm as the learning mechanism. The specification of its three main components, in the context of the PAS, is detailed below:

Population Space: in our case, each individual represents an execution of the ESG by a player, which can be different each time. Thus, an individual defines the values of the parameters utilized in each execution. In Table 1, ESG_i is the individual i, and P_j the value of the parameters j used at the execution i of the ESG.

Table 1.
Structure of an individual.

ESG_i

P₁

P₂

…

P_j

FO_i

The crossover operator generates new individuals; and the mutation operator exploits the knowledge in the belief space (it uses this cultural knowledge).

Fitness Function: it evaluates the performance of an ESG based on the objective that it must reach. In this work, the best individual must maximize the next fitness function:

F O = Σ_{i = 1}^{n} (a * P A_{i} - b * L E_{i}) / n

(1)

a and b are constants to normalize the units. The times with those parameters in the ESG is n, PA_i is the cumulative score in the ESG (it is linked with if the objective of the ESG is reached), LE_i is the execution time of the ESG according to the time, rounds, among others, which defines the faster.

Space of Beliefs: in the literature exists five types of knowledge in the beliefs space. This work only utilized four, since we consider that the topology knowledge is not useful in determining our objectives. This knowledge about spatial patterns is not useful in the context of the PAS.

Situational Knowledge: it has successful examples of ESGs. Especially, values (V_j) of the parameters (P_i), and their occurrence rates (IO_j). Additionally, the mean quality of this parameter (C_j), defined as the mean quality of ESGs where is utilized this parameter, calculated with the FO of the individuals where these values were utilized (see Table 2).

Normative Knowledge: it determines the interval of best values for the parameters of an ESG. In Table 3, LS and LI are the upper and lower limits of the parameter P_i.

Domain Knowledge: it defines the domains to use an ESG, with the appropriate values in each domain of its parameters. In Table 4, D_i is the domain i of the parameter set (P_j), and FC_i is the quality function, computed as the mean quality when the ESG has been utilized in this domain, for what is used the FO of its individuals.

Historical Knowledge: they define behavior patterns in an ESG. This knowledge uses the historical information of the game. Particularly, this work considers the events of interest in an ESG. In this case, a vector of the event type is built with the best values for each parameter P_j for what the event k occurs, and the function FE_k defines the quality of the values to produce the event (see Table 5), for what is used the FO for the individuals with these values.

Communication Protocol: they define the relationships of the belief space with the population.

Acceptance Function: in^11,13 define that with 20% of the individuals can be updated the belief space. This paper uses the same idea. The acceptance function updates the different knowledge types: for the normative knowledge, it modifies the values of LI_j and LS_j, ∀_j= 1…u. For the domain knowledge, it is only updated when exits a new domain D_s, or FC_s has changed due to the ideal values, or the ideal values have changed. For situational knowledge, it modified the values of IO_j and C_j, ∀_j= 1…u. Finally, for historical knowledge, it changes the FE_k value for the current ideal values, or the ideal values of the parameters. The updating in each case is different.

Table 2.

Situational knowledge.

P_i

V_j

IO_j

C_j

Table 3.

Normative knowledge.

P₁		P₂		…	P_u
LI₁	LS₁	LI₂	LS₂		LI_u	LS_u

Table 4.

Domain knowledge.

D_i	FC_i	P₁	P₂	…	P_u
		ideal value	ideal value		ideal value

Table 5.

Historical knowledge.

Event_k

P₁

P₂

…

P_u

FE_k

For the normative knowledge, LS_j and LI_j are modified supposing the individuals in the current population have a value greater than LS_j or less than LI_j.

For the situational knowledge, IO_j is modified according to the next expression:

I O j = N O j + I O j

(2)

Where, NO_j is the number of occurrences of the value V_j in the generation. Also, C_j is modified using the next expression:

C_{j} = C_{j} * \bar{m} + \bar{C_{j}} * m

(3)

Where,

\bar{C_{j}}

is the mean of the value

C_{j}

of individuals within 20% of the best individuals from the population with the value of V_j. and

\bar{m}

is the complement of the moment (1 - m), such that m (moment) is defined by:

m = μ / t

(4)

Where, t is the generations (t = 1, 2, 3…), and

μ

is a constant between 0 and 1.

In the historical knowledge, when the ideal values for the Pi parameters change for event k, then Table 5 must update. The ideal values must maximize Ec. (1) for an event k. However, when only the FE_i value of the ideal values must update, then the next expression is utilized:

F E_{i} = F E_{i} * \bar{m} + \bar{F E_{i}} * m

(6)

Where

\bar{F E_{i}}

is the mean value of FEi of the 20% of the best individuals of the population of the event k.

Finally, in the domain knowledge, if there is a new domain i (D_i), then Table 4 must be updated. When the ideal values for an existing domain i are modified, then they are substituted in Table 4. The ideal values maximize Ec. (1) for a domain i. Also, if the value of FC_i (quality function) of the ideal values is updated, then the next expression is utilized:

F C i = F C i * \bar{m} + \bar{F C_{i}} * m

(5)

Where,

\bar{F C_{i}}

is the mean value of FC_i of 20% of individuals of the population in the domain D_i, and FC_i is the current quality function.

b) Influence Function: the mutation operator uses the knowledge in the belief space. Normative knowledge readjusts the ranges of a parameter P_i; situational knowledge is used as a mutation to the best value V_j for parameter P_i; historical knowledge utilizes the ideal values of P_j, to generate the event k, and domain knowledge uses the ideal values of P_j according to the domain where the ESG is used.

5 Analysis of the behavior of the PAS

The objective of this section is to carry out a sensitivity analysis of the different parameters (number of generations, probability of mutation, probability of crossing, etc.) of the cultural algorithm used by the PAS. To do this, in the first part (section 5.1), the possible values of them in a specific game are analyzed, to determine the optimal values of each of the parameters of the cultural algorithm. This allows defining the set of values of the parameters of the cultural algorithm that will be used in the adaptation of serious games. Once the above is done, in the second part of this section (section 5.2), the quality of the parameters proposed for the serious game is evaluated when playing. In other words, the serious game is tested with the parameters proposed by the PAS to determine if the players reach the objectives of the serious game, and at what time (with these two values the quality of the proposed optimization is measured). The latter allows us to determine the appropriate values of the parameters used by the cultural algorithm of the PAS (see section 5.3).

5.1 Calibration of the PAS

This section defines the experiment carried out to evaluate its quality.

Tests of the prototype: its objective is to determine the best values of the parameters of the PAS (particularly, the cultural algorithm), which will be the default values for its correct operation. Specifically, the values to optimize are: the probability of using the operators, the generations, the size of the initial population and the value of µ. This experiment is carried out with an ESG called “Polygons with the Tangram”, obtained from the “Agrega” repository (http://agrega2.educacion.es) (see Figure 5).

Figure 5.

The ESG: “Polygons with the Tangram”.

This ESG consists of solving the Tangram, which is a puzzle game of Chinese origin made up of seven pieces, obtained by dividing a quadrilateral: five triangles (two large, one medium and two small) and two quadrilaterals (one square and one rhomboid), with the aim of forming a figure made up of 13 possible convex polygons (triangle, square, rectangle, rhomboid, isosceles trapezoid, rectangle trapezoid, pentagon trapezoid, pentagon, 4 hexagons). This figure has any of the following shapes: sitting man, polar bear (as shown in Figure 5), puppy, sailboat, swan, black arrow and white arrow.

The parameters of that ESG to consider are:

P₁ = Figures to build

P₂ = Convex polygons that the player has

Calibration Process

The initial limits for each parameter, required to initialize the normative knowledge, are shown in Table 6.

Table 6.

Normative knowledge of the video game “Polygons with the Tangram”.

P₁		P₂
LI₁: 1	LS₁: 7	LI₂: 1	LS₂: 13

The initial values to start the simulation are in Table 7.

Table 7.

Initial values of the pas for the video game “Polygons with the Tangram”.

Individuals	Generations	Crossov. Prob.	Mutat. Prob.	µ	a	B	PA	LE
10	20	0.1	0.1	0.1	100	100	0,1	0,1

In this test only are used situational and normative knowledge. In Figure 6, the tables of the situational and normative knowledge and the best individuals of each generation are shown. In the case of situational knowledge, the values of the parameters P₁ and P₂ come from the best individuals, each one with its occurrence and average quality. As normative knowledge, it is shown the lower and upper limits of the parameters, following the format: [LI₁, LS₁] and [LI₂, LS₂] for each parameter and generation. The figure also presents the best individuals in the generations. The individuals are presented in the format [P₁, P₂], together with the value of their FO.

Figure 6.

Tables with the situational and normative knowledge and the best individuals in each generation for the ESG: “Polygons with the Tangram”.

In Figure 6 is observed how the situational knowledge of both parameters influences the best individuals, which appears from generation 13, remaining so until the convergence of the algorithm.

Now, it is analyzed the sensitivity of the PAS parameters. The first test is by varying the number of individuals in the final population (see Table 8).

Table 8.

Variations in the number of individuals.

Number of individuals	FO of the best individual
30	77.3442
50	90.6724
70	86.7574
80	93.8339
100	90.7341

The results indicate a better FO value for 80 individuals in the population. Therefore, the number of individuals is established at 80.

The second test is by varying the probability of the crossover operator (see Table 9).

Table 9.

Variations of the crossover operator probability.

Probability	FO of the best individual
0.3	95.2848
0.5	96.6373
0.7	92.4736
0.9	96.0042
1.0	96.207

The results indicate that with a crossover probability of 50%, a better FO value is obtained. Thus, the probability of the crossover operator is set to 0.5.

The third test is by varying the probability of the mutation operator (see Table 10).

Table 10.

Variations of the mutation operator probability.

Probability	FO of the best individual
0.3	96.6871
0.5	96.207
0.7	96.2725
0.9	94.6158
1.0	97.7206

The results indicate that with a 100% of mutation probability, a better FO value is obtained. Therefore, the mutation probability is set to 1.0.

The fourth test is by varying the number of generations (see Table 11).

Table 11.

Variation of the number of generations.

Number of generations	FO of the best individual
30	97.7206
50	97.7206
100	99.1939
200	99.8165

The results indicate that with 200 generations, the best FO value is obtained. Therefore, the number of generations is set at 200.

The last test is by varying µ (see Table 12).

Table 12.

VARIATION of µ.

µ	FO of the best individual
0.3	99.1843
0.5	99.1843
0.7	99.3492
0.8	99.3492
0.9	99.3492
1.0	99.1843

The results indicate an increase in the FO value, with µ between 0.3 and 0.7. Then, with µ between 0.7 and 0.9, it begins to decrease. Thus, the µ value is set to 0.7.

5.2 Quality tests

Now, we analyze the quality of the ESG generated by the PAS, using the parameters defined in the previous subsection. The ESG has been played 26 times. For these tests, the quality criteria are:

(a)
Timespan (LE, in seconds): It is assumed that a time limit is established to meet the objectives of the ESG. Initially, 300 s (5 min) are defined, so if this value is exceeded, up to a maximum of 600 s (10 min), then it means that the player took too long to meet the objectives. If it's maxed out, then the goal has not been reached.
(b)
Scope of objectives (PA, cumulative score): The hypothesis is that 1000 points is the value when all the objectives of the ESG are met, while with a value of around 500 points, most of the objectives achieved can be considered reached.
The results of the quality criteria for the ESG “Polygons with the Tangram” are shown in Figure 7.

Figure 7.
Results of the quality test for the ESG “Polygons with the Tangram”.

Figure 7 shows the best individual, with an FO of 986,486, in Generation 110, with the following parameters:
P₁= 7 (player figures)

P₂= 12 (convex polygons)
The FO value was calculated as follows:
$F O = 1000 * 0.99674 - 600 * 0.017090833 = 986.4855$

The normalized PA is = 996.74 ≈ 997 points.

The normalized LE is = 10.2545499 ≈ 10 s.
As can be seen in the FO, with these parameters in the ESG, the students can meet the two quality criteria: duration of the game and objectives achieved.
5.3 Conclusions on the parameters of the PAS cultural algorithm

Sections 5.1 and 5.2 describe the process followed to calibrate the parameters of the culture algorithm that defines the PAS. In this section the experiments were extended to more serious games, to determine a range of desired values for those parameters. Thus, the same process was carried out with twenty serious games taken from the https://aprendomusica.com/const2/27aprendonotas8/aprendonotas8.html and http://agrega2.educacion.es repositories (for example, Catch notes, Music Trivia, etc.) to determine a general behavior of the PAS. After having carried out simulations with these ESGs used in SaCI, optimized by our PAS, the following is concluded about the five parameters of the cultural algorithms:

The ideal number of individuals in the population must be between 75 and 90, and for the number of generations between 150 and 20. In addition, for a smaller value of the population, a greater number of generations is required.

The ideal value of the probability of crossover was between 0.3 and 0.5 and of mutation above 0.9.

The µ value should be between 0.5 and 0.7 to obtain the best results. It was shown through the experiments that being larger (> 0.7) there is a decrease in the FO value of the best individual.

6 Case study

In the previous section, the behavior of PAS in the tangram serious game is explained in detail. In this section, the operation of PAS in the context of SaCI will be shown. The objective of these experiments is to show the behavior of PAS in a smart classroom (see section 6.1), in order to evaluate their ability to adapt a serious game. To do this, the behavior of the players is evaluated using the serious games, latter use the parameters proposed by the PAS. In this case, it is evaluated whether the players are capable of achieving the objectives defined in serious games in adequate time (see section 6.2).

6.1 Application context: SaCI

When a SaCI is described using the multi-agent theory,^22,25 its components are defined as agents, both its software (recommendation systems, virtual learning environment (VLE), etc.), and hardware devices (smartboard, smartphone, laptop, and tablet, etc.). The emerging serious game agent (ESGA) is one of these agents, to autonomously manage the ESGs. ESGA uses the information of SaCI (goals of the current learning process, student profile, etc.) to adapt the ESG to SaCI. For this, ESGA calls the Topic Manager and interacts with other agents of SaCI²⁴: recommender system (RS), academic system (AS), VLE, learning object repository (LOR), etc. (see Figure 8). Also, it interacts with the emerging serious game engine (ESGE) that controls other emergencies in an ESG (e.g., strategies, sequences/plots, etc.).

Figure 8.

ESGA in a SaCI. (Source²³).

6.2 Analysis of the behavior of the PAS

This case study considers a mathematic course in SaCI with 20 students, and it is required an ESG to explain and solve exercises about different topics. Specifically, the topic to be studied is square roots, and the next ESG in the domain of square roots is selected.

6.2.1 Mentally calculate the square root of 6 numbers

It is assumed that initially, the videogame emergence sub-system proposes as the initial ESG the video game of the mental calculation of the square root of 6 numbers, obtained from the “Agrega” repository (http://agrega2.educacion.es) (see Figure 9):

Figure 9.

The ESG for mental calculation of the square root of 6 numbers.

This ESG consists of a scene where six square roots appear (yellow zone), whose radicands are perfect square numbers. The numbers in the blue zone are the solutions. Then, the player will have to place them in the spaces. Once all the tiles (numbers) have been placed in the places, then the ESG will show the word “CORRECT” or “INCORRECT” in the blue zone.

SaCI uses the PAS to define the ideal values for the parameters of an ESG. The ESG “mental calculation of the square root” has 3 parameters:

P₁= Maximum numerical value of the radicand.

P₂= Square roots to calculate (the spaces to place the tiles).

P₃= The tiles (numbers) that the player has (since the player can have more tiles than square roots to calculate).

The quality criteria are the same as defined in the previous section. Particularly, the timespan and the score are defined by the class teacher. In this case, they are defined as:

Timespan: A time limit of 600 s (10 min).

Cumulative score: If the value is greater than 500 points, then the player met most of the ESG goals. The maximum value is 1000 points.

Table 13 shows the first times the ESG was played.

The first student played it with the parameters P₁= 9, P₂= 1 and P₃= 4, with FO = 631.4.

Table 13.

Initial population of the ESG “square root mental calculation” (initial population).

#	P₁	P₂	P3	FO
1	9	1	4	631.4
2	64	1	4	500.4
3	81	4	3	255.6

The FO value was calculated as follows:

F O = 1000 * 0.944 - 600 * 0.521 = 631.4

The normalized PA is = 944 points.

The normalized LE is = 312.6 s.

According to the quality criteria, it is observed that the student reached the ESG objectives (944 points) in approximately 5 min and 22 s (313 s).

The second student played it with the parameters P₁= 64, P₂= 1 and P₃= 4, with FO = 500.4.

The FO value was calculated as follows:

F O = 1000 * 0.774 - 600 * 0.456 = 500.4

The normalized PA is = 774 points.

The normalized LE is = 273.6 s.

According to the quality criteria, it is observed that the student reached the ESG objectives (774 points) in approximately 4 min and 50 s (274 s).

The third student played it with the parameters P₁= 81, P₂= 4 and P₃= 3, with FO = 255.6.

The normalized PA is 414 points and the normalized LE is 158.4 s. According to the quality criteria, it is observed that the student did not reach the ESG objectives (414 < 500 points) in a timespan of approximately 3 min (158 s).

After 200 generations, the evolution of the best individual is observed in Figure 10.

Figure 10.

The best individual for the ESG “square root mental calculation” during the generations.

In Figure 10, the best individual occurs in generation 84, with FO = 950.4. The normalized PA is 978 points, and the normalized LE is 27.6 s.

In particular, the PAS suggests the parameters P₁= 49, P₂= 4 and P₃= 4, so that the player reaches the goals of the ESG “square root mental calculation” (978 points) in the shortest possible time (∼ 28 s).

6.2.2 Learn the C-scale notes

Now, suppose a music course and the topic to study is the “Musical Notes”. Then, the videogame emergence sub-system proposes as ESG “Learn the C-scale notes”, obtained from the link:

https://aprendomusica.com/const2/27aprendonotas8/aprendonotas8.html (see Figure 11)

Figure 11.

ESG: “Learn the C-scale notes”.

The video game “Learn the C-scale notes” consists of recognizing through the sense of hearing, three musical notes, which will be shown when the player presses the correct piano key, at the correct time, determined as a vertical yellow stripe. At first, the musical staff contains: the treble clef, whose function is to determine the height that corresponds to each musical note, according to the position (space or line) they occupy in it, and three musical notes, which serve as a guide to solve the ESG.

The ESG consists of three levels, represented by the gray squares, which change color to green, if the objectives of that level are reached, and red or orange otherwise. This ESG has four parameters:

P₁= Keys that the player has.

P₂= Notes to guess on the staff.

P₃= Penalty for playing poorly.

P₄= Levels.

The quality criteria are the same as defined previously. Particularly, the timespan and the score are defined by the class teacher. In this case, they are defined as:

Timespan: the time limit is 60 s (1 min).

Cumulative score: if the value is greater than 500 points, then the player met the majority of the ESG goals, and the maximum value is 1000 points.

Table 14 shows the initial population:

The first student played it with the parameters P₁= 4, P₂= 2, P₃= 7, P₄= 2 with FO = 781.8, which was calculated as follows:

F O = 1000 * 0.81 - 60 * 0.47 = 781.8

Table 14.

The first time ESG is played.

#	P₁	P₂	P₃	P₄	FO
1	4	2	7	2	781.8
2	7	3	6	2	287
3	7	2	9	2	241.8

The normalized PA is 810 points, the normalized LE is 28.2 s. According to the quality criteria, it is observed that the student reached the ESG objectives (810 points) in approximately 28 s.

The second student played it with the parameters P₁= 7, P₂= 3, P₃= 6, P₄= 2 with FO = 287. For this student, the normalized PA is 290 points, and the normalized LE is 3 s. According to the quality criteria, the student did not reach the ESG objectives (290 < 500 points).

The third student played it with the parameters P₁= 7, P₂= 2, P₃= 9, P₄= 2, with FO = 241.8. For this student, the normalized PA is 270 points, and the normalized LE is 28.2 s. According to the quality criteria, the student did not reach the ESG objectives (270 < 500 points), playing for approximately 28 s.

After 200 generations, the evolution of the best individual is observed in Figure 12, and the best individual appears in generation 141.

Figure 12.

The best individual for the ESG: “Learn the C-scale notes” during the generations.

The best individual appears from generation 141, with FO = 983.4. In this case, the normalized PA is 990 points, and the normalized LE is 6.6 s. Therefore, the PAS suggests the parameters P₁= 4, P₂= 1, P₃= 9 and P₄= 1, so that the player reaches the objectives of the ESG (990 points), in the shortest possible time (∼ 7 s).

7 Comparison with previous works

Table 15 shows a comparison of our approach with other works, according to the next criteria:

Does it allow the parameters of a game to be adapted?

Is it a serious game?

Does the game have any kind of emergence?

Is the proposal part of a game engine?

Does it use a Cultural Algorithm?

Is it a Dynamic Game Balance approach?

Table 15.
Comparison with other works.

Criteria Aguilar et al.³ Figueira et al.¹ Rasim et al.⁷ Waris and Reynolds¹³ Singh et al.⁹ Yang¹⁰ Reynolds and Kinnaird-Heether¹¹ Jamieson et al.¹² Khodabakhshian and Hemmati¹⁴ Abdolrazzagh-Nezhad et al.¹⁵ Present work

A X X X X X

B X X

C X X X X X X X X X X X

D X X X

E X X X X X

F X X X X X X

Criteria	Aguilar et al.³	Figueira et al.¹	Rasim et al.⁷	Waris and Reynolds¹³	Singh et al.⁹	Yang¹⁰	Reynolds and Kinnaird-Heether¹¹	Jamieson et al.¹²	Khodabakhshian and Hemmati¹⁴	Abdolrazzagh-Nezhad et al.¹⁵	Present work
A	X	X			X	X					X
B			X								X
C	X	X	X	X	X	X	X	X	X	X	X
D		X	X								X
E				X			X		X	X	X
F	X	X	X			X			X		X

The works^1,3,7,9,14 use a technique to adapt the parameters to the player's level (criterion A). The works based on cultural algorithms allow the adaption of games to the environment^{11,13–15,25} (criterion E) but they are not emergent games (criterion C). There are no works that describe engines for ESG⁷ (criterion D), and the emergent games developed until now with the capability to emerge the value of the parameters are exclusively for the specific game where are used³ (criterion C).

There are very few works on serious games that adapt their parameters (criteria A and B). Furthermore, they do not present adaptation as a process that emerges as the game is played (criteria A and C). Finally, this is a very innovative use of cultural algorithms, which is capable of using the knowledge it acquires during its adaptations in different games (space of belief) to improve its optimization of parameters in new games (criterion E). On the other hand, few works have considered the problem of the Dynamic Game Balance approach (criterion F), and even fewer have seen emerging processes (criteria C and F). Our proposal is part of an ESG engine (criterion D) to enable a strong emergence of parameters in any ESG according to the given environment (like SaCI).

8 Conclusions

In this paper, we have defined a mechanism for adapting the parameters of serious games in real time, one of the types of adjustments that emerging games require. This mechanism allows players to improve their performance by parameterizing the game appropriately. In particular, this work developed a system that allows the determination of the optimal parameters of an ESG, to adapt to the characteristics of the players. Specifically, it was proposed to use a cultural algorithm for the determination of the optimal parameters, using different knowledge types in an ESG: normative, situational, and historical of the domain. Cultural algorithms allow easily the determination of the behaviors of the players to adapt the ESG to them. With this knowledge, it is possible to have sufficient information about the players, a meta-knowledge that allows the ESG to be adapted by modifying the values of its parameters. As far as we know, there are no works that enable the best values of the parameters in a game to emerge, and less in an ESG, considering the profile of the players (see Table 15). In this way, we have filled the research gap.

The Cultural Algorithm stores the historical knowledge of the parameters of the ESG to optimize their values. The metrics used allow evaluating if the adaptation of the ESG enables the player to achieve the ESG objectives (PA) in a given short time (LE). Thus, we have evaluated the quality of our proposal based on whether the players reach the objectives defined in the ESG in a reasonable playing time.

The paper presents an analysis of the sensitivity of the cultural algorithm parameters, that is, the values of the PAS variables, to determine the default values during the optimization process of the ESG parameters (the search for the best individual). Regarding the case study in a SaCI, the ESG adapts its parameters to the pedagogical context, using the PAS component of the ESG engine. These parameters are obtained efficiently using the cultural algorithms-based PAS. Specifically, the optimal parameters are defined in the best individual in the final population of the cultural algorithm (its chromosome has the optimal values of the parameters of the ESG). With these optimal ESG parameters, players should achieve the learning objectives (PA) in a short time (LE). Thus, the most relevant impact of our system is the adaptive capacity given to the ESG, in such a way as to be able to follow the behavior of the players.

One of the most important limitations is that the information was not taken in real time. Future work should be carried out in smart classrooms that allow the capture of information in real time. Furthermore, future work should analyze how to transfer the knowledge obtained from the behavior of the parameters to other games. It is also relevant to analyze how to integrate other types of knowledge that are allowed in cultural algorithms (such as situational and topographic). Additionally, future work will carry out an in-depth analysis of the impact on the learning process of our approach, particularly, in students in a SaCI. Finally, to have a complete engine for ESG, the integration of all types of emerging adaptation (schemes, strategies and parameters) is necessary. Now, there is a necessity to study the combined effect of the use of different types of emergencies.

Footnotes

Abbreviations

ORCID iD

Jose Aguilar

Author contributions/CRediT

Jose Aguilar: Conceptualization, Methodology, Formal analysis, Writing.

Francisco Díaz: Conceptualization, Methodology, Formal analysis Writing.

Jesús Gutiérrez: Development, Formal analysis, Writing.

Junior Altamiranda: Writing, Supervision; Nelson Pérez: Formal analysis, Supervision; Ángel Pinto: Formal analysis, Supervision.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability

The article does not require specific data. The considered games are available in portals indicated in the work.

References

Figueira

Nascimento

da Silva Junior

, et al. Bing: a framework for dynamic game balancing using provenance. In Proceedings of 17th Brazilian symposium on computer games and digital entertainment (SBGames) 2018, pp. 408–417.

Carrión-Toro

Santorum

Acosta-Vargas

, et al. Iplus a user-centered methodology for serious games design. Appl. Sci 2020; 10: 9007.

Aguilar

Altamiranda

Díaz

, et al. Metropolis: an emerging serious game for the smart city. DYNA 2019; 86: 215–224.

Aguilar

Diaz

Altamiranda

, et al. An adaptive system for emerging serious games using a swarm intelligence algorithm. In IEEE transactions on games, 2021.

Lima

Torres

Ramirez

. A serious game for the preservation of wildlife in danger of extinction in Venezuela. In: Proceeding third national conference on computing, information technology and systems (CoNCISA), 2015, pp. 50–60.

Aguilar

Díaz

Altamiranda

, et al. Metropolis: emergence in a serious game to enhance the participation in smart city urban planning. J Knowl Econ 2021; 12: 1594–1617.

Rasim

Langi1

Munir

, et al. A survey on adaptive engine technology for serious games. Int Semin Math Sci Comput Sci Educ (MSCEIS 2015) AIP Conf Proc. 1708 2016; 1708: 050003–1–050003-9.

Shafi

Abbass

. A survey of learning classifier systems in games. In: IEEE computational intelligence magazine, 2017, pp.42–55.

Singh

Moradian

Kobti

. A multilevel cooperative multi-population cultural algorithm. In 2018 intelligent systems and applications (INISTA) 2018, pp.1–5.

10.

Yang

. Game theory-inspired evolutionary algorithm for global optimization. Algorithms 2017; 10: 1–15.

11.

Reynolds

Kinnaird- Heether

. Population mechanics and cultural algorithms in the development of a cultural engine. In IEEE symposium series on computational intelligence (SSCI) 2017, pp. 1–8.

12.

Jamieson

Grace

Hall

, et al. Metaheuristic entry points for harnessing human computation in mainstream games. Int Conf Online Commun Soc Comput. (OCSC 2013) 2013; 8029: 156–163.

13.

Waris

Reynolds

. Optimizing AI Pipelines: A Game-Theoretic Cultural Algorithms Approach. IEEE Congress on evolutionary computation (CEC), 2018.

14.

Khodabakhshian

Hemmati

. Multi-machine power system stabilizer design by using cultural algorithms. Int J Electr Power Energy Syst 2013; 44: 571–580.

15.

Abdolrazzagh-Nezhad

Radgohar

Najme

. Enhanced cultural algorithm to solve multi-objective attribute reduction based on rough set theory. Math Comput Simul 2020; 170: 332–350.

16.

Demediuk

Tamassia

Raffe

, et al. Monte carlo tree search based algorithms for dynamic difficulty adjustment. In: IEEE conference on computational intelligence and games, pp. 53–59.

17.

Shaker

Togelius

Yannakakis

. The experience-driven perspective. In procedural content generation in games. In: Procedural content generation in games. Switzerland: Springer, 2016, pp.181–194.

18.

Arif

Nurhayati

Karami

, et al. An artificial neural network-based finite state machine for adaptive scenario selection in serious game. Int J Intell Eng Syst 2023; 16: 488–500.

19.

Hodhod

Hardage

Abbas

, et al. Cyberhero: an adaptive serious game to promote cybersecurity awareness. Electronics 2023; 12: 3544.

20.

Araujo

dos Santos

Baptista

, et al. Framework for adaptive serious games. Comput Appl Eng Educ. 2024; 32: e22731.

21.

Hare

Tang

. Player modeling and adaptation methods within adaptive serious games. IEEE Trans Comput Soc Syst 2023; 10: 1939–1950.

22.

Aguilar

Bessembel

Cerrada

, et al. Una Metodología para el Modelado de Sistemas de Ingeniería Orientado a Agentes. Inteligencia Artificial. Revista Iberoamericana de Inteligencia Artificial 2008; 12: 39–60.

23.

Aguilar

Jerez

Exposito

, et al. CARMiCLOC: context awareness middleware in cloud computing. In Proc. 2015 Latin American computing conference (CLEI) 2015.

24.

Aguilar

Altamiranda

Díaz

. Specification of a managing agent of emergent serious games for a smart classroom. IEEE Latin America Transactions 2020; 18: 51–58.

25.

Aguilar; F. Díaz

Altamiranda

Montoya

. An Emerging Serious Game Engine with a Parameter Adaptive System based on an evolutionary approach. In: Proceedings XLVI Latin American computing conference, 2020.

26.

Aguilar

. Introducción a los Sistemas Emergentes, Talleres Grafico. Merida, Venezuela: Universidad de Los Andes, 2014.

27.

Perozo

Aguilar

Terán

, et al. A verification method for MASOES. IEEE Trans Syst Man, Cybern: Part B, IEEE Soc 2013; 43: 64–76.

28.

Aguilar

Garcès-Jimènez

Gallego-Salvador

, et al. Autonomic management architecture for multi-HVAC systems in smart buildings. IEEE Access 2019; 7: 123402–123415.

29.

Morales Escobar

Aguilar

Garcés-Jiménez

, et al. Advanced fuzzy-logic-based context-driven control for HVAC management systems in buildings. IEEE Access 2020; 8: 16111–16126.

Parameter adaptive system to learning processes for emerging serious games using cultural algorithms

Abstract

Keywords

1 Introduction

1.1 Related works

1.2 Research context

1.3 Our contribution and organization of the paper

2 Theoretical framework

2.1 Parameter emergence

2.2 Cultural algorithms

4.1 Description of PAS behavior

Table 1. Structure of an individual.

5.1 Calibration of the PAS

6 Case study

6.1 Application context: SaCI

6.2.1 Mentally calculate the square root of 6 numbers

Footnotes

Abbreviations

ORCID iD

Author contributions/CRediT

Funding

Declaration of conflicting interests

Data availability

References

Table 1.
Structure of an individual.