Revista Electrónica de Investigación Educativa

Vol. 18, Núm. 3, 2016

Process for Modeling Competencies for Developing Serious Games

Arturo Barajas Saavedra (*) abarajas@correo.uaa.mx
Francisco Javier Álvarez Rodríguez (*) fjalvar@correo.uaa.mx
Jaime Muñoz Arteaga (*) jmauaa@gmail.com
Ana Cecilia Oviedo de Luna (*) oviedo.ana@gmail.com

(*) Universidad Autónoma de Aguascalientes

(Received: January 22, 2015; accepted for publishing: July 28, 2015)

Abstract

Nowadays, serious games play an outstanding role in the adoption of competencies since they provide immersion, fun, motivation and a high level of engagement among users. This research proposes a process called “Competency-based Decomposition” that allows the identification and modelling of syllabus competencies for the development of serious games to ensure the correct implementation of pedagogical aspects. This is extremely important to ensure a high level of adoption of the competencies by game users. The focus of this research is to guarantee the correct implementation of pedagogical aspects (learning activities and content, expected learning, etc.) in the production of serious games. This article presents the theoretical foundations of the research and details of the competency-based decomposition process, including the application process and its application in mathematics classes for the sixth grade of elementary school. The results obtained in this research are the successful identification of six competencies and 19 areas of knowledge, and the production of 50 short serious games based on the identified competencies and areas of knowledge. These games have been tested on students, resulting in an improvement in learning.

Keywords: Educative technology, short serious games, competencies.

I. Introduction

General basic education is geared towards achieving better levels of intelligence, feelings and personality in students. With the aim of complementing these efforts, several strategies have been implemented to incorporate technology (equipment, educational software, Web 2.0 tools, among others) within both public and private schools from elementary to higher education. (Instituto Nacional de Estadística y Geografía [INEGI], 2009)

However, despite the efforts made by teachers in elementary school, it is clear – based on the ENLACE (the Spanish initials for the National Assessment of Academic Achievement in Schools) results– that elementary school students’ level of education is barely improving with regard to basic topics in mathematics, science, and Spanish (Secretaría de Educación Pública [SEP], 2013).

There is a need to integrate more efficient mechanisms in a non-formal environment for students, and increase learning opportunities beyond the classroom. Video games are a mechanism with great potential.

One aspect that needs solving in order to achieve better learning methods is the production of educational content. In recent years, this problem has become a topic of discussion among the international research community (Ibrahim & Jaafar, 2009), and the use of serious games has been proposed as a way to reduce the gap. Researchers have published multiple models and frameworks, including development processes.

However, a literature review of this research clearly shows that no author has explicitly clarified how to implement the pedagogical aspects in educational digital resources, nor has previous research clarified how to manage the development of these resources in order to finish projects efficiently and produce quality content. Quality is given in terms of user satisfaction and the full implementation of learning needs (learning content, learning activities, expected learning, pedagogical aspects and competencies). Efficiency is given in terms of rework.

II. Problem Outline

The implementation of the “Business-Academia-Government Linkage Model for the Development of IT Capabilities of Human Resources” project (known in Spanish as “Modelo de Vinculación Empresa-Academia-Gobierno para el Desarrollo en Capacidades de Capital Humano en Tecnologías de la Información”) identified the need to convert a set of competencies (and areas of knowledge) into learning needs in order to produce educational digital resources (Velázquez, Álvarez, Garza, Sicilia, Mora, & Muñoz, 2011). At that time, our research team carried out a first approach to competency-based decomposition, resulting in the production of about one thousand learning objects.

From the experience gained during the product quality management in the project, it has been identified that those educational digital resources produced with ad hoc processes do not guarantee efficient completion of development or quality resource production, and neither do they ensure that the learning needs are implemented entirely to facilitate the adoption of competencies, because that development depends on the degree of experience of developers in different areas of knowledge (pedagogy, digital content design, game design, process management, etc.) (Barajas, A., Álvarez, F., Muñoz, J., Santaolaya, R., & Collazos, C., 2014; Álvarez-Rodríguez, Barajas-Saavedra, & Muñoz-Arteaga, 2014).

This paper presents the improved and formalized process to achieve this transformation by addressing the problems related to the correct fulfillment of the learning needs (learning content, learning activities, expected learning, pedagogical aspects and competencies) through the use of the process called “Competency-based Decomposition” (CBD), which, as said in the previous paragraphs, is a proven, successful way to accomplish the production of digital educational resources – in this case, short serious games.

The following sections present the basic concepts and definitions used in this research to design the CBD process.

III. Review of literature

This section presents a review of the literature of competencies related to serious games, in order to establish theoretical concepts that support this research. Furthermore, the definitions and concepts form the foundation for the development of short serious games.

3.1 Competencies

In terms of approach, there have been three main traditions in competency research since the middle of the last century (Mulder, Weigel, & Collins, 2007) :

1. The behaviorist approach stresses the importance of observing successful and effective job performers and determining what differentiates them from their less successful counterparts.
2. The generic approach is aimed more at identifying the common abilities that explain variations in performance.
3. The definition of competency in the cognitive approach includes all of the individuals’ mental resources used to master tasks, acquire knowledge and perform well.

Any definition of the term "competency" should include a combination of three elements: a) information, b) skill development and, c) turning it into action, in a unique situation. The best way to observe a competency is in the combination of these three aspects, which means that all competencies require the mastery of specific information, while calling for the development of a skill, or rather a skill set, derived from information processes, but in a problem situation that is a real unique situation where competition can be generated (Díaz, 2006).

The definition of “competencies” used for this research was created from the previous definitions and concepts published by Diaz (2006) and Mulder et al. (2007) and is as follows: “Competencies are all mental resources of individuals that are used to master tasks, acquire knowledge and perform well in some specified abilities with a certain skill level.”

3.2 Short Serious Games (SSG)

A “Short Serious Game” is a videogame that must have the following aspects, regardless of their purpose (training, education, etc.) and competencies:

1. Pedagogical aspects, which include the following:
   1. The learning needs of the individual or group of individuals.
   2. The social and cultural context of the individual or group of individuals.
   3. The learning methodology (includes consideration of the learning model and learning styles). This aspect covers the aspects "Pedagogic considerations", "Learner specification" and "Context" proposed by De Freitas & Jarvis (2006) and De Freitas & Oliver (2006).
2. Technical aspects including:
   1. Considerations for gameplay and the story (Zyda, 2005).
   2. The level of fidelity, interactivity, immersion, fun, etc.
3. Integration aspects that include:
   1. Considerations for game-based learning (Martens, Diener, & Steffen, 2008).
   2. Considerations for the inclusion of material in formal classes.
   3. Context-based considerations for the implementation of digital educational resources (De Freitas & Jarvis, 2006).

Bearing in mind the analyzed literature on games and learning objects, a non-exhaustive set of basic features that represent a good starting point to achieve a usable product with a good grade of quality were identified:

1. Short and focused on a single area of knowledge;
2. A pedagogically-evaluated graphical user interface;
3. Cases with formal reasoning;
4. Randomly-generated cases;
5. Challenging content that generates competition between students using the game.

3.3 Related Work

This section shows the main ideas of the reviewed articles related to this research.

The article by Tejada (2005) emphasizes that education should appoint a special economic resource for the production of learning materials, and stresses the importance of using digital learning resources. It also establishes the means and strategies for assessment of learning by competencies.

Dávila Balcarce et al. (Dávila & Velásquez, 2007) show the positive results of an evaluation of two collaborative games as a methodological resource for education. It is also stressed that for the improvement of the tested game applications for learning, it is necessary to create better educational content.

Garrido (2013) states that video games are a systems environment that shows one way to configure a set of features that are pedagogically desirable and expected.

Almiron & Porro (2014) established that the use of ICT in education is a good learning strategy. They also established that students have changed and they are not subjects for the current teaching system.

The literature review reveals that there is a need for the production of digital educational resources. Such is the case that a number of assessment tools are proposed at different educational levels, and different actors are involved in the teaching-learning process (teachers, professionals who are evaluated, students, and even parents). However, no paper proposes a way to properly develop resources and the literature makes the assumption that the production of educational (digital) resources is a competency that teachers have already fully acquired.

Therefore, this paper proposes a mechanism that ensures that this competency is acquired by those who design or produce educational (digital) resources, so that students receive and adopt competencies and learning outcomes.

The following sections present the CBD process, its application, and an example of its use including the results of its application to a real subject.

IV. Competency-based decomposition: a process for modeling competencies

The CBD process is a mechanism used to match a formal competency with non-formal content, identifying the aspects and factors that should be implemented in the production of the game so it may satisfactorily cover the expected competencies for a given school grade and thus the quality of the serious game and the fulfilment of the learning needs may be guaranteed. This procedure sets out to completely cover the content and outcomes that make up a subject in a syllabus, thus ensuring the appropriation of knowledge and learning outcomes for a particular competency.

To perform the CBD it is necessary to complete the following steps for each subject to be analyzed (figure 1).

1. Identify the Standards, Goals and Graduate Profiles (SGGP) of the subject analyzed. This step is very important as the products of the subsequent steps must be aligned to these elements.
2. Identify the contents and expected outcomes of the syllabus.
3. Group, in areas of knowledge, the contents and expected outcomes in accordance with the SGGP.
4. Organize knowledge areas in accordance with the SGGP.
5. Identify competencies from the grouping of knowledge areas.
6. Organize competencies and their knowledge areas in accordance with the SGGP.

Figure 1. Graphic view of the competency-based decomposition process

It is very important to stress that every competency and its knowledge areas must be attainable from a software engineering perspective, since, for example, the syllabus for sixth-grade mathematics published by the Ministry of Education provides a set of math competencies, namely:

1. Solve problems independently.
2. Communicate mathematical information.
3. Validate procedures and results.
4. Use techniques efficiently.

All of the above are “competencies for life”, which, from a software engineering perspective, are very complex to manage and measure due to their multifactorial nature.

The following paragraphs show the application of this CBD process, used in this research to identify all the competencies for sixth-grade mathematics in order to produce games.

V. Methodology

This section will show step-by-step the process used to apply the CBD using the sixth-grade mathematics syllabus in elementary schools in Mexico.

Step 1

The following figures show the standards (Figure 2), goals (Figure 3) and graduate profiles (Figure 4) from the syllabus analyzed.

Figure 2. Standards for the study of mathematics for elementary education, established in the 2011 syllabus published by the Mexican Ministry of Education (Secretaría de Educación Pública [SEP], 2011).

Figure 3. The purposes or goals of the study of mathematics for elementary education established
in the 2011 syllabus published by the Mexican Ministry of Education (SEP, 2011)

Figure 4. Graduate profile in mathematics for elementary education established in the 2011
syllabus published by the Mexican Ministry of Education (SEP, 2011)

Step 2

All the subjects that make up a school grade may be found in SEP (2011). Each grade is divided into blocks.

Each block is made up at its first level by the central axis, at its second level by the topics, and at its third level by the contents. This can be seen in Table 1. It should be noted that there are only three axes in the figure. This is due to the fact that the fourth axis (“Attitude towards the study of mathematics”) is implicitly evaluated by the other three axes.

Step 3

In order to perform this step, you must review in detail the contents and expected outcomes. For example, as may be observed in Table 1, in the “Number sense and algebraic thinking” axis there are two sections of content: 1) Addition problems and 2) Multiplication problems; so they can be grouped into an area of knowledge called “Operations”.

Table I. Extract of the syllabus for sixth-grade math in elementary school

Note : This figure shows the competencies to be enhanced, the expected learning, the central axes, the topics and the contents for Block I. The syllabus is made up of five blocks. Note that the fourth axis is not shown as it is evaluated implicitly in the other three. (SEP, 2011)

After identifying the contents and expected outcome, the grouping resulted in the following areas of knowledge:

Step 5

You must name the previous groups with competency names according to the SGGP. The identified competencies are:

Results of using the produced short serious game (knowledge assessment)

After the production phase of the short serious games, the team proceeded to test them in order to study the impact on learning for students exposed to this learning strategy. The participants were a group of 29 students from the sixth grade of elementary school at the “Federal Rural Cuauhtémoc” elementary school located in La Paz, Ojuelos, Jalisco. The children from this school, and the school itself, have many needs, so the use of IT to access information helps to alleviate some of them.

The process followed for the test was as follows (Hernández Sampieri, Fernández, & Baptista, 2010):

1. Potential schools were identified.
2. Tests were- designed for initial and control evaluations. The tests were designed to evaluate students’ level of knowledge in the following areas: areas, handling of solid figures, fractions, shapes and polygons, and the cross product.
3. A school was selected.
4. The group of students was selected. The group was divided into two parts, taking into account that in both groups, the students’ average grade must be equally distributed, meaning that the group was divided according to the average grades of the students.
5. An initial evaluation was applied to all students.
6. The test group used games in one-hour sessions twice a week for four weeks.
7. At the end of eight sessions, a control test was applied to identify the impact of game use.
8. The collected data was analyzed with SPSS software.

The collected data allows trends in students’ scores to be determined before and after they use short serious games. The overall findings are graphically depicted in figure 7 where the dotted line displays the results obtained during the initial examination. The solid line displays the results of the evaluation performed after the use of short serious games.

Figure 7. Students’ overall results in the different areas of knowledge tested.

As the reader can see, there are improvements in four knowledge areas, this is due to the fact that students achieved a higher level of acquisition of the competencies implemented in the short serious games they used.

The cross product game, which shows a decrease in the acquisition of the competency, was developed with a question bank instead of randomly-generated problems, so the students memorized this question bank and did not achieve the intended competency acquisition.

VII. Discussion

The correct implementation of the competencies and the characteristics of the short serious games are extremely important because this is the only way to guarantee that users will achieve a higher level of adoption of the competencies implemented in the games.

As the reader can see, the cross product game did not achieve its purpose of facilitating student adoption of the competency due to a deviation in its development.

VIII. Conclusions and future work

The CBD process enables clear identification of the competencies to be implemented in short serious games, as in many cases this aspect is not taken into account during the development process (if a development process exists). The CBD process takes common elements in the syllabus as inputs, and turns them into programmable and measurable competencies (learning needs) for short serious games.

Once the competencies have been identified, the short serious game development process is capable of building high-quality software products that are based on those learning needs and follow the correct implementation of the competencies.

This research has created and tested, with real-world students, a large set of short serious games, and achieved an improvement in competency adoption due to the correct implementation of learning needs into the games.

The established objectives of the research were:

1. To create a process that makes it easier for teachers and pedagogues to transform a syllabus into competencies and areas of knowledge and develop all the materials and resources needed to teach.
2. To test the process on a real subject and develop a set of short serious games from the identified competencies and areas of knowledge.
3. To test the developed short serious games with real students to determine if there is any improvement.

Finally, we can say objectives have been fulfilled given that we created the CBD process and applied it to a real subject, developed a set of short serious games and tested these games with students.

The next step in this research is to use this CBD process to review a bachelor’s degree to ensure this process can be applied in any context.

Referencias

Almiron, M. E., & Porro, S. (2014). ICT in teaching: an analysis of cases. Revista Electrónica de Investigación Educativa, 16(2), 152-160. Retrieved from http://redie.uabc.mx/redie/article/view/341

Álvarez-Rodríguez, F., Barajas-Saavedra, A., & Muñoz-Arteaga, J. (2014). Serious game design process, study case: sixth grade math. Creative Education, 5(9), 647-656. doi: 10.4236/ce.2014.59077

Barajas, A., Álvarez, F., Muñoz, J., Santaolaya, R., & Collazos, C. (2014). A serious games development process using competency approach. Case study: elementary school math. In A. I.-O. (AIPO) (Ed.), XV International Conference on Human Computer Interaction (pp. 572-579). España: AIPO.

Dávila, G., & Velásquez, A. (2007). Evaluation of the application of collaborative games: “Devorón and Temporal”. Revista Electrónica de Investigación Educativa, 9(2). Retrieved from http://redie.uabc.mx/redie/article/view/164

De Freitas, S., & Jarvis, S. (2006). A framework for developing serious games to meet learner needs. I ’06: Proceedings of the International Conference on Interservice/Industry Training, Simulation and Education.

De Freitas, S., & Oliver, M. (2006). How can exploratory learning with games and simulations within the curriculum be most effectively evaluated? Computers & education, 46(3), 249-264.
doi: 10.1016/j.compedu.2005.11.007

Díaz, A. (2006). El enfoque de compentecias en la educación. ¿Una alternativa o un disfraz de cambio? Perfiles Educativos, 111, 7-36. Retrieved from http://www.iisue.unam.mx/perfiles/perfiles_articulo.php?clave=2006-111-7-36&url=2006/n111a2006/mx.peredu.2006.n111.p7-36.pdf

Garrido, J. M. (2013). Strategy video games: some principles for teaching. Revista Electrónica de Investigación Educativa, 15(1), 62-74. Retrieved from http://redie.uabc.mx/redie/article/view/330/516

Hernández Sampieri, R., Fernández, C. & Baptista, P. (2010). Metodología de la investigación. México: McGRAW-HILL.

Ibrahim, R., & Jaafar, A. (2009). Educational Games (EG) design framework: combination of game design, pedagogy and content modeling. 2009 International Conference on Electrical Engineering and Informatics ICEEI '09, 1, 293-298. doi:10.1109/ICEEI.2009.5254771

Instituto Nacional de Estadística y Geografía (2009). Enciclomedia ¿Qué es? Enciclomedia, una forma diferente de aprender y enseñar . Retrieved from http://www.inegi.gob.mx/inegi/contenidos/espanol/ciberhabitat/escuela/enciclomedia/

Martens, A., Diener, H., & Steffen, M. (2008). Game-based Learning with computers-learning, simulations, and games. In Z. Pan, A. D. Cheok and W. Müller (Eds.),Transactions on Edutainment I (pp. 172-190). Liverpool, UK: Springer. doi: 10.1007/978-3-540-69744-2_15

Mulder, M., Weigel, T., & Collins, K. (2007). The concept of competence in the development of vocational education and training in selected EU member states: a critical analysis. Journal of Vocational Education & Training, 59(1), 67-88. doi:10.1080/13636820601145630

Secretaría de Educación Pública (2011). Programas de estudio 2011. Guía para el maestro. Educación Básica. Primaria. Sexto grado. México: SEP. Retrieved from http://www.curriculobasica.sep.gob.mx/images/PDF/prog_primaria/PRIM_6to2011.pdf

Secretaría de Educación Pública (2013). Estadísticas ENLACE 2012. Evaluación Nacional del Logro Académico en Centros Escolares Educación Básica. Retrieved from http://www.enlace.sep.gob.mx/ba/

Tejada, J. (2005). Competencies-based work in the practicum: how to organize and evaluate it. Revista Electrónica de Investigación Educativa, 7(2). Retrieved from http://redie.uabc.mx/redie/article/view/192

Velázquez, C. E., Álvarez, F. J., Garza, L., Sicilia, M. Á., Mora, J. M., & Muñoz, J. (2011). Una experiencias en el desarrollo masivo de objetos de aprendizaje empleando parámetros de calidad y un proceso de gestión bien definido. IEEE-Revista Iberoamericana de Tecnologías del Aprendizaje, 6(4), 155-163. Recuperado de http://rita.det.uvigo.es/index.php?content=Num_Pub&idiom=Es&visualiza=1&volumen=6&numero=4&orden=desc

Zyda, M. (2005). From visual simulation to virtual reality to games. Computer, 38(9), 25-32. doi:10.1109/MC.2005.297