Revista Electrónica de Investigación Educativa
Vol. 18, Núm. 3, 2016
Process for Modeling Competencies for Developing Serious Games
Arturo Barajas Saavedra (*) abarajas@correo.uaa.mx
Francisco Javier Álvarez Rodríguez (*) fjalvar@correo.uaa.mx
Jaime Muñoz Arteaga (*) jmauaa@gmail.com
Ana Cecilia Oviedo de Luna (*) oviedo.ana@gmail.com
(*) Universidad Autónoma de Aguascalientes
(Received: January 22, 2015; accepted for publishing: July 28, 2015)
How to cite: Barajas, A., Álvarez, F. J., Muñoz, J. y Oviedo, A. C. (2016). Process for modeling competencies for developing serious games. Revista Electrónica de Investigación Educativa, 18(3), 146160.Retrieved from http://redie.uabc.mx/redie/article/view/1071
Abstract
Nowadays, serious games play an outstanding role in the adoption of competencies since they provide immersion, fun, motivation and a high level of engagement among users. This research proposes a process called “Competencybased Decomposition” that allows the identification and modelling of syllabus competencies for the development of serious games to ensure the correct implementation of pedagogical aspects. This is extremely important to ensure a high level of adoption of the competencies by game users. The focus of this research is to guarantee the correct implementation of pedagogical aspects (learning activities and content, expected learning, etc.) in the production of serious games. This article presents the theoretical foundations of the research and details of the competencybased decomposition process, including the application process and its application in mathematics classes for the sixth grade of elementary school. The results obtained in this research are the successful identification of six competencies and 19 areas of knowledge, and the production of 50 short serious games based on the identified competencies and areas of knowledge. These games have been tested on students, resulting in an improvement in learning.
Keywords: Educative technology, short serious games, competencies.
I. Introduction
General basic education is geared towards achieving better levels of intelligence, feelings and personality in students. With the aim of complementing these efforts, several strategies have been implemented to incorporate technology (equipment, educational software, Web 2.0 tools, among others) within both public and private schools from elementary to higher education. (Instituto Nacional de Estadística y Geografía [INEGI], 2009)
However, despite the efforts made by teachers in elementary school, it is clear – based on the ENLACE (the Spanish initials for the National Assessment of Academic Achievement in Schools) results– that elementary school students’ level of education is barely improving with regard to basic topics in mathematics, science, and Spanish (Secretaría de Educación Pública [SEP], 2013).
There is a need to integrate more efficient mechanisms in a nonformal environment for students, and increase learning opportunities beyond the classroom. Video games are a mechanism with great potential.
One aspect that needs solving in order to achieve better learning methods is the production of educational content. In recent years, this problem has become a topic of discussion among the international research community (Ibrahim & Jaafar, 2009), and the use of serious games has been proposed as a way to reduce the gap. Researchers have published multiple models and frameworks, including development processes.
However, a literature review of this research clearly shows that no author has explicitly clarified how to implement the pedagogical aspects in educational digital resources, nor has previous research clarified how to manage the development of these resources in order to finish projects efficiently and produce quality content. Quality is given in terms of user satisfaction and the full implementation of learning needs (learning content, learning activities, expected learning, pedagogical aspects and competencies). Efficiency is given in terms of rework.
II. Problem Outline
The implementation of the “BusinessAcademiaGovernment Linkage Model for the Development of IT Capabilities of Human Resources” project (known in Spanish as “Modelo de Vinculación EmpresaAcademiaGobierno para el Desarrollo en Capacidades de Capital Humano en Tecnologías de la Información”) identified the need to convert a set of competencies (and areas of knowledge) into learning needs in order to produce educational digital resources (Velázquez, Álvarez, Garza, Sicilia, Mora, & Muñoz, 2011). At that time, our research team carried out a first approach to competencybased decomposition, resulting in the production of about one thousand learning objects.
From the experience gained during the product quality management in the project, it has been identified that those educational digital resources produced with ad hoc processes do not guarantee efficient completion of development or quality resource production, and neither do they ensure that the learning needs are implemented entirely to facilitate the adoption of competencies, because that development depends on the degree of experience of developers in different areas of knowledge (pedagogy, digital content design, game design, process management, etc.) (Barajas, A., Álvarez, F., Muñoz, J., Santaolaya, R., & Collazos, C., 2014; ÁlvarezRodríguez, BarajasSaavedra, & MuñozArteaga, 2014).
This paper presents the improved and formalized process to achieve this transformation by addressing the problems related to the correct fulfillment of the learning needs (learning content, learning activities, expected learning, pedagogical aspects and competencies) through the use of the process called “Competencybased Decomposition” (CBD), which, as said in the previous paragraphs, is a proven, successful way to accomplish the production of digital educational resources – in this case, short serious games.
The following sections present the basic concepts and definitions used in this research to design the CBD process.
III. Review of literature
This section presents a review of the literature of competencies related to serious games, in order to establish theoretical concepts that support this research. Furthermore, the definitions and concepts form the foundation for the development of short serious games.
3.1 Competencies
In terms of approach, there have been three main traditions in competency research since the middle of the last century (Mulder, Weigel, & Collins, 2007) :
 The behaviorist approach stresses the importance of observing successful and effective job performers and determining what differentiates them from their less successful counterparts.
 The generic approach is aimed more at identifying the common abilities that explain variations in performance.
 The definition of competency in the cognitive approach includes all of the individuals’ mental resources used to master tasks, acquire knowledge and perform well.
Any definition of the term "competency" should include a combination of three elements: a) information, b) skill development and, c) turning it into action, in a unique situation. The best way to observe a competency is in the combination of these three aspects, which means that all competencies require the mastery of specific information, while calling for the development of a skill, or rather a skill set, derived from information processes, but in a problem situation that is a real unique situation where competition can be generated (Díaz, 2006).
The definition of “competencies” used for this research was created from the previous definitions and concepts published by Diaz (2006) and Mulder et al. (2007) and is as follows: “Competencies are all mental resources of individuals that are used to master tasks, acquire knowledge and perform well in some specified abilities with a certain skill level.”
3.2 Short Serious Games (SSG)
A “Short Serious Game” is a videogame that must have the following aspects, regardless of their purpose (training, education, etc.) and competencies:
 Pedagogical aspects, which include the following:
 The learning needs of the individual or group of individuals.
 The social and cultural context of the individual or group of individuals.
 The learning methodology (includes consideration of the learning model and learning styles). This aspect covers the aspects "Pedagogic considerations", "Learner specification" and "Context" proposed by De Freitas & Jarvis (2006) and De Freitas & Oliver (2006).
 Technical aspects including:
 Considerations for gameplay and the story (Zyda, 2005).
 The level of fidelity, interactivity, immersion, fun, etc.
 Integration aspects that include:
 Considerations for gamebased learning (Martens, Diener, & Steffen, 2008).
 Considerations for the inclusion of material in formal classes.
 Contextbased considerations for the implementation of digital educational resources (De Freitas & Jarvis, 2006).
Bearing in mind the analyzed literature on games and learning objects, a nonexhaustive set of basic features that represent a good starting point to achieve a usable product with a good grade of quality were identified:
 Short and focused on a single area of knowledge;
 A pedagogicallyevaluated graphical user interface;
 Cases with formal reasoning;
 Randomlygenerated cases;
 Challenging content that generates competition between students using the game.
3.3 Related Work
This section shows the main ideas of the reviewed articles related to this research.
The article by Tejada (2005) emphasizes that education should appoint a special economic resource for the production of learning materials, and stresses the importance of using digital learning resources. It also establishes the means and strategies for assessment of learning by competencies.
Dávila Balcarce et al. (Dávila & Velásquez, 2007) show the positive results of an evaluation of two collaborative games as a methodological resource for education. It is also stressed that for the improvement of the tested game applications for learning, it is necessary to create better educational content.
Garrido (2013) states that video games are a systems environment that shows one way to configure a set of features that are pedagogically desirable and expected.
Almiron & Porro (2014) established that the use of ICT in education is a good learning strategy. They also established that students have changed and they are not subjects for the current teaching system.
The literature review reveals that there is a need for the production of digital educational resources. Such is the case that a number of assessment tools are proposed at different educational levels, and different actors are involved in the teachinglearning process (teachers, professionals who are evaluated, students, and even parents). However, no paper proposes a way to properly develop resources and the literature makes the assumption that the production of educational (digital) resources is a competency that teachers have already fully acquired.
Therefore, this paper proposes a mechanism that ensures that this competency is acquired by those who design or produce educational (digital) resources, so that students receive and adopt competencies and learning outcomes.
The following sections present the CBD process, its application, and an example of its use including the results of its application to a real subject.
IV. Competencybased decomposition: a process for modeling competencies
The CBD process is a mechanism used to match a formal competency with nonformal content, identifying the aspects and factors that should be implemented in the production of the game so it may satisfactorily cover the expected competencies for a given school grade and thus the quality of the serious game and the fulfilment of the learning needs may be guaranteed. This procedure sets out to completely cover the content and outcomes that make up a subject in a syllabus, thus ensuring the appropriation of knowledge and learning outcomes for a particular competency.
To perform the CBD it is necessary to complete the following steps for each subject to be analyzed (figure 1).
 Identify the Standards, Goals and Graduate Profiles (SGGP) of the subject analyzed. This step is very important as the products of the subsequent steps must be aligned to these elements.
 Identify the contents and expected outcomes of the syllabus.
 Group, in areas of knowledge, the contents and expected outcomes in accordance with the SGGP.
 Organize knowledge areas in accordance with the SGGP.
 Identify competencies from the grouping of knowledge areas.
 Organize competencies and their knowledge areas in accordance with the SGGP.
It is very important to stress that every competency and its knowledge areas must be attainable from a software engineering perspective, since, for example, the syllabus for sixthgrade mathematics published by the Ministry of Education provides a set of math competencies, namely:
 Solve problems independently.
 Communicate mathematical information.
 Validate procedures and results.
 Use techniques efficiently.
All of the above are “competencies for life”, which, from a software engineering perspective, are very complex to manage and measure due to their multifactorial nature.
The following paragraphs show the application of this CBD process, used in this research to identify all the competencies for sixthgrade mathematics in order to produce games.
V. Methodology
This section will show stepbystep the process used to apply the CBD using the sixthgrade mathematics syllabus in elementary schools in Mexico.
Step 1
The following figures show the standards (Figure 2), goals (Figure 3) and graduate profiles (Figure 4) from the syllabus analyzed.
Mathematics standards 
The Mathematics Curriculum Standards present a vision of a population that knows how to use mathematical knowledge. This includes the set of expected outcomes of students in four school periods, leading to high levels of mathematical literacy. This consists of:
Student progress should be understood as:

Purpose of the study of mathematics for elementary education 
At this stage of their education and as a result of their studies in mathematics, it is expected that students:

Third school term, the end of the sixth grade, between 11 and 12 years old 
During this period, the curriculum standards are centered on three thematic axes: number sense and algebraic thinking; shape, space and measure; and information handling. After the third period, students know how to communicate and interpret natural numbers, fractional or decimal numbers as well as solve problems using conventional algorithms for addition and multiplication, calculate perimeters and areas, and describe and build solid figures. They use reference systems to locate points in the plane or interpret maps. They also perform collecting, organizing, analyzing and data presentation processes. Based on the teaching methodology proposed for study in this course, students are expected to also acquire mathematical knowledge and skills, and develop attitudes and values that are essential in the construction of mathematical competence. 1. Number sense and algebraic thinking. During this period the axis includes the following topics:
The curriculum standards for this axis are as follows. The student:

Step 2
All the subjects that make up a school grade may be found in SEP (2011). Each grade is divided into blocks.
Each block is made up at its first level by the central axis, at its second level by the topics, and at its third level by the contents. This can be seen in Table 1. It should be noted that there are only three axes in the figure. This is due to the fact that the fourth axis (“Attitude towards the study of mathematics”) is implicitly evaluated by the other three axes.
Step 3
In order to perform this step, you must review in detail the contents and expected outcomes. For example, as may be observed in Table 1, in the “Number sense and algebraic thinking” axis there are two sections of content: 1) Addition problems and 2) Multiplication problems; so they can be grouped into an area of knowledge called “Operations”.
Competencies to enhance Solve problems independently • Communicate mathematical information • Validate procedures and results • Handle techniques efficiently 

Expected outcome 
Central axes 

Number sense 
Form, space and measure 
Information management 


Numbers and number systems
Addition problems
Multiplication problems

Figures and bodies
Spatial location
Measure

Proportionality and functions
Analysis and data representation

Note : This figure shows the competencies to be enhanced, the expected learning, the central axes, the topics and the contents for Block I. The syllabus is made up of five blocks. Note that the fourth axis is not shown as it is evaluated implicitly in the other three. (SEP, 2011)
After identifying the contents and expected outcome, the grouping resulted in the following areas of knowledge:
1. Areas 2. The Cartesian plane 3. Combinations 4. The cross product 5. Decimal system 6. Fractions 7. Graphic representation of results 
8. Handling of solid figures 
15. Shapes and polygons 
Step 4
As in the previous step, the areas of knowledge must be reviewed. For example, areas of knowledge 5, 6 and 11, according to the SGGP, can be grouped in one competency.
The process of organizing the knowledge areas resulted in the creation of “groups” that will later become competencies.
Competency 1

Competency 3

Competency 5

Competency 2

Competency 4

Competency 6

Step 5
You must name the previous groups with competency names according to the SGGP. The identified competencies are:


Step 6
The last step is to create a map of the competencies and their areas of knowledge. This map can be created using a chart or a table.
Figure 5 shows a graphic template for displaying the results of the CBD process.
VI. Results
This section presents the results of the application of the CBD process in sixthgrade mathematics. From the application of the CBD process in mathematics for the sixth grade of elementary school, the following areas of knowledge and competencies were identified.
Results of the production of short serious games
About fifty short serious games were produced in the following competencies and areas of knowledge:
1. Numbers and relationships

3. Measures and conversions

5. Prediction and chance

2. Geometry

4. Processes of change

Results of using the produced short serious game (knowledge assessment)
After the production phase of the short serious games, the team proceeded to test them in order to study the impact on learning for students exposed to this learning strategy. The participants were a group of 29 students from the sixth grade of elementary school at the “Federal Rural Cuauhtémoc” elementary school located in La Paz, Ojuelos, Jalisco. The children from this school, and the school itself, have many needs, so the use of IT to access information helps to alleviate some of them.
The process followed for the test was as follows (Hernández Sampieri, Fernández, & Baptista, 2010):
 Potential schools were identified.
 Tests were designed for initial and control evaluations. The tests were designed to evaluate students’ level of knowledge in the following areas: areas, handling of solid figures, fractions, shapes and polygons, and the cross product.
 A school was selected.
 The group of students was selected. The group was divided into two parts, taking into account that in both groups, the students’ average grade must be equally distributed, meaning that the group was divided according to the average grades of the students.
 An initial evaluation was applied to all students.
 The test group used games in onehour sessions twice a week for four weeks.
 At the end of eight sessions, a control test was applied to identify the impact of game use.
 The collected data was analyzed with SPSS software.
The collected data allows trends in students’ scores to be determined before and after they use short serious games. The overall findings are graphically depicted in figure 7 where the dotted line displays the results obtained during the initial examination. The solid line displays the results of the evaluation performed after the use of short serious games.
As the reader can see, there are improvements in four knowledge areas, this is due to the fact that students achieved a higher level of acquisition of the competencies implemented in the short serious games they used.
The cross product game, which shows a decrease in the acquisition of the competency, was developed with a question bank instead of randomlygenerated problems, so the students memorized this question bank and did not achieve the intended competency acquisition.
VII. Discussion
The correct implementation of the competencies and the characteristics of the short serious games are extremely important because this is the only way to guarantee that users will achieve a higher level of adoption of the competencies implemented in the games.
As the reader can see, the cross product game did not achieve its purpose of facilitating student adoption of the competency due to a deviation in its development.
VIII. Conclusions and future work
The CBD process enables clear identification of the competencies to be implemented in short serious games, as in many cases this aspect is not taken into account during the development process (if a development process exists). The CBD process takes common elements in the syllabus as inputs, and turns them into programmable and measurable competencies (learning needs) for short serious games.
Once the competencies have been identified, the short serious game development process is capable of building highquality software products that are based on those learning needs and follow the correct implementation of the competencies.
This research has created and tested, with realworld students, a large set of short serious games, and achieved an improvement in competency adoption due to the correct implementation of learning needs into the games.
The established objectives of the research were:
 To create a process that makes it easier for teachers and pedagogues to transform a syllabus into competencies and areas of knowledge and develop all the materials and resources needed to teach.
 To test the process on a real subject and develop a set of short serious games from the identified competencies and areas of knowledge.
 To test the developed short serious games with real students to determine if there is any improvement.
Finally, we can say objectives have been fulfilled given that we created the CBD process and applied it to a real subject, developed a set of short serious games and tested these games with students.
The next step in this research is to use this CBD process to review a bachelor’s degree to ensure this process can be applied in any context.
Referencias
Almiron, M. E., & Porro, S. (2014). ICT in teaching: an analysis of cases. Revista Electrónica de Investigación Educativa, 16(2), 152160. Retrieved from http://redie.uabc.mx/redie/article/view/341
ÁlvarezRodríguez, F., BarajasSaavedra, A., & MuñozArteaga, J. (2014). Serious game design process, study case: sixth grade math. Creative Education, 5(9), 647656. doi: 10.4236/ce.2014.59077
Barajas, A., Álvarez, F., Muñoz, J., Santaolaya, R., & Collazos, C. (2014). A serious games development process using competency approach. Case study: elementary school math. In A. I.O. (AIPO) (Ed.), XV International Conference on Human Computer Interaction (pp. 572579). España: AIPO.
Dávila, G., & Velásquez, A. (2007). Evaluation of the application of collaborative games: “Devorón and Temporal”. Revista Electrónica de Investigación Educativa, 9(2). Retrieved from http://redie.uabc.mx/redie/article/view/164
De Freitas, S., & Jarvis, S. (2006). A framework for developing serious games to meet learner needs. I ’06: Proceedings of the International Conference on Interservice/Industry Training, Simulation and Education.
De Freitas, S., & Oliver, M. (2006). How can exploratory learning with games and simulations within the curriculum be most effectively evaluated? Computers & education, 46(3), 249264.
doi: 10.1016/j.compedu.2005.11.007
Díaz, A. (2006). El enfoque de compentecias en la educación. ¿Una alternativa o un disfraz de cambio? Perfiles Educativos, 111, 736. Retrieved from http://www.iisue.unam.mx/perfiles/perfiles_articulo.php?clave=2006111736&url=2006/n111a2006/mx.peredu.2006.n111.p736.pdf
Garrido, J. M. (2013). Strategy video games: some principles for teaching. Revista Electrónica de Investigación Educativa, 15(1), 6274. Retrieved from http://redie.uabc.mx/redie/article/view/330/516
Hernández Sampieri, R., Fernández, C. & Baptista, P. (2010). Metodología de la investigación. México: McGRAWHILL.
Ibrahim, R., & Jaafar, A. (2009). Educational Games (EG) design framework: combination of game design, pedagogy and content modeling. 2009 International Conference on Electrical Engineering and Informatics ICEEI '09, 1, 293298. doi:10.1109/ICEEI.2009.5254771
Instituto Nacional de Estadística y Geografía (2009). Enciclomedia ¿Qué es? Enciclomedia, una forma diferente de aprender y enseñar . Retrieved from http://www.inegi.gob.mx/inegi/contenidos/espanol/ciberhabitat/escuela/enciclomedia/
Martens, A., Diener, H., & Steffen, M. (2008). Gamebased Learning with computerslearning, simulations, and games. In Z. Pan, A. D. Cheok and W. Müller (Eds.),Transactions on Edutainment I (pp. 172190). Liverpool, UK: Springer. doi: 10.1007/9783540697442_15
Mulder, M., Weigel, T., & Collins, K. (2007). The concept of competence in the development of vocational education and training in selected EU member states: a critical analysis. Journal of Vocational Education & Training, 59(1), 6788. doi:10.1080/13636820601145630
Secretaría de Educación Pública (2011). Programas de estudio 2011. Guía para el maestro. Educación Básica. Primaria. Sexto grado. México: SEP. Retrieved from http://www.curriculobasica.sep.gob.mx/images/PDF/prog_primaria/PRIM_6to2011.pdf
Secretaría de Educación Pública (2013). Estadísticas ENLACE 2012. Evaluación Nacional del Logro Académico en Centros Escolares Educación Básica. Retrieved from http://www.enlace.sep.gob.mx/ba/
Tejada, J. (2005). Competenciesbased work in the practicum: how to organize and evaluate it. Revista Electrónica de Investigación Educativa, 7(2). Retrieved from http://redie.uabc.mx/redie/article/view/192
Velázquez, C. E., Álvarez, F. J., Garza, L., Sicilia, M. Á., Mora, J. M., & Muñoz, J. (2011). Una experiencias en el desarrollo masivo de objetos de aprendizaje empleando parámetros de calidad y un proceso de gestión bien definido. IEEERevista Iberoamericana de Tecnologías del Aprendizaje, 6(4), 155163. Recuperado de http://rita.det.uvigo.es/index.php?content=Num_Pub&idiom=Es&visualiza=1&volumen=6&numero=4&orden=desc
Zyda, M. (2005). From visual simulation to virtual reality to games. Computer, 38(9), 2532. doi:10.1109/MC.2005.297