Revista Electrónica de Investigación Educativa


Vol. 5, No. 2, 2003

How Suitable are Web Interfaces for
Collaborative Learning?

Manuel Juárez Pacheco
demasa@avantel.net

Departamento de Desarrollo Académico
Centro Nacional de Investigación y Desarrollo Tecnológico

Interior Internado Palmira S/N. Col. Palmira
Cuernavaca, Morelos, México

Guillermina Waldegg Casanova
gwaldegg@data.net.mx

Departamento de Investigaciones Educativas
Centro de Investigación y de Estudios Avanzados del IPN

Calzada Tenorios 235
Tlalpan, 14330
México, D.F., México

(Received: June 9, 2003; accepted for publishing: August 25, 2003)

 

Abstract

This paper describes a pilot study carried out to compare two Web interfaces used to support a collaborative learning design for science education. The study is part of a wider research project, which aims at characterizing computer software for collaborative learning in science education. The results coming from a questionnaire applied to teachers and researchers reveal the necessity to design technological tools based mainly on users’ needs and to take into account the impact of these tools on the learning of curricular contents.

Key words: Computer uses in education, collaborative learning, computer software evaluation, groupware, computer mediated communication.

 

Introduction

Most devices3 designed to take advantage of computers and Internet for educational purposes aim to model the spaces and services of a real educational institution and simulate the conditions teachers and students are used to. This model of educational-technological development poses a question: should the space devoted to learning adhere to conventional situations or can it offer innovative methods instead?

Without entering the debate on this issue, the question reveals an imminent problem: the inadequate relationship between developers of technology for education and users of the technology (teachers, students, directors). In most cases, the developers do not have an explicit pedagogical conception or they design their products with purposes different to those of educators. On the other hand, users constantly have to adapt their educational objectives and needs to the technological tools available. Educational developments historically run behind technological developments; this makes an a posteriori evaluation of technologies necessary to determine whether they were used for the most appropriate pedagogical model.

The use of the Internet with educational purposes is new and limited in countries like Mexico; as a consequence, there are few evaluative studies of the devices used and the results obtained in learning experiences. A first problem posed by the evaluation of educational experiences in Web environments concerns the decision of what to evaluate, since factors intervene in these experiences different to those in conventional educational processes. Clearly, the evaluation of learning results is essential both in conventional experiences as well as in those that make use of computers and new technologies; nevertheless, a determining factor for the success of a computer assisted learning experience is the degree to which the software devices are tailored to the aims of the experience. As mentioned before, these devices are rarely conceived specifically for a particular educational design.

The suitability of a device to the aims of a specific didactic experience can usually be verified through three distinct yet complementary procedures:

a) The appraisal by participants in the learning experience of the device’s performance.
b) Verifying the requirements of the device, taking as a starting point the theoretical model used as the foundation for the design of the learning situation.
c) The comparison of standards that define the design, development and evaluation of software with the characteristics of the device used.

This paper presents the results of the comparison of two Web devices used in the educational design of computer assisted collaborative learning at a distance, through the use of the three steps referred to in the above paragraph. The analysis is an initial description of the appreciations of critical users (teachers and researchers), based on a questionnaire designed with categories suggested by theoretical approaches that will be described later; it will serve as a starting point for a larger scale evaluation centred on students as primary users and on teachers as guides of didactic experiences. This study is the initial exploration of a larger project that aims to characterize software support tools for computer assisted collaborative learning in a web environment, from observation and analysis of real practices of educators and students.


Tactics, an experience in progress

The Tactics project4 places in virtual contact high-school students from two Canadian and four Mexican schools. It is a pedagogical design with theoretical and methodological perspectives that are inscribed within the educational research current of computer supported collaborative learning or CSCL. The research team is composed of teachers, researchers and post-graduate students; they design pedagogical modules from a socio-constructivist perspective that integrate information and communication technologies, seeking to build knowledge on transversal themes in high-school curricular science disciplines: physics, chemistry, biology, ecology. A series of research projects are developed from this design, which assess the variables and relationships involved in this type of experience (see Waldegg, 2002).

The collaborative learning model used in Tactics is a modified version of Slavin’s jigsaw model (1978). This model favours collaboration because teamwork is divided into equal yet interdependent parts. Participants become “experts” on a specific aspect of the subject at hand, and are in charge of reporting on the information gathered to the collaborative “base” group so that all members of the group can benefit. In other words, each participant is responsible for learning something on a subject matter and teaching it to the members of his or her team. Figure 1 shows the organization of a collaborative group. Each group expert belongs to a different school, and the three schools are located in different geographical settings.

Figure 2 represents interaction among group experts, who work on the same subtopic and exchange information and sources in order to understand better.

Figure 2. Interaction among group experts in Slavin’s jigsaw model
adopted by the Tactics project.


Tactics groups have the following characteristics:

a) each collaborative group is composed of three subgroups of experts;
b) each subgroup comes from a different school (Mexican of Canadian);
c) each subgroup has from three to five students from one same school.

The pedagogical design includes the definition of a series of tasks common to all teams (regardless of the subject dealt with) with the aim of fostering dynamics of collaboration. Table I shows the collaborative work process.

The characteristics and requirements of the process imply the use of software devices that work on the Web and support collaborative learning. The type of software with these characteristics has been developed in what is known as “Computer Supported Collaborative Work (CSCW)” and commercially as “groupware”.

There are currently two approaches to groupware evaluation. The first one emerges from the field of usability (Potts, Morse, Gutwin & Greenberg, 1999, Notess, 2001) and focuses on the internal aspect of software and its interface with the user. This approach analyzes the ease with which the technological artefact carries out specific tasks within specific contexts. Usability aims to determine to what degree the artefact facilitates interactions or makes them more difficult (interactions such as communication among different users, searches, drawing, writing, modeling, visualization or learning) according to the tool’s objective. The concept of usability involves the perspective of final users, as opposed to other indicators that only evaluate the functional performance of the tools. This concept includes not only the analysis of interfaces, but also their relationship to a determined user’s ability to carry out concrete tasks within a given context.

The second approach is called “systemic” (Ramaje, 1999) and complements the usability method with the study of the effects of software on users, work groups and even the structure of the organizations where the groupware is inserted.

During the operation of the Tactics project two web devices have been used to support cooperative learning. Although other similar projects exist, they have not reported any changes of the Web devices used, which makes this experience unique.

The devices used in Tactics were changed due to the fact that the first device used did not have a system that eased the management of the collaborative process, and also because of the Mexican provider’s frequent failures when accessing the system. At that moment, however, the possible benefits of this device as a support for collaborative learning were not analyzed.

What is the use of comparing both Web devices? We know that neither one was explicitly designed to work with from a collaborative learning perspective; however, this is how they were used and it is fitting to analyze which of the two, from the teachers’ point of view, has adapted better to collaborative work and which one is easier to use. From there we intend to identify the characteristics of the devices that have best adapted to Tactics’ educational design needs.

 

Web devices

a) E-groups

The first Web device used in the Tactics project was the e-group. It is free access software that runs on the Internet through the Yahoo site (http://mx.yahoo.com). Although it is clearly not an education platform, this device was chosen because its free character allowed us to explore the large-scale application of a design within the educational system at the same time as it permitted the management and supervision of student participation.

Access to e-groups is similar to subscribing to Yahoo electronic mail: a user-name and password are established; from there, one can form a new group or join an existing one. The group offers the following services for communication at a distance and shared work: individual messages and discussion lists; chat; database and shared files; polls; space for storing photos; agenda; links of interest. Figure 3 shows an e-group homepage.

For the aims of the Tactics project, one e-group was defined for each base team; each student was assigned a user number, a password, and an email address within the “yahoo.com” domain. Students could make use of all services in the group; the administrator and owner of all groups was a project researcher.

b) WebCT

The second Web device used by Tactics was Web Course Tools (WebCT). Originally developed to support the needs of the University of British Columbia, it complies with the characteristics of an education at a distance platform (Ministère de l’Education Nationale, de la Recherche et de la Technologie, 1999). Currently, WebCT is commercially distributed at a cost that ranges from 500 to 5000 dollars according to the quantity of users.

WebCT is a computer server that uses Internet and a series of tools that permit the design and development of interactive courses as a complement to teaching in the classroom. These tools add communication and evaluation activities in a structured and contextualized way, as well as a certain number of utilities that complement the student’s workspace. WebCT anticipates three kinds of participation: administrator, course creator or instructor, and student. There is also an additional role, that of the tutor who intervenes in the teaching to evaluate and comment on students’ work.

A course was defined for the Tactics project on the University of Montreal’s5 WebCT6platform (jointly responsible with CINVESTAV for the Tactics project). It was adapted as a space for interaction and exchange, but not linked to specific curricular contents. Figure 4 shows University of Montreal’s electronic service homepage.

The platform administrator (a researcher from the project) assigns a username and password for entering the area.

The space used by Tactics offers the following services, similar to those of e-groups: chat, discussion forums for communication within the base team and among expert groups; personal messages and messages for the work group; shared files for documents created by the students; space for personal homepages; spaces for participating schools; agenda. Figure 5 shows how these services are presented on the screen.

Although the space is general for all participating students, each base team belongs to a specific group and has access only to the work created within it, although it has the possibility of sending emails to all students in the project. The chat has three spaces available: the first one allows communication in real time with the base teams; the second one communicates to all Tactics project participants; and the third one is shared with all University of Montreal students. The first chat can be recorded for supervisory ends.

 

Instrument for the comparison of both devices

To know the opinion of Tactics teachers and researchers on the two devices employed, a questionnaire was designed based on some of the standards for the evaluation of groupware and the usability of software. One of the elements considered in particular was what is defined as mechanics of collaboration (Potts et al., 2002).

The questionnaire has two sections; the first one seeks to characterize the level of knowledge and familiarity with Internet use and with the different services by teachers involved in the Tactics project. The second one is geared towards the comparison of both tools used in the project; this part is divided into six subsections:

a) functionality,
b) efficacy and usability,
c) effects on the individual,
d) effects on teamwork, and
e) pedagogical effectiveness.

A total of 77 statements are given that emerge from the sentence: “in your opinion, which software is better for...”, with five answer options: (a) e-groups, (b) WebCT, (c) both, (d) neither, and (e) no opinion.

At the end of each subsection, a blank space was left observations or additional comments.

 

Discussion

Of the 462 answers received (see Table II), 32% lean towards WebCT and 42% opt for “both”; only 3% of the answers favour e-groups and 4% say “neither”.

It should be noted that a general 19% chose the “no opinion” option, which could mean that teachers as well as researchers have certain difficulties to visualize technology as a tool articulated to their work and to define their own role in the face of technology.

The differentiated answers of teachers and teacher-researchers deserve more detailed analysis. Teachers’ answers show that, in their view, WebCT is better adapted to collaborative work (in the categories: efficacy and usability 77%; effects on individual aspects 75%, effects on work in group 55% and pedagogical effectiveness 67%). Conversely, researchers (who, as noted before, are more familiar with the Internet and have a global vision of the project) do not make a distinction between e-groups and WebCT.

Teachers’ and researchers’ differentiated answers are similar to those reported in other studies made for CSCW (Potts et al., 1999), where it has been seen that previous experience and work practices allow users to avoid or overcome the tool’s problematic areas.

If the devices analyzed had been developed for this project, we could say that there were problems of design in both, since even when teachers lean towards WebCT, a consensus of opinions between teachers and researchers would be expected in aspects such as “effects on teamwork” and “pedagogical effectiveness”, central to this experience; as noted before, this consensus did not come about.

This sort of design fault, which evinces the software’s lack of adaptation to the task, has been reported in other areas of CSCW (Scott, 1997; Potts et al., 2002). It has been pointed out that to overcome these faults, the software must be an improvement on the work system already developed by the user group, which is why a greater understanding of how work is organized and developed is required.

In this case, support for coordinating the interactions between expert groups (which is not reflected in the answers on the effects of teamwork) is an important deficiency in e-groups as well as in WebCT, since these interactions are central to the “jigsaw” model structured by the Tactics educational design.

On the other hand, our study tends to indicate that the user (teacher or researcher) has still not incorporated into his or her pedagogical reflections which requirements software should have to foster teamwork and each participant’s learning in the collaborative experience. This happens despite the fact that the importance of analyzing the capacities of software on learning results has been suggested in some studies on the use of groupware in educational processes (Alavi, 1994). This same author suggests that: “This line of research would have important implications for the development of software environments that are directed towards supporting learning group processes” (p.172).

In the case presented here, both devices have advantages and disadvantages with regards to the proposed pedagogical design. The fact that e-groups have no cost paradoxically presents an advantage and a disadvantage: although its free-of-charge aspect does not weaken the support for communications among different groups, the lack of greater autonomy in the administration and therefore the coordination of group tasks does result disadvantageous. Another adverse element of e-groups that can even become an obstacle is the propaganda for articles or services that constantly appears within the work area or its associated services, such as email, since it distracts students’ attention.

The main advantage of WebCT is the fact that it is a platform designed for distance learning. However, like the greater part of commercial platforms, it presents an architecture adapted to the model of teacher-student didactic relationships and to the tasks associated conventionally to each one, even though it has elements for work among students. This architecture is not sufficiently modifiable to adapt it to essential requirements implied by the dynamics of collaborative learning.

Other collaborative work experiences, for example computer-supported intentional learning environments (CSILE) (Scardamalia & Bereiter, 1994; Bielaczyc, 2000) show computing developments appropriate for each perspective and didactic conception. Some others, for example the Virtual Learning Environments, contemplate explicit didactic approaches during the process of software development (Paquette, 2002) and its later use in real didactic situations with a collaborative focus (Henri &Lundgren-Cayrol, 1998).

This pioneering study breaks new ground in the field of Web-device evaluation for CSCL. Although it is crucial to improve the instruments used, they allow us to specify some areas for characterizing adequate software for collaborative work.

Although the results obtained until now provide an answer to the question we proposed above, it is still necessary to get information from the students, as they were users of both devices. However, it is no longer possible to obtain the opinion of the students that used e-groups, since they have left the educational level in which the project is developed. The evaluation of the students that are current WebCT platform users is therefore pending.

 

Results

1) Sample profile

The sample was made up of nine teachers involved in the Tactics project. Four of them are also researchers; this fact marks a difference in the sample profile. Although in general it could be said that participants have a good level of technological ability in the use of the Internet, in this case the researchers have managed to incorporate it more clearly into all their activities.

All sample participants except for one have Internet access from their homes. The researchers have used email for 10 years on average, while teachers have only used it for 2.6 years. Researchers have from 2 to 3 email accounts, while the majority of teachers have only one. Researchers receive and send over 10 messages a week, and teachers from 5 to 10. Researchers prefer to use Google as a search engine (more appropriate for academic searches), whereas teachers use Yahoo more (more appropriate for seeking commercial information). In general it can be seen that researchers use the Internet for a greater variety of activities and that they can transfer these abilities more easily to spaces other than academic.

2) General comparison

Table II shows a synthesis of answers given on the aspects evaluated in the second part of the questionnaire. It should be made clear that only six teachers that had the experience with both platforms answered this second part; of these, four are researchers from the project and two are participating teachers. Each aspect evaluated will be seen in the following section.



3) Functionality

Within the language of software engineering, functionality refers to aspects such as the reliability, efficiency and robustness of the system. In this part of the questionnaire, questions refer to how accessible devices are when working in diverse circumstances, how long they can be worked on without being blocked or without extensive waiting times, how easy is it to use other devices at the same time, etc.

If we take the sample total, the participants’ opinion on either one of the devices is not conclusive. In fact, 45% of the 162 answers in this section correspond to option (c) “both”, with 26% of the answers favouring WebCT and only 4% leaning towards e-groups.

Nevertheless, if we take the differentiated answers of teachers and researchers (Table III), preferences appear: the majority of teacher answers (57%) favour the WebCT option, with 0 answers in e-groups, while the majority of researcher answers (57%) do not show a definitive advantage in either one of the devices, and some even prefer e-groups (6%). It is interesting to note that 18% of the total answers choose the option “No opinion”, which is perhaps telling with regards to the fact that these issues related to pedagogical design are not stimulating reflection either in teachers or researchers.

4) Efficacy and usability

Software efficacy and usability refer to whether it favours (or is an obstacle for) specific collaborative work tasks such as communication, planning, coordination, tutoring, etc. In this part of the questionnaire, questions refer to the device’s speed, security and ease with which tasks are carried out by user groups.

In this case there is also not a conclusive opinion for the total sample of answers (156), since 42% of them correspond to “both”, with a greater percentage (33%) that has an inclination for WebCT, vs. only 3% that feel e-groups is better. The differentiated answers, on the other hand, again indicate preferences (Table IV): 77% of the teachers’ answers are grouped in option (b) WebCT, versus 57% of researchers’ answers who think “same in both”. It is also striking here that a fifth of the total answers are grouped in “No opinion”.


5) Effects on the individual

According to Ramage’s definitions (1999), this refers to significant psychological questions when evaluating the effect of the system on individual users; one of the questions asked, for example, is whether the user can situate his work in relation to the group, if he or she can develop certain tasks by him or herself, etc.

Of the 48 answers given in this section, 38% favour “WebCT” and an equal percentage leans toward “both”. However, it should be noted that 16% of the answers correspond to the option “No opinion”: once more, this suggests that software requirements that could foster each participant’s work and learning in a collaborative experience have not been defined. Again, the differentiated answers (Table V) give us a point of reflection since 75% of teachers’ answers are grouped in position (b) “WebCt”, versus 19% of researchers’ answers; answers for option (a) “e-groups” again are of 13% in the case of teachers and 0% in the case of researchers; in the case of option (c) “both”, the answers are of 25% of teachers versus 44% of researchers.


6) Effects on group work

This section refers to the effects that foster working in a group that can be attributed to the software. This aspect is different to that of usability in the sense that more emphasis is placed on inter-group relations propitiated by the software, rather than on specific tasks.

Of the 60 answers received, 27% favour Web CT, but 42% lean towards the option “both”. In this section there was no answer that favoured e-groups, but 10% correspond to “neither” and 21% to “no opinion”. The teachers’ answers (Table VI) favour WebCT (55%) versus 13% of researchers. It is interesting to note how in the option “both” these percentages are almost inverted: 15% of the teachers and 55% of the researchers.



7) Pedagogical effectiveness

Pedagogical effectiveness (Notess, 2001) of the device refers to the software’s capacity to promote effective online collaboration to reach educational goals.

Of the 36 answers received, 42% lean towards the option “both”, although 36% favour WebCT. Again, there was not a single answer in favour of e-groups, but 19% correspond to “no opinion”. The differentiated answers (Table VII) again favour WebCT with 67% of the teachers and 21% of the researchers.

 

References

Alavi, M. (1994, June). Computer-Mediated Collaborative Learning: an Empirical Evaluation. MIS Quarterly, 18 (2), 159-174.

Bielaczyc, K. (2000). Designing social infrastructure: The challenge of building computer-supported learning communities. Chestnut Hill, MA: Learning Communities Research and Technology Group.

Henri, F. & Lundgren-Cayrol, K. (1998). Apprentissage collaboratif et nouvelles technologies. Montreal: Centre de Recherche LICEF.

Ministère de l’Education Nationale, de la Recherche et de la Technologie (1999, September). Etude comparative technique et pédagogique des plates-formes pour la formation ouverte et à distance. Une étude de la Direction de la Technologie, sous-direction des technologies éducatives, des technologies de l'information et de la Communication (DT/SDTETIC). Paris: Author.

Moliner, M. (1997). Diccionario de usos del español (Vol. 1, 20th. edition). Madrid: Gredos.

Notess, M. (2001, August). Usability, user experience, and learner experience. E- Learn Magazine: Education and technology in perspective [In-Depth Tutorials Section]. Retrieved November 15, 2002 from:
http://php.indiana.edu/~mnotess/

Paquette, G. (2002). Modélisation des connaissances et des compétences. Un langage graphique pour concevoir et apprendre. Québec: Presses de l´Université du Québec.

Potts, M., Morse, E., Gutwin, C., & Greenberg, S. A. (1999). Comparison of usage evaluation and inspection methods for assessing groupware usability. Gaithersburg, MD: National Institute of Standards and Technology

Ramage M. (1999). The learning way: Evaluating co-operative systems. Doctoral thesis published online. Lancaster University, United Kingdom. Retrieved December 1st, 2002 from:
http://systems.open.ac.uk/objects/MagnusR/LearningWay.pdf

Scardamalia, M. & Bereiter, C. (1994). Computer support for knowledge-building communities. The Journal of the Learning Sciences, 3 (3), 265-283.

Scott, J. (1997, May 5). CSCW. Retrieved October 15, 2002 from:
http://farrer.riv.csu.edu.au/~jscott/Cscw_1.htm

Slavin, R. E. (1978). Using student team learning. Baltimore: The Johns Hopkins Team Learning Project.

Waldegg, G. (2002). El uso de las nuevas tecnologías para la enseñanza y el aprendizaje de las ciencias. Revista Electrónica de Investigación Educativa, 4 (1). Retrieved June 8, 2003 from:
http://redie.ens.uabc.mx/vol4no1/contenido-waldegg.html

1Project supported by CONACYT (National Council for Science and Technology, code number G33909-S).

2 A summarized version of this paper was sent to the VII National Congress for Educational Research (Guadalajara, Jalisco, October 2003).

3 The term “device” is translated from the word “dispositivo” in Spanish, as defined by the Moliner Dictionary for Spanish Use (1997): “a group of combined elements that are used to facilitate or do a job or to perform a special function”. We do not use the terms “site”, “page” or “portal”, since their characteristics do not coincide with the software we describe further ahead. One of the software programs analyzed is clearly an education at a distance “platform, but the other one cannot be called that.

4 Tactics is an acronym for “Collaborative Work and Learning with Information and Communication Technologies in the Sciences” (Trabajo y Aprendizaje Colaborativo con Tecnologías de Información y Comunicación en Ciencias) with meanings in French and Spanish, both the languages of the project.

5 The University of Montreal uses this platform since January 2000. The version it currently uses is the 3.6 Standard Edition.

6 http://www.coursenligne.umontreal.ca/

Please cite the source as:

Juárez, M. & Waldegg, G. (2003). How suitable are Web interfaces for collaborative learning. Revista Electrónica de Investigación y Educativa, 5 (2). Retrieved month day, year from:
http://redie.ens.uabc.mx/vol5no2/contenido-juarez.html



Licencia


Copyright (c) 2019 Revista Electrónica de Investigación Educativa

Licencia de Creative Commons
Este obra está bajo una licencia de Creative Commons Reconocimiento-NoComercial 4.0 Internacional.