Revista Electrónica de Investigación Educativa


Vol. 2, No. 1, 2000

Design and Evaluation of an
Online Freshmen Course

Lewis McAnally Salas
mcanally@uabc.edu.mx

Instituto de Investigación y Desarrollo Educativo
Universidad Autónoma de Baja California

Km. 103 Carretera Tijuana-Ensenada
Ensenada, Baja California, México

Carmen Pérez Fragoso
cperez@uabc.mx

Instituto de Investigación y Desarrollo Educativo
Universidad Autónoma de Baja California

A.P. 453
C.P. 22830
Ensenada, Baja California, México

(Received: September 7, 1999; accepted for publishing: April 4, 2000)

 

Abstract

This paper describes the development of a prototype for an undergraduate online course tested with freshmen students taking mathematics for a B.Sc. in Oceanography.  The design of the prototype focuses on the six basic components for the selection and use of technologies in online teaching and learning; namely, a) delivery and access, b) control, c) interaction, d) symbolic characteristics of the medium e) social presence and f) human-machine interface. The results report the degree of satisfaction with the prototype and the course.  These results, obtained through questionnaires and semi-structured interviews, address issues concerning the structure and design of the online course.  The participants’ answers highlighted the easiness of navigation within the course and the effective communication obtained with the use of the course's electronic tools.

Key words: Online education, learning networks, Internet.

 

Introduction

In designing applications for online courses, the interface is considered to be one of the determining elements for success (Reigeluth, 1994). In the teaching-learning process, the interface is the bridge of communication between what one desires to teach and the student, since this is the doorway to the course, as well as the structure on which its content rests (Schwier and Misanchuck, 1993). Because the interface must be designed with the user in mind (Graham, 1993), it must have characteristics that invite and motivate the student to advance in the construction of knowledge (Schwier and Misanchuck, 1993; Graham, 1993). It must, as well, have the capacity to communicate stability and clarity by means of the content, which is discovered little by little (Lynch, 1996). No matter how complicated the internal structure of the course may come to be, the student must perceive it as clear, orderly, and posessing concrete objectives. A lack of these characteristics produces anxiety in the student and, in consequence, a low rate of utilization for the material (Lynch, 1996). This article describes the procedure followed in designing the prototype, e.g. the factors considered in selecting the specific types of technology needed for online education.

 

Fundamentals of the design and structure of the prototype

The course was structured around four basic sections with well-defined functions: General Information, Lessons, In Contact!, and Support Programs. Using frames, the interface design was made up of spaces which defined principal areas: two vertical windows, each 100 pixels wide, located on the left side of the screen; and one window of approximately 400 pixels, occupying the rest of the screen. The first small window gives access to the principal sections of the course; the second shows the parts which compose each one of these sections; and finally, the large window shows the information or the content of the course itself (Figure 1).

In the development of the prototype, the principal sections have the following characteristics: General Information, whose objective is to offer the student access to all the information regarding the characteristics of the course, the school semester calendar with beginning and ending dates for the course, the calendar of examinations and assignments, the evaluation criteria, and so forth; Lessons, which includes the units that make up the course (Figure 2a). Each one of the units was organized visually in the same way as the lessons themselves, including already-solved exercises presented step by step as practical cases relating to the lesson theme (Figure 2b), as well as exercises assigned to be solved in the traditional way (Hativa, 1988), and turned in to the teacher on established dates. In Contact! is the core of the prototype’s development, composed of three modules enabling communication among the participants: Email, asynchronous discussion organizers (Figure 2c), and a chat program in real or synchronous time; Support Programs, which gives access to the tools used in the Mathematics I course; a calculator and a program for creating graphs.

For the design of this online course, a prototype for a learning web, the model proposed by McIsaac and Gunawardena (1996) was used; this program describes the principal factors which must be taken into consideration when selecting and using technologies for courses designed for distance learning. According to these authors, the six most important characteristics which must be considered for the adoption and use of technologies specifically oriented toward online education are: a) delivery and access, b) control, c) interaction, d) symbolic characteristics of the medium, e) the social presence created by means of the medium, and f) the human/machine interface.

 

Delivery and access

Considers the way in which technology distributes the course material and the ease users have in approaching this channel of distribution. In this case, by the use of the Internet as a means of distribution, and by an interface design permitting ease of use and navigation.

 

Control

The intrinsic characteristics of the course, conveyed by an easy-to-use technology, whose hyperlinks permit a nonlineal learning trajectory and the use of asynchronous communication, directly influence the student's feeling of control. According to Bayon (1992), control is something more than the user's independence, since it requires a marked balance among three factors: the independence of the student (the opportunity to make choices), his or her competence (ability and skill), and the required support (human and infrastructural). Consequently, this factor may be defined as the user's ability to control the medium according to his/her needs, because of the flexibility it gives as to place and time of use (McIsaac and Gunawardena, 1996).

 

Interaction

To facilitate the success of an online course, every prototype must promote interaction not only with content on an individual scale, but also among all the participants.

Interaction is the concept which underlies what Moore (1991) calls transactional distance, meaning the distance which exists in instructional relationships. For Moore, distance is determined by the quantity of dialogue between the student and the instructor, as well as the level of structure existing in the design of the course. There will be, therefore, a greater transactional distance when in the course there is more structure than dialogue between the students and the instructor. Also, for online education, interaction with fellow students is fundamental, since the construction of knowledge is considered to be both an individual and a group activity.

In the prototype, interaction with the content is promoted primarily by the possibility of non-lineal navigation in using the hypertext, while interaction between participants is facilitated through the use of Email (asynchronous), deferred discussion (asynchronous) and real time chat (synchronous) in the section In Contact!

 

Social presence

This is the ability of the students to see each other as real persons; the medium used must be well-selected so as to be flexible and to provide a social climate favorable to learning. Gunawardena (1995) defines social presence as the capacity for connection, for communicating with students in different locations, and for allowing the students to view the professor and the other participants as real people. To achieve this, the prototype facilitates interaction (treated in the previous point) through the integration of synchronous and asynchronous modalities of communication; however, the available means of communication do not guarantee the establishment of a social presence conducive to learning, but rather the teacher in charge establishes this, supported by the instructional design of the course.

 

Symbolic characteristics

This refers to the symbolic system to be used during the course: iconic, digital and analogical. The designed course is basically supported by an icon system, using pictorial representations; and the text or digital system, using written language.

The icons: The icons used in the course followed the recommendations of authors like Schwier and Misanchuck (1993), and Lynch (1996), representing an action and the characteristics of the content in a graphic manner, as well as describing their function in just a few words. The use of color in the icons and graphics supported the study environment, and not merely the esthetics of the design, since it was believed that the use of color in the interface should not distract the user from his or her primary objective: learning.

The texts: Following the recommendations of authors like Parker (1989); Miles (1989); Siebert and Cropper (1993); Schwier and Misanchuk (1993); and Apple Computers (1996b), when the lessons were composed, not only was the length of the texts taken into account to avoid piling up the text on the pages, but also, sought optimal use of blank space. In this case, the blank space is not considered as “excess space with no text”, but as an element permitting the separation or unification of ideas, giving “breathing room” to the page, and producing a sensation of cleanliness and order. Another important element in giving defininition to the text are the patterns of recognition the letters and paragraphs themselves can have. Due to the fact that reading is not done letter by letter, but rather word by word or in blocks of words, the patterns of recognition support and facilitate the reading flow because they help the reader to recognize the letters and words rapidly, as well as locate the reading in the paragraphs.

 

The user/machine interface

Ergonomics is a very important factor in the design of human/machine interfaces, both in research and in the development of work areas related to the human element. The type of interface employed by the technology has implications for the type of training or orientation which students and instructors must receive in order to be competent users in the instructional medium (McIsaac and Gunawardena, 1996). Literature about educational interface design points out that to achieve the sensation of stability in the interface, a definite and finite number of objects must be used, as well as a definite and finite number of actions these objects perform (Apple Computers, 1996a). In the prototype developed, the interface has a high graphic component. This means that it starts with the use of icons which represent an object or an action; the number of objects and actions are sufficient to make the interface functional in the context of its objective.

 

Evaluation of the prototype

The prototype was tested in a Mathematics I course at the School of Marine Sciences of the Autonomous University of Baja California (UABC). The study participants were new students in this school. The initial sample was made up of 16 students averaging 18 years of age. The students were given two two-hour training sessions on the efficient management of the navigator and the Email, as well as a booklet with instructions and suggestions for using the system.

To evaluate the students' level of satisfaction with the development of the course, an opinion questionnaire was designed; this was applied three weeks before the end of the course (N = 10). Students answered the questionnaire anonymously; in it they were asked to rate the course using a Likert scale (1 strongly disagree - 5 strongly agree), and to express their reactions and opinions in 22 statements regarding six general dimensions. Next were analyzed the two dimensions related with the objective of this work: Pedagogical Technique and Communication; Table I presents the table of frequencies in these dimensions. To complement the information, semi-structured interviews were conducted with the instructor of the course, with the psychologist of the Psychopedagogical Department of the School of Marine Sciences, and with three students who took the course on line; these students were designated according to their grades as having low, medium and high performance. The interview took place privately through an informal conversation which touched on themes including the students' general impression of the course and their personal perception as to its effectiveness, structure and interface.

Table I. Table of frequencies for the opinion survey of online
students (N=10), regarding pedagogical technique and
communication during the course

As we can see, one of the answers showing the highest rate of agreement, that is, which is concentrated at one of the extremes, was “Moving from one section of the course to another is easy”, with 50% in agreement, and 50% in strong agreement.

Table II presents comments representative of the interviews regarding the effectiveness, structure and interface of the course; all the comments were positive, underscoring the friendliness of the interface and the ease of navigation within the course.

Table II. Personal perception regarding the functionality of course structure
and interface. Student 1 (Low-level achievement);  Student 2  (Intermediate-level achievement); student 3 (high-level achievement)

The comments about the communication achieved during the course were very diverse; a wide gamut of responses was observed, going from the extreme of rating the course as “bad”, or the same as achieved in courses where the student is physically present, to considering it as “better” than other courses where the student is physically present. As for the teacher, she considers herself to have achieved excellent communication with her online students (Table III).

Table III.  Opinion of the subjects interviewed regarding
the communication achieved during the course

 

Discussion

All those interviewed made very positive comments about the effectiveness of the structure and the interface of the course; however, their opinions regarding student/teacher communication were located between two extremes: from the opinion that it was “bad”, through “it depends on how the teacher is”, but without believing that it should be different. At the other extreme was someone who considered it to be “better than with other teachers”.

The willingness of the great majority of the students, nine out of ten, to take other courses on line, indicated that the sum total of pedagogical technique (structure and organization of the course), along with the method of instruction, complies with the expectations the students have for a formal curricular course. The preceding leads us to suppose that in the course the four types of interaction: student/content, student/instructor, student/student and student/interface, which Moore (1991) and Gunawardena (1995) state as essential for the good development of distance courses were successfully achieved.

Based on the commentaries expressed, it may be said that the student/interface interaction was very efficient. Gunawardena (1995) mentions that the interaction between student and technology is a critical component that normally has been absent in literature, and which includes the knowledge and use of the interface for all the interactions, since the students have to make a conduct change in order to use this type of technology. The satisfaction level concerning the effectiveness of the structure and interface of the course was very high; there were no negative comments about the ease of navigating in the course and finding the desired information. Everything indicates that the training given the students before the semester began permitted their interaction with the interface to be adequate for exploring the content and structure of the course and for allowing them, with time, to handle it well.

Although the high level of agreement on the ease of use is not uncommon among students who have experienced this modality, the combination of the instrumentation of the content in the medium and the instruction method used are the factors which contribute most to the acceptance of the modality (Hiltz, 1993; 1994; Arvan et al, 1998); Kearsley (1995) and Hanson et al (1997) make an interesting point regarding the interaction. They consider that the satisfaction the students feel toward the interaction level of a course is intimately related with their perception of the overall interaction offered by the system, and not necessarily by the interaction they have personally experienced in the course.

This integration of components is so important that theoreticians who subscribe to so-called technological determinism propose that the characteristics of the computer programs (software) and equipment of the system determine the behavior of the user and the success level of the application (Hiltz, 1994). Similarly, Kling (1980) identifies system rationalists as those who propose that effective and efficient computerized systems will produce an effective and efficient behavior in its users. Also, the technicists within Mowshowitz's topology (1981) define the success or failure of a computational application in terms of the design and instrumentation of the system. As an example of the foregoing statement, the work of Turner (1984) may be cited. This work shows how the form of the interface in an application system used by an insurance company affected workers' attitudes toward the system as well as how they carried out their work, and their satisfaction with it. Supposing these assertions to be correct, we might expect that the behavior and attitudes of users toward the online course would be produced by the characteristics of the equipment used (characteristics of the computers and course implementation on the university network) as well as the characteristics of the course itself on line.

Comments related to the student-instructor interaction were generally favorable (with the exception of the low-performance student who never learned to use the Email) and lead us to suppose that a good level of social presence was achieved; the teacher points out that she perceived an excellent communication with her “cyber-students”, a greater level of closeness than in the traditional classroom. For example, she reports having received 75 personal messages from her students relating to Unit I of the course (Moren y Valle, 1999). In spite of the fact that this is her first experience in teaching on line, it appears that she was able to elicit the copious participation of the majority of her students through her opportune answers to the questions she was sent, and through the friendly, informal tone given her contributions to the deferred discussion and to her Email messages. The results obtained echo the findings of Hiltz (1994) and Kearsley (1995), among others, who say that the determining factor for making the students feel that the online modality is better or worse than the traditional is probably the quantity and quality of the teacher-student and student-student interaction.

In conclusion, we may tentatively say that the results obtained in this investigation validate the efficacy of McIsaac and Gunawardena’s model (1996) for the design of online course interfaces in our context.


Translator: Lessie Evona York Weatherman
School of Languages
Autonomous University of Baja California at Mexicali

 

References

Apple Computers. (1996a). Apple web design guide. Retrieved June 10, 1996 from:
http://www.cybertech.apple.com/hi/web_design/intro.html

Apple Computers (1996b). Macintosh human interface guidelines. Retrieved June 10, 1996 from:
http://www.cybertech.apple.com/hi/hi_resources/hi_princ/intro.html

Arvan, L., Ory, J. C., Bullock C. D., Burnaska K. K., & Hanson M. (1998). The SCALE efficiency projects. Journal of Asynchronous Learning Networks 2 (2). Retrieved October 7th, 1998 from:
http://www.aln.org/alnweb/journal/vol2_issue2/arvan2.htm

Aspillaga, M. (1992). Implications of screen design upon learning. Journal of Educational Technology Systems, 20 (1), 53-58.

Bayton, M. (1992). Dimensions of "control" in distance education: A factor analysis. American Journal of Distance Education, 6 (2), 57-66.

Burke, C. (1989). From verbal to visual: How to use type as a graphic element on your pages. Publish, 65-71.

Graham, D. (1993). Quick solutions to great layouts. Cincinnati, OH: North Light Books.

Gunawardena, C. N. (1995). Nuevos caminos en el aprendizaje, nuevas formas de evaluar. In P. J. Dirr & C. N. Guanwardena (Eds.), Cuadernos de educación a distancia 3: Enfoques sobre evaluación de los aprendizajes en educación a distancia II. México: Universidad de Guadalajara-CECAD.

Hanson, D., Maushak, N. J., Scholosser, C. A., Anderson, M. L. & Simonson, M. (1997). Distance education: Review of the literature. Washington, DC: Association for Educational Communications and Technology-Research Institute for Studies in Education.

Hativa, N. (1988). Differential characteristics and methods of operation underlying CAI/CMI drill and practice systems. Journal of Research on Computing in Education, 258-270.

Hiltz, R. (1993). Correlates if learning in a virtual classroom. International Journal of Man-Machine Studies, 39, 71-98.

Hiltz, R. (1994). Education, innovation and technology. In R. Hiltz (Ed.), The virtual classroom: Learning without limits via computer networks (pp. 19-29). Norwood, NJ: Ablex Publishing.

Kearsley, G. (1995, May 18-21). The nature and value of interaction in distance learning. Proceedings of the Third Distance Education Research Symposium, University, Pennsylvania: Penn State University.

Kling, R. (1980). Social analyses of computing: Theoretical perspectives in recent
empirical research. Computing Surveys, 12 (1), 61-110.

Lynch, P. J. & Horton, S. (1999). Web style manual: Basic design principles for creating web sites. Boston, MA: Yale University Press.

McIsaac, M. S. & Gunawardena, C. N. (1996). Distance Education. In D. H Jonassen (Ed), Handbook of research for educational communications and technology: A proyect of the Association for Educational Communications and Technology (pp. 403-437). New York: Simon &Schuster-Macmillan.

Miles, J. (1989). Pause for effect: When white space isn´t wasted space. Publish, 37-38.

Moore, M. G. (1991). Distance education theory [Editorial]. The American Journal of Distance Education, 5 (1), 1-6. Retrieved March 12, 1997 from:
http://www.ajde.com/Contents/vol5_3.htm#editorial

Moreno, M. & Valle, C. (1999, May 17-21). Experiencia de un curso en línea de matemáticas básicas para estudiantes universitarios de primer ingreso. Paper presented at the IV Taller Internacional de Educación a Distancia, La Habana, Cuba.

Mowshowitz, A. (1981). On approaches to the study of social issues in computing. Communications of the ACM, 24 (3), 146-155.

Parker, R. C. (1989). The book. Chapel Hill, NC: Chapel Hill Press.

Reigeluth, C. M. (1994). Instructional design theories. In T. Husen & T.N. Postlethwaite (Eds). The international encyclopedia of education (2nd. ed., vol. 7). Oxford: Pergamon Press.

Schroeder, E. E. (1994, February 16-20). Navigating through hypertext: Navigational techniques, individual differences, and learning. In Proceedings of the selected research and development presentation at the 1994 National Convention ot the Association for Educational Communications and Technology, sponsored by the Research and Theory Division, Nashville, TN.

Schwier, R. A. & Misanchuk, E. R. (1993). Interactive multimedia instruction. Englewood Cliffs, NJ: Educational Technology Publications.

Siebert, L. & Cropper, M. (1993). Working with word & pictures. Cincinnati, OH: North Light Books.

Tinkel, K. (1989, August). Setting type in style. Personal Publishing, 62-65.

Turner, J.A. (1984). Computer mediated work: The interplay between technology and structured jobs. Communications of the ACM, 27 (12), 1210-1217.

You can find information related to this article on the Web sites of:

Hung, D. (2002). Situated cognition and problem-based learning: Implications for learning and instruction with technology. Journal of Interactive Learning Research, 13 (4), 393-414.
http://www.aace.org/dl/files/JILR/JILR134393.pdf

Goolnik, G. (2002). E-learning for smaller rurally based businesses: A demand-led challenge for Scottish education institutions. Online Journal of Distance Learning Administration, 5 (2).
http://www.westga.edu/~distance/ojda/summer52/goolnik52.html

Links' last uptade: 09/01/03

 

   

Figure 1. Screen showing the relative size and position of the frames containing the icons for the principal sections (maroon), the icons for the options of the principle sections (orange), and the contents of the course (green). The arrows indicate the response order of the frames when selectingan icon or hyperlink

 
a



b


c



d

Figure 2. (a) Screen for accessing the modules of the course and the window for entering the module Lineal and Quadratic Functions; (b) theme window: solving equations; (c) In Contact! screen, with the communications options and the mailbox window having the names of the participants and links to their Emails, and (d) window for deferred discussion

Please cite the source as:

McAnally, S. & Pérez, C. (2000). Design and evaluation of an online freshmen course. Revista Electrónica de Investigación Educativa, 2 (1). Retrieved month day, year from: http://redie.uabc.mx/vol2no1/contents-mcanally.html