Revista Electrónica de Investigación Educativa


Vol. 4, No. 2, 2002

Using Cybernetics for Early Detection of
Instructional Problems in the Chilean
Educational System

Martín Schaffernicht
martin@pehuenche.utalca.cl

Departamento de Informática de Gestión
Facultad de Ciencias Empresariales

Universidad de Talca

Avenida Lircay s/n
Talca, Chile

Patricio Madariaga Araya
pmadar@pehuenche.utalca.cl

Dirección de Tecnologías para el Aprendizaje
Universidad de Talca

Avenida Lircay s/n
Talca, Chile

(Received: January 29, 2002; accepted for publishing: September 17, 2002)

 

Abstract

We have posulated that in order to avoid the loss of opportunities in education, it is desirable and feasible to develop a detection system for problems during the teaching processes. We have based this on two disciplines. From cybernetics we took the idea of the logically autoreferential system which adjusts itself automatically and is analogous to Beer’s viable system model. From theories of learning, we presented a learning-action focus that distinguishes various hierarchical cycles. We distinguished two time horizons: one, for courses that managed the improvements between two versions of themselves; the other, concerned with improving the processes while the course is being carried out. At this level, we differentiated between the student, the group of students, and the teacher, as possible sources of imbalances. The general process of three cycles proposed indicators for the early detection of problems; furthermore, it has shown an capability for self-adjustment, and is under further investigation. The experience has been developed in the context of computer-training associated with the ENLACES Network which the University of Talca offers to teachers of middle schools and high schools, as well as subsidized private schools in the Maule Region of Chile.

Key words: Improvement of teaching, total quality management, cybernetics, information management systems.

 

Introduction

When an educational institution creates courses, do these define the structure for organizing the service expected to affect the users’ learning? There is in fact a division established in two stages: on the one hand, there is the course design, which influences their execution; on the other hand is their implementation with a specific set of users, which can influence the next stage of the design. Thus, there are two scenarios: the performance can show problems alien to the design; or the design itself can present deficiencies. Not overcoming the first type of problems leads to the loss of learning opportunities; not taking care of the second type equals a future repetition of the loss of opportunities.

To overcome a problem, one first has to detect it. It seems obvious—and in fact industries practice it for quality assurance; nevertheless, in the field of education it is not a common practice. Then, how can there be organized an information process to detect early problems—that is to say, while there is still time to take correct action?

This article presents our response to this central question.

The proposal is the fruit of our participation in the ENLACES project, part of the present Educational Reform of the Ministry of Education in Chile. This consists of installing laboratories with computers connected to the Internet in the country’s public schools and subsidized private schools, as well as training the teachers. The objectives are the effective assimilation of the computer and Internet into the pedagogical and administrative practice of the establishments so as to achieve a minimal autonomy of the teachers. The Ministry has formed a network of universities that operate as “Zonal Centers” (ZC) in different regions of Chile. They carry out subcontracts with other universities that act as “Executant Units” (EU), and these, in turn, administer training with a certain degree of freedom in their internal management and conduct of courses.

This article presents the approach developed in our EU for managing this training process, according to the doctoral research of one of the authors (Schaffernicht, 2001) and the responsibility of others in directing that project.

First, there are presented two facets of the theoretical framework.

  • Cybernetics is the science of the control of systems; it suggests considering that the systems regulate themselves by means of closed loop control relying on the feedback process. This type of error correction is important in our work.
  • The action-learning approaches emphasize the importance of action for learning; in fact, only action reveals the validity of ideas, and feeds reflection. We understand quality improvement as a learning challenge and, therefore, we position ourselves in this field.

Farther along in this work there is detailed the approach we designed. Given that it is based on various computers, we explain how we calculated with qualitative values. The cycles of management and the environments of application in the instructional process are defined. Finally, we present the general design of the double-loop correction and application.

 

1.Theoretical basis

Our work is interdisciplinary: we use concepts and methods from various scientific domains; therefore, the focus includes aspects of several disciplines. In this section, to argue our proposal, we present what we have taken from each.

The two theoretical issues set out below represent an overview of global concepts that should be considered when designing and implementing these explicit designs of information. An important issue for this purpose will be, then, cybernetics in management, because it represents the way systems progress through feedback processes. Consistent with the above, we connect the learning processes around the existence of cycles to optimize management processes on a cybernetics basis.

1.1. Cybernetics in management

The neologism cybernetics was proposed by Norbert Wiener (1958), to separate this new discipline from the previous ones framed in the reductionist paradigm. Under this name, there were investigated mechanisms of circular causality in biological and social systems (von Foerster and von Glaserfeld, 1999). This had to do with governing systems because they are autonomous units: although they interact with their environment, they make internal changes and adaptations only in relationship with interior elements. Therefore, such systems cannot be externally predetermined; on the other hand, they can be influenced in their behavior.

A system has cyclic processes that can detect certain internal imbalances and take corrective action to restore balance. These are called “control systems”, and require feedback. For each aspect distinguished by the system, the current situation is compared with the goal position; if there is a difference, it means that the internal process that leads from a stimulus to an action must be corrected. The correction is made based on the difference observed.

The human body can be used as an example of the above: if an athlete is running, his1 muscles produce heat. As a result, his body temperature can rise beyond what is “normal.” If this happens, it activates an automatic correction process called “sweating”, which will reduce body temperature until it regains the balance of approximately 37 º C.

Cybernetics is betting that these principles which have worked so well in the development of life on our planet, must also be valid for designing systems in other domains, specifically in management. We define “management” as the transformation of information into action (Forrester, 1994), and vice versa.

We call the reader’s attention to these questions: “Who defines the dimensions for comparing a target with an observation, the levels that are determined as the goal and the way the system works to correct differences? And how are they defined?” These are important questions for those who design systems; our proposal provides a possible answer.

Beer (1981) proposes the concept of the “viable system”, in which organization is designed as an ecology of subsystems in five levels.

Table I. Levels of the viable system model

From cybernetics we retrieve notions of autonomy and control (in the sense of “error correction”).

1.2. Double-loop learning

We define “learning” as a change of behavior (action) that gives better results, according to the criteria of the system that speaks of learning (Maturana, 1997). This definition is compatible with two others which, apparently, contradict each other (Weick, 1991):

  • Learning is when a new behavior is observed when facing the same stimulus.
  • Learning is a successful conduct when facing a new stimulus.

These two definitions speak of observable behaviors. However, the behavior can be internal (not observable from the outside): an idea, an understanding, or greater “certainty” about an idea. Our conceptualization approaches all the cases in such a way that the motor of learning is to recognize a difference between what one wants to obtain, and what one has. In this sense, we recognize the cybernetic idea.

Thus, the necessity of learning is considered from different perspectives. For instance, Bateson (1990) posits various levels (0: becoming informed; 1: changing the behavior, 2: learning to change e behavior). Bohm (1980) argues that when a course of action gives unsatisfactory results, then our understanding fails. If we try to solve the problem without a prior correction of our ideas, we will only reproduce the problem in another form.

Argyris (1993) proposes a simple cycle of action-control, like the adaptation of internal values within a stable framework, and a double loop as the review of this same framework, which would satisfy Bateson’s Level 2 as well as Bohm’s idea. In Kolb (1984) there are shown more cyclical focuses, but no double loop; Kolb himself does not follow an approach with ordered cycles.

We appropriated the idea of considering two levels of learning. Furthermore, we follow Argyris when he says that the representations actors make of their activities are not appropriate by default: they have to be put into practice to see if they are valid.

Learning in a directed (focused) manner requires specifying the rules and respecting such designs; thus the knowledge expressed is empirically confronted, and clarity is obtained regarding its validity.

1.3. Quality Management

By “quality” we understand the degree to which the perceived qualities of something—product or service—satisfies the expectations of the users or recipients. It is obvious that there may exist differences between expected and perceived qualities.

Under this premise we consider as a “controlled variable”—controlled by the system itself—the phenomenon known as “Heisenberg’s property” (Martinez, 1998), which consists, in the words of Oppenheimer, of “any action to take a measure or to study what happens in the atomic world, creates, in spite of the whole order of this world, a new situation, unique and not fully predictable “(Oppenheimer, 1954, p. 62). We understand that while we exist, none of our actions is exactly the same in its consequences as other possible acts; so we cannot escape the need to choose, and we find ourselves projected into the world (Heidegger, 1993).

This makes it plausible to use organizational learning techniques to improve management quality.

 

2. Proposal of a focus for self-improvement

2.1. About the calculation with qualitative information

In all education systems where the scores “very good”, “good”, “satisfactory” or the like have become ordinal numbers (1, 2, 3 ...), one begins to calculate as if they were cardinal numbers. However, the distances between a score of 5 and 6, for example, are not necessarily the same as that between one of 2 and one of 3. What, then, does it mean when we say that one student has 5.5 and another, 5.6?

Conversely, there is a tendency to assess in qualitative terms, that is, no calculations are made to sort students by name. While this avoids meaningless calculations, it opens the door to inaccurate evaluations.

In a logic that seeks evaluation to promote corrections (improvement by eliminating the cause of the error), exact information is required, especially if we consider an assessment formula that does not distort the information, but permits various degrees of globality/detail. The following considerations clarify how:

  • For each competition, at some point something can be said about the degree of control that each student shows; this is equivalent to a score at the level of micro-granularity.
  • For each competition, there can be specified a threshold domain that each student must show to display a minimum threshold of approval.
  • Each competition is part of a larger grouping (theme, unit or subsector). Each grouping has, then, a specific number of members who present at any time a profile of achievement per person.
  • For each student there can be determined, at any time, the percentage of competencies in which she has already obtained or gone beyond the threshold of satisfaction by group, thus including her overall situation. By comparing a goal with that percentage, we can know whether a student has a problem or not.
  • A comparison of two consecutive images reveals learning as the change in the domain of competencies.

2.2. Two management domains with different temporalities

When teaching a set of skills to different sets of students in different periods of time, one tries to teach repetitively: thus, each course can be understood as a particular experiment for showing that set. We distinguish two levels in this operation:

  • Between two repeats of a course, there is an opportunity to learn from what has happened, and to improve the course.
  • During the operation of a course, one must take advantage of all emerging information to prevent or resolve problems.

In this article we concentrate on the second of these cycles.

2.2.1. The long-term domain: course design

In a course, there is proposed a set of activities, developed in the form of a set of materials, to permit students to achieve competencies. There are here certain freedom of choice and design spaces, depending on the level of the education system where we are located.

Table II. Decision-making Competencies in the ENLACES organization

We appreciate that some elements are designed at the higher level, and level “system 1” should accept them as such; let us emphasize that the quality of the design is the responsibility of the designer: the EU cannot improve what the CZ must design. In turn, the CZ cannot improve what it designs if it does not have adequate information provided by the EU, which accounts for the quality of its activities and materials.

Consequently, at the end of an iteration of the course the success of its activities, materials and skills can be evaluated. If objectives (thresholds of satisfaction) have been specified, then they can be compared with the student population’s achievement, and they can proceed to the changes that seem to be appropriate. If an activity is designed to enable all students to learn 100% of the competencies mentioned, but the students’ average achievement is less than 100%, then we know that this activity should be improved.

2.2.2. The short-term domain: operation of courses at three levels

During an iteration of the course, activities, materials and skills cannot vary except in cases of emergency: this means recognizing that what was planned would be worse than an improvisation, and obliges to reformulate the elements in question by official communication. It is hoped to maintain the course as provided in the plan and use different compilations of information to evaluate the elements based on the recorded experience:

  • Each student is individually responsible for her learning and for conformity with her evaluation.
  • Students who learn together make up a group.
  • The teacher is responsible for the course and for the learning success of his groups of students.

Each of these levels of control is found at a level of granularity: The teacher has groups, and each group has students. Therefore, the smallest unit of assurance will be the student, and the greatest, the teacher.

Our process operates with a set of indicators defined for each of these levels. On this basis, there are mounted three double-loop cycles (Argyris, 1993), which express more fully the incorporation of cybernetic concepts, double-loop learning and quality management, which are presented below.

2.3. The improvement cycles

2.3.1. The general double-loop intervention and self-adjustment


The following overview diagram presents the cycle stages (the same design for the three levels we distinguish):


Figure 1. General schematic of the process of early detection

In this diagram, the ovals represent our activities. By answering the open question in the section on cybernetics, we recognize that we have defined the goals that are important to us (activity 1). For each objective, we defined the required standards, we called them “thresholds” (activity 3). In the second point we defined what to do to correct differences. After defining these three concepts we were able to use them to: make observations (activity 4), calculate basic indicators (activity 5) and specific indicators for each level (activity 6), distinguish problems (activity 7), and intervene (activity 8).

In this way three cycles were established: cycle 1 (indicated by the open-tipped arrows) distinguishes between individuals (students, groups, trainers) who had problems (their profile did not satisfy the thresholds) and those who did not have. To each problem was assigned a tactic of intervention, according to the rules defined.

According to the experiences of intervention and the fulfillment of expectations, the second cycle (solid-tipped arrows) proceeded to adapt the thresholds of classification. Along with the threshold is mentioned the value adjustment. If during two iterations, the problem to which the threshold points was not resolved, then we considered that the threshold was not adequate. If it was minimal, it went up; if it was maximal, it went down, according to the fixed adjustment step. If for two iterations there was no problem, then the threshold was analogously relaxed.

Finally, there was a third cycle (arrows with dotted lines) which consisted in redefining the dimensions and forms of intervention. This cycle was activated when the other two cycles, together, did not achieve the expected results. By its nature we could not automate it; therefore, the first two cycles were implemented on computers and the third remained a human task.

In general terms, the expectation was that possible problems should be resolved during the period that elapsed between two iterations of the respective cycle.

Let us now review the particular aspects of each level.

2.3.2. The operating cycles of courses at three levels

2.3.2.1. The cycle for students


The basis of all calculations are two indicators. Let G be the g groups in training, and A, the a students; we have C competencies that can be classified as satisfactory (+) or not (-), and S the total number of work sessions, of which s is the last performed. We can define:

  • Indicator of mastery of competencies at the time s (corresponding to the last of the S sessions of the course carried out so far):
    the number of competencies mastered in relation to the number of competencies the students should learn.
  • Indicator of advancement (the group) at the time s:
    the number of sessions completed in relation to the total number of sessions.
  • Each student was supposed to achieve or exceed the threshold of satisfaction of each competency, in the time available. In general, we expected that between two times, the achievement level of each student would be raised. It would also be reasonable to believe that there would not be much difference between students in the same group. Finally, we believed that the problems which might appear at one point or another, would be resolved in an acceptable period (we specified a month for all the levels).

Encouraged by these ideas, we could identify the areas that interested us:

  • Aspect of progress between sessions s and s-1 (the previous):
  • Aspect of domain-time synchronization: With 100% of the sessions covered, students had to master 100% of the skills; the relationship between the IDC and the APS anticipated whether the progression was appropriate.
  • Aspect of cohesion with the group: the relationship between the IDC of a trained individual a with the IDC average of her group g, at the time of the last session s held.

For each of these aspects, there was defined a threshold of satisfaction: if the aspect was satisfied, we were fine; if not, we had a problem in that area. We could define:

problem Pa1:
problem Pa2:
problem Pa3:

We also distinguished the aspect of overcoming problems, which was the basis for defining:

problem Pa4:

We would say that a student has a problem when Pa1 + Pa2 + Pa3 + Pa4 > 0.

2.3.2.2. The cycle for the groups of students

Each student group was expected to move forward in a fairly homogeneous manner: if there were too many individuals with problems, we would say that the group had a problem. We also compared each group with other groups taught by the same teachers, and if a group fell too far behind the others, there was a problem. As in the case of students, we wanted to avoid potential problems.

The indicators we constructed for this purpose were:

  • Aspect of homogeneity of the group: , where n is the number of students.
  • Aspect of cohesion with the meta-group: , where m is the number of groups g in G. IDC is the relationship of the group g with the average of the IDC of all the groups in G.

    For each one of these aspects, there was defined a threshold of satisfaction: if it was right, then we were fine; if not, then we had a problem in this aspect. We could define:
problem Pg1:
problem Pg2:

We also distinguished the aspect of overcoming problems, which was the basis for defining:

problem Pg3:

We would say that a group had a problem when Pg1 + Pg2 + Pg3 > 0.

2.3.2.3. The cycle for the teachers

In a manner very similar to that of the groups of students, we could distinguish various aspects related with the teachers. We could observe in the homogeneity between their groups: how many of the total number of groups had problems? And how did the indicator for mastery of competencies in their group look when compared with the total population of groups? As in the other levels, temporary problems were not supposed to persist.

The indicators we constructed for this purpose were:

  • Aspect of homogeneity of the meta-group: ,
    where n is the number of groups.
  • Aspect of cohesion of the meta-group with the universe:
    , where m is the total number of meta-groups.

For each one of these aspects, there was defined a threshold of satisfaction: if it was the right one, then we were fine; if not, then we had a problem in this aspect. We could define:

problem Pd1:
problem Pd2:

We could also distinguish the aspect of overcoming problems, which was the basis for defining:

problem Pd3:

We would say that a group had a problem when Pd1 + Pd2 + Pd3 > 0.

 

3. The first experiment

In the framework of the Executant Unit of ENLACES, of the University of Talca (Chile), we had adopted the system described. The process had been implemented in an information system. Only in this way was it possible to process the number of base observations: for a population of 2,000 students who had to learn 100 competencies, there were 200,000 records registered. With the challenges of computer implementation overcome, there is now in operation a system of web pages by which to access a database which organizes all the information.

The configuration of the thresholds was adjusted during the first phase of their use; as a result of some iterations of automatic classification evaluated by the coordinators, appropriate thresholds have been assigned.

Starting with the time the information system began to function, we have observed changes in two domains: the roles played by the various actors in the organization of the ENLACES project, and the configuration of the problem-detecting process.

Table III. Roles and role changes

We observed that when we used the process designed, there emerged a form of organization comparable to the “viable system model” (Beer, 1981). The indicators are a relatively objective basis for discussion among the actors—students with teachers, teachers with coordinators—and communications have become more productive.

The temptation to see the system as Big Brother is great. However, in its current definition, the roles and the use of the indicators with different time horizons keeps as much autonomy as possible at the students’ level.

The introduction of our system has had its effects on the process itself, by changing the indicators due to the discovery of problems not previously considered. For example, it has incorporated an indicator of punctuality for the observations to help the teachers register their considerations promptly. We accept as punctual the set of observations regarding a student with at least one upgrade during the past month. This assumes a percent of current PRA records, which can vary between 0 and 1:

problem Pa3b:

We also discovered the need for incorporating the current grade, for the sake of those students who had problems in making a mental connection between their profile and their indicators with that grade. The grade is determined by the relation between the points the student has obtained to date, and those which he can obtain on completing the course.

It should be emphasized that the indicators do not help diagnose the causes of apparent problems, but merely point out that something does not work: they are like a fuse. The focus is open that is, at any time we can understand that another indicator is needed, and perform steps of higher order learning that will allow us to improve the way we detect and correct problems.

We now know and manage each individual case, improving the quality of our service to students who really want to learn. Each participant in the process (students, teachers, coordinators, supervisors, etc.) has the opportunity to learn with a high degree of specificity the strengths and weaknesses of the competencies acquired during the teaching process. Thus, the more or less 1,000 students for 2001 have access to detailed information. The automatic calculation of grades provided by the system not only accounts for a quantitative aspect, but also a qualitative one, since it can be verified by any interested party. However, we cannot say that the difference between the grades obtained before using the system, and at this time, account for the difference between the learning achieved then and now. In spite of a greater current exigency, no conclusions can be drawn about performance. Still, we can say that the system fosters a stimulating environment for the autonomy and self-reliance of our students and teachers, and that our certification process is based on an objective, transparent and intersubjectively stable process. Our process leads us to recognize and anticipate problems, which is a prerequisite for improvement.

As presented here, we have an imperfect system, but one able to perfect itself so as to serve those who want to learn. We feel ourselves fully identified with the Indo-European root of the word learning, which means following a path.

 

4. Conclusions

We propose that a process of detection of possible problems with proactive resolution tends to improve training. We have shown the general design of this process: it means a system of rules organized in cycles at various levels, which not only classifies individuals, but also explains how these rules are configured from our own experience. This system can be understood as a cybernetic device and as a learning-action device.

In spite of our limited empirical experience, we hold that this design is useful (its description goes beyond the framework of this article) for the following reasons:

  • We have been able to detect and overcome problems in the behavior of the trainers;
  • We have identified and overcome weaknesses of the system of indicators itself;
  • We have obtained a change in the roles of the students and the teachers;
  • The transparency of the evaluation system has contributed to diminishing the number of complaints about grades.

However, we must point out that today we are the only users of this cybernetic focus; we suspect that the low rate of adoption is related with the effort required, and with the high transparency it generates.

We believe that this proposal is applicable in other training contexts. Obviously, our affirmation is subject to future research in case studies. Our efforts are currently directed toward the university environment.

 

References

Argyris, C. (1993). Knowledge for action: a guide to overcoming barriers to organizational change. San Francisco: Jossey-Bass.

Bateson, G. (1990). Vers une écologie de l’esprit. Paris: Editions du Seul.

Beer, S. (1981). Brain of the firm (2nd ed.). Chichester: John Wiley.

Bohm, D. (1980). Wholeness and the implicate order. London: Routledge.

Foerster, H. von & Glaserfeld, E. von. (1999). Wie wir uns selbst erfinden. Heidelberg: Carl Auer Systeme.

Forrester, J. W. (1994). Policies, decisions and information sources for modeling. In J. D. W. Morecroft & J. D. Sterman (Eds.), Modeling for learning organizations (pp.51-84). Portland, OR: Productivity Press.

Heidegger, M. (1993). Sein und zeit (17th ed). Tübingen: Max Niemeier.

Kolb, D. (1984). Experiential learning. Englewood Cliffs, NJ: Prentice Hall.

Martínez, J. (1998). La investigación cualitativa etnológica en educación. Mexico: Trillas.

Maturana, H. (1997). La objetividad: un argumento para obligar. Santiago de Chile: Dolmen.

Oppenheimer, R. (1954). Science and the common understanding. New York: Simon and Schuster.

Schaffernicht, M. (July, 2001). A method for managing organizational learning, proceedings. Work presented in the World Multiconference on Systemics, Cybernetics and Informatics, Orlando, Florida.

Weick, K. (1991). The nontraditional quality of organizational learning. Organization science, 2 (1), 116-124.

Wiener, N. (1958). Cibernética y sociedad. Buenos Aires: Sudamericana (Original work published in1950).

Translator: Lessie Evona York-Weatherman
UABC Mexicali

1Translator’s note: Before the feminist movement arose, in situations including both genders it was customary to use the masculine pronoun. Today, however, pronouns of both genders are used to avoid what is now seen as sexist language. To avoid the awkwardness of continually using “s/he”, “his/her”, we shall, in this paper, sometimes use the feminine pronoun, and sometimes the masculine.

Please cite the source as:

Schaffernicht, M. (2002). Using cybernetics for early detection of instructional problems in the Chilean educational system. Revista Electrónica de Investigación Educativa, 4 (2). Retrieved month day, year from: http://redie.ens.uabc.mx/vol4no2/contents-schaffernicht.html