Revista Electrónica de Investigación Educativa


Vol. 11, Num. 1, 2009

Validation of the Defining Issues Test
in Medical Students at
El Rosario University,
Colombia1

Isabel Regina Pérez-Olmos
isabel.perez@urosario.edu.co

Oficina de investigaciones
Facultad de Medicina
Universidad del Rosario

Kra. 24 # 63C-69
Bogotá, Colombia

Mónica María Dussán Buitrago
monimad@msn.com

Oficina de investigaciones
Facultad de Medicina
Universidad del Rosario

Cll. 170 # 68-73
Bogotá, Colombia

(Received: June 26, 2007; accepted for publishing: June 21, 2008)

 

Abstract

Learning medicine as a discipline is a complex process that demands more dedication every day. The acquisition of technical knowledge must be part of an integral education which favors the development of high level humanistic values in order to provide the physician with personal tools for the proper resolution of the ethical dilemmas that he/she will face during his/her education and professional practice. The assessment and improvement of these aspects are likewise necessary in the physicians’ education. For the purposes of this research, the reliability of the Opinion Questionnaire on Social Problems (COPS, for its acronym in Spanish) was evaluated. This questionnaire is the Mexican version of the Defining Issues Test, and it was adapted by the authors. The test item performance was determined in medical students from El Rosario University at Bogota, Colombia in 2006. The obtained Cranach’s Alpha was 0.76, similar to the registers in scientific literature. COPS is considered to be a reliable instrument for the evaluation of moral judgment in medicine students at this University.

Key words: Test validity, medical education, moral development, moral judgment.


 

Introduction

Learning processes of medical students are more complex every day, and they also demand more dedication. Students face ethic and moral dilemmas since the beginning of their education, and frequently, academic training is insufficient for such problems. This situation makes necessary the improvement and continuous evaluation of the integral education programs for the future physicians.

From this perspective, it is essential to find a way to evaluate moral development, and more specifically, to know the instruments for its examination. There are questionnaires to appraise moral competences and development. Kohlberg’s Moral Judgment Interview (Kohlberg, Power, and Higgins, 2002) is one of them, with its modified versions, like Gibbs’ Sociomoral Reflection Measure (Basinger and Gibbs, 1987); another one is the Rest’s Defining Issues Test, DIT (Rest, 1979).

The first one was developed by Lawrence Kohlberg in 1958 and it consists in posing a series of dilemmas, such as the Heinz dilemma (Heinz steals the drug) where the subject assessed has to make a decision about the posed conflict, and the reasoning that lead him/her to make that decision is what is relevant. The interviewer asks a series of questions which lead the subject to consider some elements, such as personal interests, the law and justice, among others, with the affirmations that were previously classified to correspond each moral development level as described by Kohlberg. Finally, the interviewer analyses the answers in order to determine the subject’s level of moral judgment development which can be pre-conventional, conventional, or post-conventional (Crain, 1985), as Table I shows below.

Table I. Lawrence Kohlberg’s six stages of moral
development, with norms and mental operations

Erasmo’s test (Villegas, 1994, 1998, 2002) is also considered. According to its author, this test integrates the assessment of the cognitive aspects of moral judgment with the interviewee opinions on the moral feelings and consequences that he/she anticipates for his/her hypothetical action before the posed dilemma.

With regard to the instruments for the evaluation of the moral competence of medical students, there is the original English version of Hebert, Meslim, and Dunn’s Test (1992). This test consists of clinical vignettes used to determine the ethical sensibility of medical students. The purpose of the test is to establish the ethical principles involved in each of the posed situations, and more specifically, to determine if students identify the autonomy, beneficence and justice principles of each vignette. To the date of this research, no Spanish validated version of this test has been found (Rest, 1979; Hebert, Meslim, and Dunn, 1992; Self, Baldwin and Wolinsky, 1992; Savulescu, Crisp, Fulford, and Hope, 1999).

The Defining Issues Test (DIT) (Rest, 1979), based on Kohlberg’s theory, formulates a series of socio-moral problems and dilemmas devised by James Rest and his collaborators from the University of Minnesota in 1979. Its approach allows for flexibility in the rigid structure of Kohlberg’s six stages of moral judgment. Besides, it lays a foundation for the social nature of morality development from the approach of neo-kohlbergian authors (Barba, 2002).

Starting with the answers given by the subject who is assessed on the posed dilemmas in the DIT Test, the correspondence among them and the level of Kohlberg’s post-conventional moral development (stages 5 and 6) are identified. Afterwards, the P index, a score calculated from the values assigned in the test’s handbook (Rest, 1979) is also determined. This is the DIT most popular score; it is expressed in percentage and it determines the importance given by the subject to the most advanced moral principles when considering a social dilemma.

This test is originally written in English and requires the authorization from the University of Minnesota (Rest, 1979) so that it can be used. Two versions have been developed; one of six dilemmas, and another one of three. Both versions have been translated into Spanish and are currently going through a validation process in Mexico (Barba, 2001, 2002; Romo, 2004). Internal consistency was 0.7, according to Cronbach’s Alpha, in the long English test version, and between 0.74 and 0.76 in the short version. Recently, the Spanish short version was validated in Venezuela as well. Venezuelan authors obtained a Pearson’s test-retest reliability coefficient of 0.65 and a Cronbach’s Alpha of 0.71 (Zerpa and Ramirez, 2004).

The DIT test has also been used in research on U. S. medical students. The results showed significant differences in the P index over the different semesters. In one study, the average P index found in first year students was 47.7, while 53.7 was found in second years and 56.5 in third years; which clearly shows the indicator rises as medical training advances (Self, Olivarez, and Baldwin, 1998). However, these findings differ from the results found in most of the revised literature, where as the semester advances, the level of moral development decreases (Self and Baldwin, 1998).

Other revised research contain evidence of significant improvement on moral judgment after educational training with different pedagogical approaches, such as medical ethics classes (Self, Baldwin and Wolinsky, 1992), or case discussions (Self, Olivares, and Baldwin, 1998).

 

I. Methodology

With the support of the authorities at El Rosario University and its School of Medicine, the Opinion Questionnaire on Social Problems (COPS, for its acronym in Spanish), a Mexican abbreviated version of the James Rest’s Defining Issues Test, was administered to 896 students from the 1st through the 12th semester of the Bachelor’s Degree in Medicine.

The first objective was to validate the test and to determine the test item performance. For this purpose, the students were called according to a schedule which was previously arranged with the coordinators of each semester, as part of the integral education activities, and so attendance was mandatory. COPS answer sheet was designed to be read with an optical test score reader and with it anonymity was granted to students, which stimulated truthful answers. An Excel database was created with the answers and statistic analysis was carried out with SPSS 14.0 for Windows.

To establish the test reliability on this research, the Cronbach’s Alpha of COPS, the Mexican short version of DIT adapted by the authors, was determined. The test item performance was also determined by the medical students at El Rosario University in Bogota, Colombia, in 2006.

1.1 Determination of reliability for COPS answers

The DIT short version in Spanish included three stories: Armando and the medicine, the school newspaper, and the physician’s dilemma. The latter was given more importance since it posed an euthanasia related dilemma. Test administration and scoring were based on the original handbook. For this purpose, evaluations were first tested for internal consistency in order to discard those that did not achieve the minimum level of answer reliability. One of the reliability components was the M score; it was given by the score the subject assigned to certain items of each story, and that might seem important for the dilemma analysis due to their sophisticated vocabulary; although, they were in fact meaningless for the test and did not correspond to Kohlberg’s stages of moral development. Questionnaires with an M score higher than 4 were excluded according to the author’s scoring guide for the short version of the test.

Other components to assess the consistency of the answers were two tests of internal consistency reliability. The first one consisted on revising answers in the second section of every story. If more than nine items were scored with the same level of importance in two stories, the questionnaire had to be discarded.

Finally, the second consistency test revised the two items scored as the most important ones in the third section of every story. If one of them was classified as the most important one, there could not be a more important one in the second answer section. If an item was classified as the second most important in the third answer section, the one that could be more important in the second section was the one classified as the most important one. If a questionnaire had a story with more than eight inconsistencies, or had one inconsistency in all three stories, it was discarded.

 

II. Results

The valid sample for the analysis consisted of 784 questionnaires. The COPS internal consistency reliability test discarded 12.5% (n=12).

The results of the first section, where students had to make decisions on the posed dilemma, are shown below:

On the second section of the test, the 12 items of every story had to be scored by the subject who was evaluated according to the level of importance that was assigned by him/her to the posed social dilemma. These levels were scored with the Likert scale (0-4), from none to high importance. Each item corresponded to a stage of moral development: it did not have a meaning (item M) or it was an anti-establishment item (item A).

In the story Armando and the medicine, the most important items (average score higher than 3) corresponded to stages 6 and 5, equivalent to the post-conventional moral development level and stage 2 of the pre-conventional level. These items showed heterogeneous answers with a standard deviation (SD) near to 1. The ones scored with little importance (averages lower than 1.5), corresponded to stage 3 (conventional level), stage 5A (post-conventional level), anti-establishment item or item A, irrelevant; they showed more heterogeneous answers on the best scored ones (SD higher than 1). The rest of the items obtained scores of moderate importance and the values of standard deviation showed heterogeneity (see Table II).

In the second story, The school newspaper, the higher averages corresponded to the items of the post-conventional level with standard deviations that also showed heterogeneous answers. As for the items with low score averages, something similar to the obtained in the first story happened, the lowest score was for an irrelevant item, followed by one of conventional level, and answers were heterogeneous (see Table II).

In The physician’s dilemma, the items with higher average scores had less answer heterogeneity; for example, the item 5B of post-conventional level; while the item with a lower score, of level 2, showed high heterogeneity in the answers (Table II).

Table II. Arithmetic mean and standard deviation of every item on the COPS’ three stories in medical students at El Rosario University, second semester 2006.

The SPSS software was used for the COPS reliability analysis with the Cronbach’s Alpha statistic test. The research sample consisted of 706 registers after the software discarded the questionnaires that lacked of the required information. The COPS test, short version (three stories) showed a global Cronbach’s Alpha of 0.766. Stability was shown in the internal consistency of the test, since the obtained Cronbach’s Alpha did not depend predominantly and exclusively on just one item of the questionnaire (Table III).

 

III. Discussion and conclusions

The reliability analysis of the COPS, Mexican short version of Rest’s DIT, showed a Cronbach’s Alpha of 0.766, similar to the 0.71 registered in the Venezuelan literature as well as in the United Sates literature, where the values obtained with the short version of DIT were between 0.74 and 0.76 (Zerpa and Ramirez, 2004). This showed consistency of the reliability levels obtained from the original short English version and from the Mexican version (in Spanish). On the other hand, reliability showed that the test is acceptable for the evaluation of the moral judgment development of Colombian university students, in particular, medical students at El Rosario University in Bogota, Colombia.

Internal consistency of the test was homogeneous, since the variation in the Cronbach’s Alpha was minimal when one of the items in the three stories was eliminated, which means that the test reliability is explained by the set of items which constitute the three stories (Table III). In addition, when calculating the Cronbach’s Alpha separately for every story diminished the statistic indicator, which caused loss of the test reliability obtained from the three stories.

Regarding the score average of the post-conventional level items obtained in the three stories, the different distribution shown by the scores of each story is remarkable (as seen in Table II). While the first two stories had two post-conventional level items with a high average, the third one had only one post-conventional item.

This can be explained by the specific content of the moral dilemma described in each story, which could make the decision easier or harder, and also by the importance assigned to each item. Thus, while on the first story the confronted values are life preservation versus theft of private property, in the second story the dilemma is the defense of freedom of speech versus the defense of the principle of authority. Finally, in the dilemma of the third story, the situation implicates a hard decision on euthanasia and the freedom of self-termination of life on a terminal patient. No mention about these aspects of the DIT was found on the revised literature.

Table III. COPS reliability analysis, Cronbach’s Alpha coefficient in
medical students at El Rosario University, Colombia, in 2006

The short version of James Rest’s (1979) Defining Issues Test (DIT) on its Mexican short version, Opinion Questionnaire on Social Problems (COPS, for its acronym in Spanish) (Barba, 2001), and adapted by the authors, showed an acceptable reliability with a Cronbach’s Alpha of 0.766 when administered to medical students at El Rosario University, Bogota, Colombia.

 

References

Barba, B. (2001). Razonamiento moral de principios en estudiantes de secundaria y bachillerato. Revista Mexicana de investigación educativa, 6 (13), 501-523.

Barba, B. (2002). Influencia de la edad y de la escolaridad en el desarrollo del juicio moral. Revista Electrónica de Investigación Educativa, 4 (2), 23-25.

Basinger, K. S. & Gibbs, J. C. (1987). Validation of the sociomoral reflection objective measure-short form. Psychological Reports, 61, 139-146.

Crain, W. C. (1985). Theories of development. New York: Prentice-Hall.

Hebert, P., Meslin, E. M., & Dunn, E. V. (1992). Measuring the ethical sensitivity of medical students: a study at the University of Toronto. Journal of Medical Ethics, 18, 142-147.

Jaramillo, R. & Martínez, R. (2004). Evaluación del desarrollo moral en instituciones educativas oficiales (Internal document). Unpublished manuscript, Ministerio de Educación Nacional, Bogota, Colombia.

Kohlberg, L., Power, F. C., & Higgins, A. (2002). La educación moral (2nd. reimp.). Barcelona: Gedisa.

Rest, J. (1979). Developing in judging moral issues. Minneapolis, MN: University of Minnesota Press.

Romo, J. M. (2004). Juicio moral en estudiantes de bachillerato. Un diagnóstico. Revista Iberoamericana de Educación, 35 (1). Retrieved 19 May, 2007, from: http://www.rieoei.org/edu_val17.htm

Savulescu, J., Crisp, R., Fulford, K. W. M., & Hope, T. (1999). Evaluating ethics competence in medical education. Journal of Medical Ethics, 25 (5), 367-368.

Self, D. J. & Baldwin, D. C. Jr. (1998). Does medical education inhibit the development of moral reasoning in medical students? A cross-sectional study. Academic Medicine, 73 (10), 91-93.

Self, D. J., Baldwin, D. C., & Wolinsky, F. D., (1992). Evaluation of teaching medical ethics by assessment of moral reasoning. Medical Education, 26,178-184.

Self, D. J., Olivarez, M., & Baldwin, D. C. (1998). Clarifying the relationship of medical education and moral development. Academic Medicine, 73, 517-520

Villegas, M. (1994). Validación de un modelo para explicar y predecir la acción moral (Final report). Unpublished manuscript, Colciencias, Bogota, Colombia.

Villegas, M. (1998). Nueva validación de un modelo para predecir la acción moral (Final report). Unpublished manuscript, Colciencias, Bogota, Colombia.

Villegas, M. (2002). Educación para el desarrollo moral. Bogota, Colombia: UNIANDES.

Zerpa, C. & Ramírez, J. (2004). Un instrumento de medición del desarrollo moral para estudiantes universitarios: Defining Issues Test (DIT). Revista de Pedagogía, 25 (74), 427-450.

Translators: Eleonora Lozano and Ayla de la Cruz

1The authors thank Dr. Bonifacio Barba and the Mexican collaborators who facilitated the Opinion Questionnaire on Social Problems (COPS), Mexican version of the Defining Issues Test (DIT). They also thank other researchers who facilitated bibliographical material on the subject, more specifically, the Venezuelan researcher engineers: Carlos Zerpa and Jorge Ramirez; Dr. Rosario Jaramillo from the National Ministry of Education of Colombia; the sociologist Myriam Buitrago, supervisor of the District Department of Education of Bogota ; Dr. Andres Molano, psychologist from Universidad de los Andes. To all the coordinators and professors of each semester at the School of Medicine of El Rosario University (Colombia), to the medical students, the directives from the School of Medicine and El Rosario University, and the administrative staff that supported this research.

Please cite the source as:

Pérez Olmos, I. & Dussán-B, M. (2009). Validation of the defining issues test in medical students at El Rosario University, in Colombia. Revista Electrónica de Investigación Educativa, 11 (1). Retrieved month day, year, from: http://redie.uabc.mx/vol11no1/contents-perezolmos.html