Revista Electrónica de Investigación Educativa

Vol. 3, No. 2, 2001

A Combination of Quantitative and Qualitative
Strategies in Educational Research:
Reflections on Three Studies

Silvia Schmelkes del Valle
schmel@data.net.mx

Instituto de Investigaciones para el Desarrollo de la Educación
Universidad Iberoamericana, Ciudad de México

Cárpatos 15,
Col. Los Alpes, 01010
México, D. F., México

Abstract

This paper reviews three experiences of empirical research done by the author herself. In these an attempt is made to combine quantitative and qualitative methodologies of investigation. The first study focuses on the relationship between education and rural productivity. This work was carried out in four corn-producing areas of Mexico, and combines a survey with life stories. The second investigation is the study of projects that pursue combined aims of post-literacy and job training in thirteen Latin American countries. This work employed a questionnaire directed toward the managers of a large number of projects, plus case studies from a reduced number of the. The third study considered here analyzes the factors which affect the quality of elementary education in Mexico, and combines several instruments with a quantitative multivariate analysis and ethnographic observation in the classroom, school and community. The author extracts the lessons learned from these three experiences, and concludes that it is irrational to consider that each of the quantitative and qualitative methodologies belongs to the paradigms of social interpretation encountered. The combination of methodological approaches, in the three cases, permitted findings that would have been impossible through the isolated use of either of the two methodological approaches.

Key words: Educational research, quantitative methods, qualitative methods.

Introduction

It often happens in social research, and also in socioeducational research, that the choice of research strategy determines in advance the sector of the educational community with which one can communicate. Based on my experience, this is a mistake.

I have pointed out elsewhere (Schmelkes, 1993) that there is a common belief, derived from the historical connection between theoretical and methodological frameworks, that the techniques identified by the researchers are automatically located within a given theoretical current which, in turn, entails a particular option for dealing with the educational event as an object of transformation, and therefore, as a particular political option.

There is no denying that there can be a better adaptation of a particular set of techniques to the type of questions posed by specific theoretical frameworks. Therefore, it would be difficult to calculate the amount of human resources required to establish the flow of educational demand without resorting to statistical methodologies, and without bringing together a great accumulation of quantitative information. In the same way, it would be difficult to prove the advantages of a new teaching method or those of a new curriculum without having access to an experimental design.

However, the mechanical application of a paradigm to techniques suffers from an over-simplification, and hinders important processes of educational research, both because it establishes barriers, often fictitious, between researchers, and because it generates mechanisms of social control that limit the researchers’ options. What really makes the difference between the researchers themselves is their epistemological approach, and not the selection of particular techniques. This is ultimately the way the researcher believes she1 can know reality, the value the data gives her, her way of perceiving the processes, the method by which she arrives at tentative explanations, the way she submits them to testing, the style of controlling what she anticipates concerning reality with what reality indicates to her, what constitutes essential differences in the way to approach the scientific task in education.

But the selection of techniques is often independent of the researcher’s epistemological approach. The techniques are selected by a set of additional factors, among which, the nature of the question itself ought to outweigh the research. However, other factors are involved, and must be considered: the time available; the resources and who provides them; the previous knowledge accumulated about the specific topic; the specific study’s level of connection with other studies on the same topic, and so on.

Thus, the way each researcher collects the data for obtaining information required by his research question must be judged not in itself, but in terms of their relevance to the kind of question, and considering the material possibilities that sometimes limit their options. More important are the questions related to the type of data he collects, and how these data will be used, because no choice of investigative techniques, no matter how innovative they are, frees the researcher from theoretical categories. Nor do they free the researcher from expressing the objective of contributing to the theoretical construction—which makes it essential to transcend the empirical data, to contextualize it in time and space; and to explain and project it in its meaning regarding the scientific task’s object of study.

In this paper I will summarize the results of three studies in which I participated, and in which I have combined and tried to link up quantitative and qualitative strategies for collecting and analyzing data. I will endeavor to show how the use of this approach, unorthodox and possibly eclectic, leads to discoveries that could not have been achieved with the use of either strategy alone.

II. The three studies

A. Productivity and learning in the rural setting

The first study I am going to talk about was carried out between 1982 and 1984. It was part of a comparative study done cooperatively with three other Latin American countries (Paraguay, Brazil and Peru), and sought to understand better the relationship between education (formal, nonformal, informal) and productivity, both of corn and of the rural economic unit as a whole. In the case of Mexico, the research was also performed as comparisons between four seasonal corn-growing zones with very different features and productivity (Schmelkes, Renteria and Rojo, 1986).

The quantitative aspect of the study consisted of applying a questionnaire to a representative sample of around 400 farmers in four selected areas (southern Nayarit, Zapopan, Jalisco; Calpan, eastern Puebla and Morelos). Data from this questionnaire were subjected to statistical analyses of various types, including multivariate analysis, both parametric and non-parametric.

Some of the most important results of this research are the following:

The quantitative study allowed us to see that the most productive strategies were different in each of the regions studied. For example, in Jalisco the most productive strategy for growing corn is embodied in the more modernized producers and those with more access to formal education. In contrast, in Morelos, the most successful producers are those who have opted for a more “rural”strategy: the less proletarianized—those who have diversified their agricultural production more. In Puebla, it is the less-modernized producers and the most proletarianized who have the most productive strategies. It is the qualitative study of the characteristics of the regions, however, that permits us to understand this. Therefore, Zapopan, in Jalisco, is the most productive seasonal-corn region in the country. It is a more modern region, less “peasant”, mostly integrated into urban life, above all, through the studies and the work of its children. The modernization process is already consolidated, with a little-diversified agricultural production and a production model with little dynamism. Morelos, however, is a region undergoing rapid and multidirectional transformation, with a widely-diversified agriculture, in which consumption, and therefore the cultivation of corn, has come to occupy a secondary role. Farmers who have been less proletarianized, and who maintain a strategy of autoconsumption—and because of that, consider corn more important—turn out to be the most productive. The area of Puebla is a poor region of Indian origin, with adverse conditions for agriculture, with strongly rural features, but with unfavorable agricultural work conditions that surround seasonal semiproletarianization. The modernization process has been taking place among those farmers who have more land. However, those who have followed the policy of modernization, developed by the Puebla Plan for a much larger region and poorly suited to this specific micro-region are, as a result, the least productive.

In the quantitative study we found that in all the regions there was a group of farmers who benefited from agricultural extension, and another group that was suffering from it. Through this study, we were able to describe both groups. The in-depth study conducted with agricultural technicians allowed us to discover, in all the regions, the existence of a single prevailing agricultural-extension model, which differs between regions, but which operates consistently within them. Meanwhile, the quantitative study also permitted us to confirm the existence, in all the regions, of a considerably-heterogeneous population of producers. The combination of both phenomena gives us as a result the selective effect of agricultural extension. Managing a single extension model leads not only to the exclusive advantage of those whose conditions fit the model, but what is worse, harms those who do not possess those conditions.

We also discovered that in all the regions there was a group of small producers who are more productive when they have not been in contact with informal education (agricultural extension). In the qualitative study we found that technicians generally disregard the farmers’ experience and employ a strategy of vertical transmission of technical information which, in overwhelming that of the farmer, partially destroys the technical knowledge derived from experience. This results in a decrease in productivity and income in certain sectors. The interviews with the technicians enabled us to confirm that they are not familiar with the rural reality. Their relationship with the farmers is generally domineering, punctual, distant, authoritarian; they clearly perceive the farmers as ethnocentric, ignorant, passive, apathetic, traditional, closed to change. This has led us to the conclusion that the relationship between technicians and farmers is not an educational relationship.

The in-depth interviews with producers (which were conducted on at least four occasions, with an average duration of five hours) allowed us to go through the process they have followed throughout their independent productive lives, in the introduction of modifications to their global production strategy. It was in these interviews that we discovered, among many other things, that farmers generally experiment with a productive innovation before adopting it, and in doing so, adapt it little by little to their conditions. These same interviews allowed us to analyze in greater depth the heterogeneity of the rural population, and consequently, their production strategies.

Thus, the quantitative aspect of this study allowed us to discover and give dimensions to a series of problems, while the qualitative aspect allowed us to explain them. In this manner, the set of recommendations formulated is the fruit of a new vision, enriched by the relationship between education and rural economic life. This was made possible, primarily, by the fact that we incorporated, in an organized manner, the qualitative strategy. Note also that this combined approach allowed the design and accomplishment of an action-research project aimed at transforming the educational practice of agricultural extension. The quantitative aspect of the research allowed us to justify this need. The qualitative provided us with the foundation to give it content.

B. Post-literacy and work in Latin America

The second study to which I will refer was conducted between 1986 and 1987, and attempted to learn how various programs—official and non-governmental—were dealing with the relationship between education and work illiterate in populations and those with precarious literacy. This study, coordinated under my directorship, was conducted in 13 Latin American countries, (Schmelkes, 1990).

Also, in this case, we adopted a research strategy that combined the quantitative with the qualitative. The quantitative aspect consisted in applying a questionnaire to those in charge of 76 programs linking education with productive work, or work which offered job training. The qualitative aspect consisted of making case studies of a selected group of 13 of these programs. Interviewed were the program’s various actors—supervisors, instructors, participants, graduates, and in some cases community members who used their services. Direct observations (educational sessions and team meetings) were made of the specific environment. Local teams conducted the study—both aspects of it—in their own countries. The questionnaire for the 76 programs was identical in every case. In contrast, for the case studies, only selection criteria were indicated, and general guides were developed, from which each team designed its own strategy. In addition, each local team made a report on the set of programs studied in each country. In Mexico, the questionnaires were cataloged according to the categories of analysis that guided their design.

I shall mention only some of the findings which clearly show the connection between the two research strategies, and which are also the most important. The first relates to the target subjects (recipients) of the programs. Besides the characteristics derived from the quantitative analysis of the subjects as marginalized, of precarious literacy, subject to the consequences of living in a situation of structural poverty; we discovered in this analysis the presence of the organized subject, especially in the rural setting. The case studies allowed us to delve into the role played by the target subject in the definition of alternative actions in programs now in action:

There are programs created by the demands made by the subjects themselves.
There are programs whose claims exceed the original expressions once they are assumed by the farmers’ organization promoted by the original program.
In some programs the very conditions of the specific sector addressed are imposed on the original proposals and give rise to new projects.
In other cases, the popular sector presents itself in some forms of resistance. Thus, high dropout rate in plainly educational programs; the difficulty of extending the attempts at collectivization in agricultural production; and the marked preference for accepting jobs outside the blue-collar environment in a program aimed toward the working woman, should be considered as evidence of resistance.

It is evident that there are programs more penetrable than others by the explicit or silent exigencies of the subjects, and by the contextual conditions that restrict the achievement of the expected impact. The first are programs more dynamic and more ambitious, whose focus is clearly focused on achieving the intended objectives, and not on the administration of an educational service. The degree of penetrability of the program by the subject and by the context seems to be decisive in both the variety of program objectives and their outcomes among the beneficiaries.

The second conclusion concerns the role of the educational in the programs. The quantitative study enabled us verify that most of the programs understand the relationship between education and production under the assumption that “to learn to do, it is necessary to continue incorporating the knowledge to do”, or assume the principle of “learning by doing”, although there are several ways to understand it. The majority of the programs are inclined toward the systematic and intentional incorporation of knowledge into the production process. Others allow the production process to express the educational demands and requirements encountered when these appear as necessities. Others believe that the production process itself is educational. Finally, much fewer are those who maintain a dialectical confrontation between what the subjects already know and the scientific and technological knowledge already accumulated, but external. These programs incorporate reflection into the production process as central educational methodology.

However, it is in the case studies that these different ways of understanding the role of education, and of understanding the relationship between education and production plainly appear. In the case of programs that centrally attempt the production of goods and services, or the transformation of the economic conditions of production, education operates as something that accompanies the core processes and provides conditions for better development, providing the necessary training for the proper achievement of the objectives. The goal of transformation is that which marks content, pace and integration of educational activities within the process. Education clearly comes to fulfill an instrumental role. However, as there is a space for the educational task, this activity has the potential to transcend its instrumental role, and there are introduced contents which the productive activity itself does not provide, but which are also not strictly necessary for its development. Therefore, education separates from the instrumental; in so doing, it opens the doors—and advantage is taken of this in every case—to offer educational content oriented toward becoming aware, toward organizational and evaluative training and training for self-management.

In contrast, in the programs that are educational at their core, major differences are observed in the types of content prioritized and included in the programs. Thus, the programs that define themselves as job training, differ in the degree to which there are introduced contents oriented toward consciousness-raising, organization or values formation. The programs of postliteracy or basic education show obvious problems linking their contents with issues related to production and work. In one case, the programs are bifurcated: one part emphasizes the productive aspect; the other, the aspect of postliteracy.

Paradoxically, the educational activity is much clearer in the case of programs whose core objective is economic or social rather than educational. Here the instrumental role of the educational activity is accepted in all cases, and in many of these, its function is recognized as potentializing the participation and organization necessary for the economic and social process promoted. In contrast, in the case of essentially-educational programs, education is seen as a prerequisite for access to employment or self-employment (and then there are added different sorts of educational content), or is seen as a good in itself. In these latter cases the role education plays in the transformation of economic and social conditions of the individual and the group is theoretical, and medium to long term, under the assumption that the literate subject will always have more opportunities; or in the case of basic education, there will always be an opportunity to get it.

The study concludes by discovering both potential and limitations of the educational programs related to productive work. The quantitative work allowed us to classify the programs, to relate their types with their results and the impact of describing strategies followed and the difficulties encountered. The qualitative study allowed us to interpret the above, and to generate theoretical categories of analysis that can open new channels both for action and for future research.

C. The quality of primary education services in Mexico

The third case whose findings I would like to summarize, is a study on the quality of primary education in the state of Puebla between 1990 and 1992. This study was also part of a comparative investigation, coordinated by the International Institute for Educational Planning of UNESCO in Paris, and carried out Mexico, Guinea, China and India. The study attempted to delve more deeply into the factors that explain the quality of the process and the educational results of introducing the contextual factor as an essential element for understanding this phenomenon better.

In all the countries, the objective was the same, and the principal variables were identical. However, the approach depended on each of the local teams responsible for the study. In the case of Mexico, the study involved a comparison between schools in five zones, homogeneous to its interior, but highly contrasted with one another: a middle-class urban area, a marginal urban area, a developed rural area, a marginal rural zone, and an indigenous zone (Schmelkes, 1997).

In each of the areas we studied some 16 schools (77 in total). In every school, we interviewed principals, teachers, people who knew the community and school, students in the fourth and sixth grades, and 12 parents. In addition, students in the fourth and sixth grades were given a test of basic skills (communication, functional use of mathematics, and individual and collective health care). Also, we observed the process of teaching and learning in the classroom (twice, for an hour, in each of the grades studied), we observed the normal running of the school for at least three days. A collective interview of all teachers from each campus was done. Regionally, in-depth, unstructured interviews were conducted with at least three qualified sources in the region (mayor, priest, doctor, historian or writer, teacher with many years’ work experience in the region).

The quantitative analysis consisted of meticulous work with the data proceeding from all the instruments. The most extensive part describes the conditions of offer and demand for primary education. Bivariate and multivariate analyses were also conducted. The qualitative analysis allowed a detailed description of each of the regions. In addition, the textual information from all the previous instruments and researchers’ field diaries was used to make an in-depth analysis of the schools with the highest and lowest scholastic achievement in each of the areas studied. This was done in order to obtain a more holistic view of the causes of the qualitative differences found.

With great effort to summarize the main conclusions of this study, we noted the following:

The quantitative study shows that the quality of the results of primary education in the schools studied is a serious problem, manifested by the fact that the vast majority of students who are about to graduate fail in functional literacy.
The quantitative analysis also shows with great consistency that the distribution of educational quality is enormously uneven. Indeed, we found diametrically-opposed situations that show the existence of two very different realities: that of the schools of the middle-class urban zone, and that which prevails in all the rest (marginal urban, developed rural, marginal rural and indigenous). These completely opposite situations are such that the sixth-grade scholastic achievement in rural and marginal schools is equivalent to—and in many cases, far less—than that of the fourth grade in the urban middle-class schools.
The study sizes up something we all know already: that the characteristics of the demand for primary education vary enormously in terms of socioeconomic conditions, of health and nutrition, and culture; as well as in attitudes and behaviors of families concerning the school and their children’s educational trajectory. These characteristics are correlated with the learning objectives, but also are strongly correlated among themselves, so that the effect of these characteristics regarding learning is not linear, but synergistic.
Perhaps one of the most interesting results of the study has to do with the finding that the educational system not only ignores these differences, in the sense of implementing homogeneous educational strategies in heterogeneous conditions, but that the situation appears much more serious. In reality, the educational system in its daily operation, is offering—both quantitatively and qualitatively—differential inputs to different contexts. This differentiation operates in exactly the same direction as the demand conditions, so that in those contexts where families are in difficult socioeconomic conditions, the parents lack the necessary education to help their children with their school work, often require the labor of their children at home or in the field, and have poor health and nutrition. This is where the schools are poorly equipped, lack an official director, and are seldom visited by the supervisor. Thus it seems that the quality of scholastic inputs and their daily function tends to conform to the conditions of the demand. Therefore, educational policy not only perpetuates existing inequalities, but aggravates them. This panorama is revealed in both quantitative and qualitative analysis, which are reliable in their results and provide complementary elements.
The most important factor in explaining the differences in educational quality is the teacher. This result is consistently obtained in both quantitative and qualitative analyses. A good teacher, or a good team of teachers, sometimes led by the director, is able to get students in a poorly-equipped school in a very marginal-gain community to achieve learning outcomes equivalent to those of the best schools in privileged contexts.

However, these teachers are the exception, not the rule. The reason for this is clearly derived from the qualitative study:

Teachers do not always control the content of primary education.
In general, teachers are not adequately trained in effective teaching practices. The predominant teaching model in the schools studied focuses on the teacher; it addresses the class as a whole; it is based solely on the textbook as a source of information and practice, aided by the blackboard as the only teaching support. Teachers generally lack strategies to cope with multigrade realities. Few teachers encourage student participation; fewer still are those who know how to deal with special learning problems. Teamwork among students is unknown. Almost never is advantage taken of community resources to provide material and learning experiences. Learning experiences are generally monotonous, and consist mainly of reading the textbook, copying in the notebook, and doing exercises dictated by the teacher or written on the blackboard. Teachers give no importance to the processes of reasoning and application of knowledge.
Teachers are left alone to work in absolute solitude. They receive no educational support either from the principal or from the supervisor. They rarely have access to refresher courses. They read, but what they read is barely related to their professional activity. Their contact with parents and the community is sporadic.
The teachers, in general, are unhappy with their working conditions. Although the majority chose the profession as a vocation, the low wages they receive make them wish they had picked a different career. They need additional income to support a family. Since opportunities for earning this are limited in rural areas, teachers want to leave those as soon as possible.

In this study, perhaps the back-and-forth swinging between the findings derived from quantitative and qualitative analysis is clearer. The quantitative study found problems, related them, quantified them, and measured their severity. The qualitative went into their causes in depth, described the way they function, and finally improved the hypothetical path toward solution.

III. Final Reflections

My personal experience, my direct participation in these three studies, allows me to be sure that the most important results of each investigation could not have appeared if either of the two strategies had been used separately. In no case have we attempted to address the reality from two different perspectives, but from only one, which connects two strategies. What has been achieved over time lies in the intensity of the relationship between the two strategies. While in the first two studies, the interaction between both methods was privileged—although never exclusive—in the final interpretive moment, in the final study, these were was permanent: the partial results of the quantitative study determined the questions I had to formulate in each zone so as to describe it, as well as the questions I had to pose so as to express the most qualitative information proceeding from the instruments. Similarly, the partial results of the qualitative information generated new questions about the quantitative analysis. This was what allowed us, for example, to reach the important conclusion about the centrality of teacher and school.

It is not my intention to give the impression that the combined quantitative-qualitative strategy is simple. It is full of difficulties, and I want to end by mentioning three of them:

There is a clear, continuing tension between the two strategies. This is because, as is evident, each has its own conceptual apparatus and its consequent demands for consistency and rigor in each step of the analysis. In turn, the coupled movement between the two generates its own exigencies and the need to respond to a dogmatism still under construction. The uncertainty of the researcher confronting this is enormous.

We have already mentioned the existence of two circles in the academic community: the quantitavists and qualitavists. The strategy to which we have referred, since it is a “hybrid”does not satisfy either of them. The challenge of convincing on the basis of the solidity of the findings sometimes goes beyond the capabilities of the research team.

The skills of one type of research are different from those of another. It is practically impossible to expect a researcher to unite them. More difficult still is it to work as a team with people who specialize in one type or the other.

The process of connection between the two strategies is obviously an adventure, an unfinished proposal, and therefore, full of shortcomings and difficulties. Personally, I think I have overcome the epistemological problems implicit in attempting to link a more “empiricist”concept with a more “constructivist”or interpretive one. The results convince me that the adventure is worth it.

References

Schmelkes, S., Rentería, A., & Rojo, F. (1986). Productividad y aprendizaje en el medio rural: Un estudio en cuatro regiones de México. Revista Latinoamericana de Estudios Educativos, 16 (2), 15-56.

Schmelkes, S. (Coord.). (1990). Postalfabetización y Trabajo en América Latina. Pátzcuaro: CREFAL, UNESCO-OREALC.

Schmelkes, S. (July-December, 1993). Aspectos conceptuales y metodológicos de la investigación educativa. Dimensión Educativa, 1, 15-22.

Schmelkes, S. (Cord.). (1997). La calidad de la educación primaria: Un estudio de caso. Mexico: Fondo de Cultura Económica.

Translator: Lessie Evona York-Weatherman
UABC Mexicali

1Earlier in the twentieth century, English, like Spanish, used the masculine possessive pronoun in generalized statements to indicate both genders of humankind. Since the advent of the feminist movement, however, such usage in English has been considered sexist, is generally avoided, and has been replaced by expressions such as “his and her”, “s/he” etc. (Fennel, Francis, 2002). While these non-sexist devices can be comfortably employed now and then in a work, their constant and continual use becomes awkward. In this work, in order to avoid the annoying repetition of such constructions, we shall at times use the feminine pronoun (she, her, etc.) and at times, the masculine (he, him, his, etc.).

Please cite the source as:

Schmelkes, S. (2001). A combination of quantitative and qualitative strategies in educational research: Reflections on three studies. Revista Electrónica de Investigación Educativa, 3 (2). Retrieved month day, year from the World Wide Web: http://redie.ens.uabc.mx/vol3no2/contents-schmelkes.html