As with science, there is no universally accepted philosophy of evaluation. The paucity of a univocal philosophy, however, has not precluded extensive discussion and debate on the philosophical assumptions to the epistemology, value, and theory of educational evaluation. Consequently, a diversity of evaluative approaches has evolved based on the differing philosophies of knowing and establishing truth (epistemology). A treatise of such is not intended at this point. For those interested in the more detailed discussions on evaluation philosophy and theory, the extensive work by House is recommended (e.g.,1980, 1983, 2003) and drawn upon for the subsequent cursory discussion.
Based on epistemological philosophy, evaluative approaches may be grouped into two categories: objectivism and subjectivism. Derived primarily from the empirical history of the social sciences, objectivism requires evaluation information to be scientifically objective. Thus, data collection and analytical techniques produce results that are replicable and verifiable by other reasonable and competent individuals using the same techniques and instruments. Subjectivism relies on accumulated experiences of the evaluator rather than a scientific method. The validity of a subjectivist evaluation lies not in replication and verification, but the relevance of the evaluator’s background, qualifications, and keenness of her/his perceptions. Thus, an objectivist evaluative approach depends on replicable facts as the basis of knowledge and truth, whereas, a subjectivist evaluative approach depends on the accumulated and internalized experience of the evaluator(s) as the basis of understanding.
The choice of an objectivist or subjectivist epistemological approach informs different evaluation designs and methods, thus resulting in the current diversity of evaluation approaches. A classification schema that aides interpretation of this diversity and parallels House’s distinction between objectivism and subjectivism can be viewed as a continuum, anchored by utilitarian versus intuitionist-pluralist evaluation, where the distinction is that of principles for assigning values. This schema is depicted in the Figure presented here, with a utilitarian evaluation closely related to an objectivist approach, and an intuitionist-pluralist evaluation to a subjectivist approach (Fitzpatrick, 2004).
A utilitarian approach to evaluation determines value by assessing the overall impact of an education on those affected. In essence, a utilitarian evaluation accepts the value premise that the greatest good is that which will benefit the greatest number of individuals. The evaluation focus is on data measuring total group gains on common indices (e.g., average board scores or pass rates), with the best educational practices determined by those producing the greatest gains on the criterion or criteria selected to determine worth. An intuitionist-pluralist approach to evaluation determines value by the impact of the program on each individual. There are no common indices of “good” and the evaluation serves to portray the different values and needs of individuals. Data typically comes from qualitative interviews and testimonials of program participants. Weighting and balancing the data and criteria inherent in this approach is largely intuitive, and there are no algorithms to help reduce complex evaluative information to any unequivocal recommendation.
The School of Pharmacy and Health Professions uses the Objectives-Oriented Evaluation Approach to programmatic and school-based evaluation activities. Specifically, the Tylerian Evaluation Approach is employed (Tyler 1942, 1950). Originally developed in the1930’s as an alternative approach to the more traditional forms of evaluation, Ralph W. Tyler is credited as the most influential individual in evolving and refining the Objectives-Oriented Evaluation Approach to education. The Tylerian Approach conceives evaluation as the process of determining the extent to which the educational objectives of a school, program, or curriculum are actually attained (i.e., a precursor to the outcomes movement). The specific steps include:
Establish broad goals or objectives
Classify the goals or objectives
Define objectives in behavioral terms
Identify situations in which achievement of objectives can be demonstrated
Select/develop measurement techniques
Collect performance data
Compare performance data with behaviorally stated objectives
Tyler further advocated using general goals to establish purposes, rather than prematurely becoming preoccupied with formulating behavioral objectives, resulting in programs focused on arbitrary and restrictive objectives that do not support their underlying purpose or attainment of credible outcomes. Several leaders in evaluation theory and practice have further refined Tyler’s approach adding flexibility and insight to programmatic evaluation within the Tylerian tradition. Such advances as mixed methods designs and contemporary focus on the measurement of educational outcomes may be tied to Tyler’s approach that has emphasized outcome orientation and alternative instruments/datasources from its inception. One such leader was Hammond (1973), whose interest went beyond determining whether goals or objectives were achieved, to evaluating why some educational innovations/practices failed while others succeeded. This work serves as the foundation for the systematic evaluation framework that will guide such activities within SPAHP.
References:
House, E.R. (1980). Evaluating with validity. Beverly Hills, CA: Sage.
House, E.R. (1983).Assumptions underlying evaluation models. In G.F. Madaus,M. Scriven, & D.L. Stufflebeam (Eds.), Evaluation models. Boston: Kluwer-Nijhoff.
House, E.R. (Ed.). (1983). Philosophy of evaluation. New Directions for Program Evaluation. No.19. San Francisco: Jossey-Bass.
House, E. R. (2003). Evaluation theory. In T. Kellaghan & D. L. Stufflebeam (Eds.), International handbook of educational evaluation (pp. 9–14). Boston: Kluwer Academic.
Krathwohl, D.R., Bloom, B.S., & Bertram, F.M. (1973). Taxonomy of educational objectives, the classification of educational goals. Handbook II: Affective domain. New York: David McKay Co Inc.
Simpson, E.J. (1972). The classification of educational objectives in psychomotor domain. Washington, DC: Gryphon House.
Tyler, R.W. (1942).General statement on evaluation. Journal of Educational Research, 35, 492-501.
Tyler, R.W. (1950). Basic principles of curriculum and instruction. Chicago: University of Chicago Press.