Creighton University, School of Pharmacy and Health Professions

The School of Pharmacy and Health Professions uses the Systematic Evaluative Rubric (SER) Model as the framework to guide the continuous evaluation and assessment across all programs. A modification of Hammond’s EPIC Evaluation Model (Hammond 1973; Fitzpatrick, Sanders, & Worthen 2004), the three-dimensional SER Model addresses the specific disciplinary (Program), contextual (Institution), and evaluative (Assessment) framework inherent to the structure and function of SPAHP (Figure 1). The SER Model is composed of dimensions, variables, and factors that affect the success of any institutional program. The rubric includes the three dimensions of program, institution, and assessment. Each dimension is comprised of multiple variables (e.g., occupational therapy, curriculum, outcomes). The interaction of a variable from each of the three dimensions results in a factor (also called a cell). The importance of any factor, or combinations of factors, is determined by the focus of a specific program evaluation. As illustrated in the SER model, a total of 72 factors/cells may be recruited in designing an overall evaluation of the School. Respectively fewer factors/cells are available for designing evaluations at the academic department, program, or program pathway level. A brief description of each dimension and related variables follows.

Programs Dimension

The Programs Dimension of the model describes the unique disciplinary characteristics of the four academic program variables that currently comprise SPAHP. These variables include the three doctoral programs of Occupational Therapy, Pharmacy, and Physical Therapy. Each doctoral program provides two curricular pathways for obtaining a terminal doctoral degree, a traditional on-campus pathway and a distance pathway. The fourth program variable captures all non-doctoral programs that offer either a Certificate of Completion or Master’s of Science degree. These currently include the certificate programs in Health Services Administration and Emergency Medical Sciences, and the Master of Science degree program in Pharmaceutical Sciences.

It should be emphasized that the borders of each factor/cell while depicted as discrete, are actually considered fluid in the SER model. This is of primary interest within the Programs Dimension because a primary component of the SPAHP Mission Statement explicitly promotes a focused emphasis on interprofessional collaboration. Thus, evaluations can be designed to assess the interdisciplinary nature of specific SPAHP programs related to other variables of interest. For example, how the Pharmacy, OT, and PT programs integrate a School-wide interdisciplinary component on professionalism (or Ignatian values) across each of their curricula may be explored and evaluated (i.e., merging the three Program variables of Pharmacy, OT, and PT with the Institution variable of Curriculum, and the Assessment variable of Process).  Thus, the fluidity of the cell borders allow for hybrid designs to accommodate evaluative questions that crossover multiple cells.

Institutional Dimension

The Institutional Dimension is defined by the variables of Student, Faculty, Administration, Student/Alumni Services, Facilities & Costs, and Curriculum. Any given program is influenced by the unique qualities of the individuals involved, supportive services provided, environmental facilities and fiscal support, and the curriculum guiding the educational development of students. Specific evaluations may further stratify the Institutional variable of interest into sub-variables that have a direct, or indirect, influence on a given program. For example, the Student variable may be stratified into sub-variables of age, gender, grade level, previous academic degrees/experience, ability, achievement, learning style, and socio-economic factors. Or as will be discussed shortly, the Curriculum variable may be stratified into the environmental contexts where specific educational activities occur; didactic, laboratory, and clinical.

Assessment Dimension

• Structure: Assessment of structure assumes that given the proper environment, personnel, and resources, a quality education will ensue. Structural assessments may be stratified to such measures as the adequacies of facilities and equipment (clinical, laboratory, etc.), qualifications of faculty, administrative structure, fiscal organization of school/program, etc. Evaluating “structure” means determining the degree to which the necessary components and relationships are in place, operational, and sufficient to accomplish the educational goals at the program and school level.

• Process: Combined with structural measures, process measures have historically predominated academic assessment. Evaluating “process” means measuring the level of performance of the students, staff, faculty, and administration individually, and of the school, specific program, pathway, and curriculum as a whole. Good process measures require that a great deal of attention is given to specifying the relevant dimensions, values, and standards to be used in the assessment. Additionally, substantial supportive evidence is needed linking the specific process with the expected outcome.

• Outcomes: Measuring outcomes focuses on the changes of, and impacts on, different parties of interest as a result of involvement with a program or school. Outcomes measures typically assess the evidence that the intended learning and subsequent behavior of a student occurred. While the outcomes measures frequently focus on the student/graduate as the data source, other sources of evidence may include patients/clients, employers, the profession, and society. The resulting outcomes can range from immediate impacts (e.g., recent graduates’ educational experience assessed by exit interviews/surveys) to long-term professional attributes and achievement (e.g.,. publications, professional contribution, specialty certification).

SER Model Summary and Example

References

Hammond, R.L. (1973). Evaluation at the local level. In B.R. Worthen & J.R. Sanders, Educational evaluation: Theory and practice. Belmont, CA: Wadsworth.

Donabedian, (1980). Definitions of quality and approaches to its assessment. Foundation of the American College. New York, NY. 

Fitzpatrick, J.L., Sanders, J.R., Worthen, B.R.. (2004). Program evaluation: Alternative approaches and practical guidelines (3^rd Ed.). New York: Allyn & Bacon.