Do we teach to the evaluation?

Student evaluations of courses and professors have been around for decades. Typically, they are given at the end of the semester and serve as a tool for evaluating the quality of instruction and, thus, of the instructor. At most institutions, student evaluations, along with other measures like a teaching portfolio, a statement of teaching philosophy and some kind of peer review, form an important component in a tenure/promotion review.

Even at institutions where there are a variety of “measures” of teaching effectiveness, the student evaluation dominates, perhaps because it is seen as “data” and so gives the impression that it’s an objective measure. Controversy rages about student evaluations. How do you know if a teacher/course is “highly effective”? On a five point scale, with 5 being the highest, what score is good enough? 3.5? 4? 5? At my institution, an “excellent” teacher is one whose courses consistently result in 80% or more of the students giving ratings of 4 or 5 on most/all of the questionnaire’s items.

Given how important these ratings are for a professor’s career, it seems pretty crucial that the items on the questionnaire are clearly understood by the students and that the students have the knowledge required to provide an objective rating.

How do some of the items on questionnaires measure up?

Here’s one example I took from a questionnaire from the University of Illinois, Chicago (https://www.uic.edu/depts/counselctr/testing/Sample%20Questionare015.pdf):

The scale here was (1) Strongly Disagree to (5) Strongly Agree

Instructor was conscientious about being well prepared.

How do students know this? Can they render an informed rating?

How about this one:

Methods of evaluations measured achievement of course objectives.

Or (about a lab course):

I learned skills and techniques applicable to my career.

Are the students really qualified to answer these questions? Does a student know about his/her career after taking an introductory chemistry course as a first year student? Are the student’s ratings objective?

Here are a couple of questions from my institution’s questionnaire. To the right of each entry are spaces to select a number from 5 to 1. Below the “5” is the word “High”, below the “3” is the semi-word “Med” and below the “1” is the word “Low.” Then, a list of items grouped by “course” or “instructor” or “general” or “only if applicable.”

Openness to student’s points of view

 Examinations

What do these mean? I confess that, after 23 years teaching here, I still have no idea what “openness to student’s points of view” means. And, what are we asking with “Examinations”? Do we want to know if there are lots of exams (“High”)? Or are the exams hard (“High”)? I think my students rate this based on how difficult they perceive the exams to be. But I don’t really know. What does the College want to know here? Do you think students find these questions clear and know what kind of information we seek? We faculty don’t even seem to know what’s being asked.

There have been lots and lots of articles written about biases within student ratings of teaching. This blog post is not really about that (I don’t think). What I want to know is whether we “teach to the evaluation” in order to keep our jobs, to get tenure, to get a decent pay raise, or to uphold our reputations with our administration.

A number of recent studies (and some older ones, too) explored possible influences on student ratings. The grade that a student expects to receive in a course has an influence on how he/she rates the course and the professor. Grade inflation has continued virtually unabated since the 1980’s, a time when student evaluations began to rule the roost in promotion and merit pay decisions. Coincidence?

image from: http://www.businessinsider.com/an-a-is-not-what-it-used-to-be-2014-9

 

Perhaps the solution to grade inflation is to develop a different way of measuring teacher effectiveness. (Princeton tried putting a cap on the number of high grades that students can receive….they might be backing away from that, I have read. http://www.huffingtonpost.com/2014/08/08/princeton-grade-deflation_n_5662208.html) If student evaluations are involved in promotion/merit pay, faculty are going to try to please the students, so that they are satisfied and give high ratings. This will happen unwittingly or knowingly. We are only human. Smart ones, at that.

 

Share