Reducing Discrimination in the Field: Evidence from an Awareness Raising Intervention Targeting Gender Biases in Student Evaluations of Teaching

Encouraging students not to incur in gender discrimination is likely to be ineffective; a more effective approach is to present them data of past gender discrimination made by students. 

Introduction

Feedback and evaluation tools in organizations are prone to unconscious bias, given that feedback is largely subjective. Evaluation tools are particularly important inside academic institutions and universities, as they often rely on student evaluations to make decisions about the promotion and retention of instructors. For this reason, it is important to ensure that student evaluations are debiased to the furthest extent possible. 

Anti-bias awareness-raising campaigns are a tool that has been used with the goal of fighting negative bias at organizations, including at academic institutions. Generally, their message is “normative”, meaning messages simply state that discrimination is wrong and should be avoided. 

However, evidence suggests that awareness-raising campaigns might not be an optimal solution when they rely solely on normative messaging. The biases and stereotypes that drive discrimination are frequently unconscious, and thus individuals receiving anti-bias messages might think the campaign does not apply to them. Therefore, it is necessary to find successful interventions to make unconscious biases “conscious” and thus reduce discrimination against candidates. 

The authors of this paper analyzed gender discrimination in student evaluations of teaching (SETs) in a university in France. Specifically, they compare two different types of awareness-raising campaigns, one with a normative treatment and another with an informational treatment. The normative treatment encouraged students to avoid gender discrimination when completing the SETs, while the informational treatment included information about a past study and explicitly described discrimination in SETs. 

Findings

The normative treatment had no significant impact on students’ evaluations of teaching (SET), while the informational treatment significantly reduced gender discrimination. 

  • After informational treatments, female teacher’s SET scores increased (from a mean rating of 3 to 3.15 out of 4 points), independent of whether the student had received the informational email or not, suggesting students who did receive the email discussed its contents with students who did not receive the email. Thus, all students on the informational treatment campus were treated through the email or through discussions with their peers.
    • Male teacher’s scores did not change significantly following informational treatment.
  • After informational treatment, male students gave higher overall satisfaction scores to female teachers (from a mean rating of 2.89 to 3.2), while female students were not affected. However, the difference between the two effects is not statistically significant because of small sample sizes. 
  • Normative treatment did not have a statistically significant impact on either male or female teacher’s SET scores. 
Methodology

The field experiment was conducted in a French university, on a cohort of 1570 students across the university’s seven campuses. The experiment consisted in sending two different emails to students based on their campus location. The normative treatment encouraged students to avoid discrimination, especially gender discrimination, by showing the following message:

“Considering the importance of these evaluations, we would like to remind you that your evaluations must exclusively focus on the quality of the teaching and must not be influenced by criteria such as the instructor’s gender, age or ethnicity. We ask you to pay close attention to these discrimination issues when completing your student evaluations. The goal is to avoid a situation in which, for instance, gender-based biases or stereotypes would systematically generate lower evaluations for women instructors compared to their male colleagues.”

For the informational treatment, precise information was added to the normative statement. It explicitly stated that students had applied gender bias in the past, in the exact same context. The email cited the paper by Boring (2015), a study on SETs carried out in the same university that showed evidence of gender biases against female teachers in previous academic years (2008–2013), with male students being particularly biased in favor of male teachers. The email ended with the same statement as in the normative treatment. 

Authors created a difference-in-difference setting using the university’s seven separate campuses. The normative treatment included students from three campuses: Menton, Poitiers and Reims. The informative treatment included the campuses of Le Havre and Paris, while the campuses of Dijon and Nancy were the control group. 

There were a total of 1549 students, but for each treatment, the email was sent to only half of the students. The final dataset included a total of 1509 evaluations for normative treatment, 2331 evaluations for informative treatment and 656 evaluations for the control group.

Related GAP Studies