Is Blinded Review Enough? How Gendered Outcomes Arise Even Under Anonymous Evaluation

In evaluating research proposals, the focus on writing style resulted in a gender bias.

• Despite implementing a blind review process, female applicants were approximately 15% less likely to receive a silver rating and 20% less likely to receive a gold rating from reviewers compared to male applicants.
• Reviewers mainly based their decisions on the writing style of proposals, favoring the use of broad words (for example, “control” or “detection”).
• Female applicants tended to use narrower, more topic-specific phrasing - word choice associated with low reviewer scores (for example, “contraceptive” or “brain”). After controlling for the impact of word choice, the gender-based score disparity effect size dropped by over 50%.
• However, applicants with broad phrasing in their proposals tended to underperform after receiving a Gates Foundation grant, with fewer publications and National Institute of Health grants.
• Gates Foundation grants had a larger impact on women’s careers in innovation compared to men.
  • Without funding, women performed worse than their male counterparts in terms of articles, new journals, coauthors, and MeSH terms derived from applicant publications.
  • With funding, women performed on par with their male counterparts. In terms of National Institute of Health grant funding, female Gates Foundation grantees even outstripped their male counterparts.

In short, even when proposed innovations were actually high quality, women received lower scores because of differences communication and writing styles, as they tended to use excessively narrow words in their proposals. Compared to male applicants, the failure to achieve funding posed more of a barrier to future innovation by female applicants. Based on these findings, innovation-driven organizations have room to improve their application procedures in selection of high-quality projects and allocation of resources to underrepresented innovators.

Using a sample of 6,794 proposals submitted by 5,058 applicants (34% female) to the Gates Foundation’s Global Challenges: Exploration program between 2008 and 2017, the authors analyzed two major components of the interplay between diversity and innovation. First, the authors examined the determinants of diversity in innovative organizations by examining the role of gender in explaining reviewer evaluations of innovative proposals. Second, the authors constructed a difference-in-difference estimator to explore the interaction between funding and applicant gender, in order to identify the differential impact of funding across applicants. In these analyses, the authors focused on a homogenous sample of US-based life science researchers, and identified the causal impact of gender on both reviewer scores and subsequent outcomes.