, 3 min read
Gender and peer review
Modern science works in the following manner. You do the research. You write a paper. You publish the paper. For historical reasons, “publishing the paper” typically means “submit it to a committee of your peers and get their seal of approval”.
We can rightfully be concerned about the unwanted biases that this committee of peers might have. For example, maybe these peers have unconscious biases against women?
It is not unreasonable. We have evidence that peer review is strongly biased in favour of authors from prestigious institutions. For example, Peter and Ceci took already accepted articles from authors at prestigious institutions, and they resubmitted them as authors from lesser institutions: they encountered an 89% rejection rate.
reviewers were more likely to recommend acceptance when the prestigious authors’ names and institutions were visible (single-blind review) than when they were redacted (double-blind review) (87% vs 68%) and also gave higher ratings for the methods and other categories.
That, by itself, is not as worrying as it seems. It is a fact that Stanford researchers are better, on average, than researchers from that school you never heard from. Everything else being equal, it makes sense to put more trust in work from Stanford.
But gender is something else. We have no rational reason to trust the work done by men more than the work done by women.
So is peer review sexist? Squazzoni et al. examined 145 journals in various fields of research, including about 1.7 million authors and 740,000 referees. They find that:
manuscripts written by women as solo authors or coauthored by women were treated even more favorably by referees and editors.
Webb et al. in Does double-blind review benefit female authors? write:
We found a significant interaction between gender and time (P < 0.0001), reflecting the higher female authorship post-2001 than pre-2001, but there was no significant interaction between gender and review
Ceci and Williams review the evidence regarding biases against women…
The preponderance of evidence, including the best and largest studies, indicates no discrimination in reviewing women’s manuscripts
(…) recent meta-analysis suggests that claims of gender bias in peer review “are no longer valid”. For example, if there is gender bias in review, we would expect double-blind conditions to increase acceptance rates for female authors. However, this is not the case. Nor are manuscripts by female authors disproportionately rejected at single-blind review journals. Even when the quality of submissions is controlled for, manuscripts authored by women do not appear to be rejected at a higher rate than those authored by men. Meta-analyses and large-scale studies of grant outcomes found no gender differences after adjusting for factors such as discipline, country, institution, experience, and past research output.
the influence of author gender on bidding or reviewing behavior is not statistically significant.
The result extends to grant reviews:
We found no evidence that White male principal investigators received evaluations that were any better than those of principal investigators from the other social categories, and this conclusion was robust to a wide array of model specifications. Supplemental analyses suggest that any bias that is present is likely below the threshold of pragmatic importance.
Li and Agha found in Big names or big ideas that grant reviews are fair:
We find that better peer-review scores are consistently associated with better research outcomes and that this relationship persists even when we include detailed controls for an investigator’s publication history, grant history, institutional affiliations, career stage, and degree types
It even seems that women are slightly better off:
Our results indicate that female principal investigators (PIs) receive a bonus of 10% on scores, in relation to their male colleagues.
Ersoy and Pate find that:
Our results suggest that male economists at top institutions benefit the most from non-blind evaluations, followed by female economists (regardless of their institution).
In other words, they find a bias against males at non-elite institutions.
So it seems that we have good news. Peer review is not, statistically speaking, sexist.