Ending gender-based harassment in peer review

By Michael Eisen | Published: May 3, 2015

A few days ago Fiona Ingleby, a postdoctoral fellow at the University of Sussex (she’s an evolutionary biologist who works on sex-specific behavior and other phenotypes in Drosophila) sent out a series of Tweets reporting on a horrifically sexist review she had received after submitting a paper to PLOS ONE. 

There is so much horrible and wrong here, it’s hard to know where to begin. It is completely reprehensible that anyone would think this, let alone write it; that someone would think it was OK to submit a formal review of a paper that said “get a male co-author”; that they would chastise someone for supposed biases without seeing their own glaring ones; that the editor asleep on the job and didn’t look at the review before sending it out or, worse, read the review and thought it wasn’t problematic; that the editor was willing to reject a paper based on an obviously biased review; that the editor didn’t realize that one of their most important roles is to make sure that reviews like this never get sent out or factored into publishing decisions; that PLOS not only allowed this happen but didn’t respond to the authors’ complaint until they took to Twitter several weeks later.

(Let me just disclose for anyone reading this who doesn’t know – I am a founder of PLOS and am on its Board of Directors.  I’ll probably get chastised for commenting publicly on this, but I think it’s important to not just subject PLOS to the same scrutiny and criticism I would bring to the table if it were some other publisher, but to hold PLOS to an even higher standard. This should not have happened, and PLOS needs to not only learn from this, but fix things so that it never happens again. Also, I should add that I have no inside information about this case – I know nothing about it except what has been written about publicly.)

I wish I could say that this review was shocking. But sadly it’s not. As anyone who is paying even the slightest bit of attention should know, science has a serious sexism problem. These kinds of attitudes remain commonplace, and impact women at all stages of their careers in myriad ways. And so it defies credulity to think this is an isolated incident in publishing – if one review like this got through, one has to assume many more like it have been and will be written (indeed they have been) and so we not only have to respond to this event, but we have to do whatever it takes to stop it from ever happening again.

Furthermore, I’ve seen all manner of profanity applied to the review and reviewer – all deserved – for their awful sexist attitudes and acts. But it’s critical that we not dismiss this as just an asshole being an asshole. This happened in a professional setting and clearly targeted the gender of the authors in a way that was not only inappropriate, but which would have had a negative effect on their careers by denying them publication and appropriate credit for their work. So let’s call this what it is – an unambiguous case of harassment.

So what do we do about this? Obviously gender-based harassment happens all over the place. But this particular case happened in the context of science publishing, and PLOS in particular, and I am writing this to ask for help in thinking about what PLOS should do to prevent this from happening (and just to be clear – I don’t run PLOS – but I will do everything I can to make sure all good ideas get implemented).

How do we respond to this reviewer and any future reviewer who engages in harassment in their review?

Once the case became public, PLOS quickly removed the reviewer from its reviewer database, and presumably they will never be asked to review for PLOS again. (I’m still not 100% sure exactly what this means – it seems like we need to do more than remove them from the database – they need to be blacklisted in some manner so that they are never asked to review for PLOS again).

This is obviously a necessary response. But it is also insufficient. First of all, it’s  pretty light punishment – it’s not like people are clamoring to review for PLOS (or any other publisher for that matter). But more importantly, PLOS is but one of many publishers, and accounts for only a few percent of all published papers. This reviewer is still in a position to review for the thousands of other publishers on the planet, so not very much has been accomplished with this action. One can hope the reviewer has learned something from the public discussion of their review, but we certainly can not count on that. So something else needs to be done.

Which bring us to a sticky issue. To do anything more than PLOS has already done would require revealing the reviewers identity either publicly, or at least to the publishers of other journals for which they are likely to review – and reviewers agree to review with the clear expectation that their identity will be kept secret unless they choose to reveal it. While publishers clearly have a duty to protect the anonymity of their reviewers, they also have a responsibility to protect people from harassment. And in this case these two are in conflict. My first instinct is to say, “You do something like this, you lose the right to hide behind the veil of anonymity”, but it’s not as clearcut as I’d like it to be.

It’s no secret to people who read this blog that I have long been against anonymous peer review. But I do recognize that it has a real value, especially to people who are at vulnerable stages in their careers and would not feel comfortable giving their honest opinions if they had to attach their identity to it. In the long run I think we can change the culture of science so they wouldn’t feel that way, but that’s a separate issue. The fact is that right now reviewer anonymity is the norm, and I think it would make a lot of people nervous if publishers granted themselves the right to reveal reviewer identities.

But surely, publishers would reveal reviewer identities in some situations – say if a reviewer physically threatened an author or engaged in some other frankly illegal activity in their review. So clearly anonymity is not inviolable, and the question is whether sexist and harassing reviews raise to the level where the publisher’s interest in protecting others from abuse trumps its interest in preserving reviewer anonymity. I think it does, and furthermore feel it’s a cop out on the part of publishers to hide behind review anonymity here. Engaging in harassing behavior in peer review should void your guarantee of anonymity, full stop.

Obviously, one superficial way to resolve this conflict is to intercept all harassing reviews and make sure they never are seen by the authors – a sort of “no harm, no foul” response. But while this protects the authors from the proximal harms of a biased and sexist review, it doesn’t deal with the harasser. The responsibility of the journal to prevent others from being harassed shouldn’t change because their behavior was caught early.

There are serious challenges in implementing something like this – for example, who would make the decision that something is harassment? – but I am confident we can figure them out. One thing that all publishers can do is to spell out very clearly the kinds of behavior that are unacceptable and what the consequences are for engaging in them. It seems like you shouldn’t have to say “don’t harass people”, but clearly you do. And having very clear policies would likely both help prevent harassment and make it easier to deal with harassers. When this case first came to my attention, I looked around to see if PLOS has some kind of “code of conduct” policy for reviewers, but I couldn’t find one. Maybe I missed it, but if so, then it’s likely not being seen by reviewers. I thought I might find them at the Committee on Publication Ethics, but their code of conduct policy doesn’t seem to deal with this either. Does anyone know of such a policy? I was at a meeting last month sponsored by India Bioscience – the program guide has a great “Code of Conduct” for meeting attendees – this would be a good place to start. [UPDATE: A comment from Irene Hames pointed me to this “Ethical Guidelines for Peer Reviewers” from COPE].

I’m very curious what other people think about this, especially because I’m a bit concerned that my overall feeling that anonymous peer review is bad is coloring my judgment here. But seriously, what could be more important for a publisher to do than protect their authors from harassment? If they’re not willing to do whatever that takes, they should just close up shop.

The role of editors in preventing harassment

It’s hard to fathom how a review as blatantly sexist and harassing as this one was not only sent back to the authors, but used as the sole basis for a negative publication decision on the submission. There are really only two possibilities – neither of them good: the academic editor handling the manuscript failed to fully read the review, or they read it and didn’t find its contents objectionable. So either the editor doesn’t take their job seriously or they are complicit to harassment. Whatever the answer, they shouldn’t be handling manuscripts, and PLOS has asked them to resign their position (and, presumably, will not send them any more manuscripts even if they don’t formally resign).

This editor (again, I don’t know their identity, or anything about their past performance for PLOS) was one of approximately 7,000 academic editors who handle manuscripts for PLOS ONE. The vast majority of the people who edit and review for PLOS take their work seriously and are constructive in their reviews. However, with that many editors it’s inevitable that some are going to do their job poorly. But we can’t just write this off as a bad editor. PLOS has intentionally (and for good reasons) devolved a lot of autonomy to its editors. But in doing so it has magnified the effect that a bad or negligent editor can have, and this increases the need for PLOS to train its editors well, to oversee their work carefully, and to respond rapidly when problems arise – all of which PLOS failed on here.

One issue has to do with the way that editors conceive of their job. It’s always seemed to me that many academic editors think that their primary responsibility is to identify reviewers and then to render decisions on papers after reviews are in. They recognize that they sometimes have to adjudicate between reviewers with different opinions – making them a kind of super reviewer. But I seldomly hear academic editors talk about another – arguably more important – aspect of their job, which is to protect authors from lazy, capricious or hostile reviewers. In my experience most editors almost always pass on reviews to authors even if they disagree with them or think they were inadequate – it’s somehow felt to be bad form to have asked for a review to then turn around and not use it. This needs to change. I would argue that protecting authors from reviewer malfeasance or malignancy is the most important role for editors in our current publishing system. Maybe PLOS and other journals already do this, but every academic editor should be trained to recognize and deal with the various types of harassment and other bad reviewer behaviors that we know exist.

But training can only go so far, and we have to assume that there is going to be considerable variance in the manner in which editors work and that some fraction of papers will be handled poorly, especially for a journal like PLOS ONE where a large number of the editors are young and relatively inexperienced. PLOS knows this, of course, and has long wrestled both with how to get more consistent behavior out of its editors and to deal with problems when they arise. There are two general possibilities: there could be a second layers of more experienced editors or staffers who review every decision letter for its adherence to PLOS’s editorial standards and code of conduct before it goes out, or PLOS could assume that most decisions are good and rely on feedback from authors (aka complaints) to identify problems.

You can understand why PLOS ONE and most other journals that already rely heavily on academic editors generally choose the later solution – it’s hard enough to find people to handle manuscripts – adding a second layer of review would slow things down even further and make them more expensive. But if you’re going to use this strategy, then it seems like it’s imperative that you respond to issues – especially serious ones – quickly. And PLOS failed to do this – the authors say they had been waiting for almost a month for PLOS to respond to their complaint about how their manuscript was handled.

PLOS really has to fix this. But I also think they should consider what it would take to have every decision letter screened before sending it out to authors. This would not only go a long way towards preventing harassment in the review process, but also ensuring that the whole process is more fair (I’ve fielded a fair number of complaints about the failure of editors to properly implement PLOS ONE’s editorial policies – one decision letter I saw described a paper as “technically sound, but not of sufficient interest to merit publication in PLOS ONE” – a clear contradiction of PLOS ONE‘s standards for inclusion).

How much would this cost? Seems like you could hire someone who looks at 2-3 decision letters an hour, so lets say 20 a day, or 5,000 a year. Even if you pay this person a very good salary, you’re only talking $20-$25/article to make sure people aren’t being harassed and are otherwise being treated fairly. Considering that we spend around $6,500/published article on average across the industry, this seems like a pittance.

Protecting authors in an open review/post-publication review world

I’ve written a lot about why I think the whole system of pre-publication peer review that dominated science publishing needs to be replaced with a system where papers are published whenever authors feel they are ready, and peer review happens post-publication and is not limited to 2 or 3 handpicked reviewers. I’m not going to rehash why I think this system is better – you can read my arguments here and here. PLOS will begin the first stages of this transition soon. More open peer review will discourage some of the bad behavior that takes place when reviewers are anonymous. Taking away the power individual reviewers currently have to influence the fate of a paper and thus the careers of its authors should make review more fair. However, protections the formal structure of peer review affords authors from bad reviewer behavior could easily be undermined if we try to rely too heavily on the wisdom of the crowd to police peer review.The sexist attitudes that reared their ugly head in this case are not going to go away because we change the way peer review works. So it’s very important that, in trying to fix other aspects of science publishing, we don’t end up increasing authors exposure to abuse. In this world I think the things discussed above – very clear codes of conduct for reviewers, and proactive policing of reviews – become even more important. And while I’ve been convinced that it’s important to allow reviewers to be unnamed to authors and readers, it’s imperative that they not be truly anonymous – somebody (publisher, scientific society, etc…) has to know who reviewers are so that harassment and other abusive behaviors can be discouraged and dealt with appropriately when they occur.

Please let me know what you think about these issues. I’m sure others have better ideas than I do about how to prevent and deal with harassment in science publishing today and in the future.

 

UPDATE: Several people on Twitter have noted that the term “sexual harassment” is specific to cases involving unwanted sexual advances. The terms “sexist” and “gender bias” were suggested by some, but I don’t think that captures the egregiousness of the offense, so I changed the title and text to “gender-based harassment”, which I think is more appropriate.

This entry was posted in science and tagged , , , . Bookmark the permalink. Both comments and trackbacks are currently closed.

44 Comments