Many things in the world of research are great. Many other things aren’t.
But, the question is: how do we improve them?
What would you do if you could change one thing about the most frustrating parts of researching?
Peer review is a cornerstone of the scientific process, yet we do not seem to take it seriously. This puts immense pressure on the system, especially at the moment when many of us have to cope with a higher workload. For instance, as an editor, I currently have to contact around 10-20 potential reviewers to get the minimum of two reviewers for each submission. As an author, I feel frustrated by the long time it takes to get my submission peer-reviewed and by the sometimes shoddy quality of the reviews.
There is no easy way to fix this, but I think one change could make enough of a difference. Namely, to pay people. At the moment, we all take on various roles for the benefit of the scientific community that we have to do in our free time and in addition to our other commitments. To improve peer review, we should pay the reviewers and pay the editors. The associated cost should be covered by the authors, which should also reduce the number of submissions. This professionalises peer review and rewards reviewers and editors the time and expertise that they provide.
4 Likes
I agree with @Joe_Bathelt that the current system isn’t ideal. As an early-career researcher who’s done <10 reviews, I still get excited whenever I get an invitation and make sure to find time to do a very thorough job at it –– but if I had a much larger volume of invitations, and a much busier schedule, I can easily imagine how it’d be hard to maintain a high standard of reviews.
Joe’s suggestion of paying reviewers and editors sounds like it’d make a huge difference, although I wonder about how to get it to work economically, especially given the need to reduce/remove publishing fees…
One thing I’d like to see more of is for journals to anonymously publish the reviews alongside articles, like the journal eLife does (example). I think this would have several benefits, including (1) allowing readers to take reviews into account when reading an article, including criticisms that the authors may not have fully addressed or unreasonable requests from the reviewers that the authors may have been forced to incorporate; (2) further incentivising researchers to address legitimate reviewer concerns, knowing that these will become public; and (3) encouraging reviewers to be more polite and thorough (I think this would happen even with reviews being published anonymously).
I’d be curious to hear if anyone can think of downsides to this suggestion!
2 Likes
By the way, some of you might find this article by Heesen and Bright interesting: Is Peer Review a Good Idea?
Their abstract is pretty unequivocal: “Pre-publication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behavior of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing pre-publication peer review can be evaluated with a reasonable level of confidence based on presently available evidence, they are either positive or neutral. We conclude that on present evidence abolishing peer review weakly dominates the status quo.”
1 Like
These are greats points @Joe_Bathelt @Joshua_TM!
One thing I’d like to see more of is for journals to anonymously publish the reviews alongside articles, like the journal eLife does (example)
100% agreed here. We should be able to scrutinize the scrutiny!
Namely, to pay people .
What would you both say to people who argue that this commercialises something that should be done for the ‘good of science’?
1 Like
I like the open peer-review concept but it also has some disadvantages. For instance, a more junior reviewer may feel less comfortable to provide their honest opinion when the paper under review is by an influential author in the field. There also other potential biases that may creep in when review is not blinded.
1 Like
I think we could do without the traditional publishers and their gigantic profit margins. Instead, we could have online journals run by editor and reviewers who get paid for the contribution. This could be combined with open peer-review to make it transparent. In the end, this could be less costly than the current model.
I agree that the idealism behind the original model of providing free peer review is laudable. However, I think the scientific community will have to think about alternatives as the system comes under increasing strain.
1 Like
Agree that fully open review might have some unintended consequences, and perhaps particularly for junior scholars. The way eLife does it seems to avoid this since the reviewers are anonymous. Of course, this means that the incentive to leave a thorough and polite review doesn’t really change since you wouldn’t get credit for the anonymous review, although, if people are anything like me, there might still be some folks who’d take extra pride in having a well-written review out there, even if anonymously.
1 Like
Another potential intervention would be post-publication review, which I know that some journals like F1000 Research has been doing.
1 Like
This was a great discussion!
We’re having another one this week. Got any strong opinions on Journals?
1 Like
