, 3 min read
Peer review without journals or conferences
Almost all scientists ask their peers to review their work outside of the formal process offered by journals and conferences. Young scientists are routinely told to take their manuscripts to a senior scientist and ask for a review.
With some success, I have even used my blog as a peer review device: e.g., we received many comments on our manuscript on fast integer decoding through a blog post I wrote. If you read some of my recent research articles, you will find that I acknowledge people who commented on this blog. (I have even ended up writing papers with some of them, but that is a different story.) In effect, my blog has helped me get extra reviews for some of my work! I couldn’t be more grateful!
I am not really interested in making journals and conferences go away, but I am interested in going beyond them. I fear that they are often limiting us:
- In fields such as computer science, limiting the review to journals and conferences effectively cuts most non-academics out. They are also limiting the review to a narrow band of experts. If you are trying to solve problems that matter, this might be entirely wrong.
- Traditional peer review is anonymous. In principle, this makes it fair and transparent. In practice, it can be needlessly alienating. For example, I would much rather have an open exchange with some authors that I criticize rather than just send an anonymous note. We might both benefit from the exchange. One pattern I have noticed is that some of my (well meaning) comments get ignored. Even when they require little work and can only benefit the authors. We have put walls in the peer review process, and there are good reasons for these walls, but we could do without them if we reinvented the process.
Hence, I have launched an open invitation to the world: send me your drafts, and if I find them interesting, I will review them and then tell the world about them (through social networks) if you revise them to my satisfaction.
My goal is not to replace journals and conferences all by myself, but I do see a growing trend whereas people point to papers that have not yet been peer reviewed and say: I read this, it looks good to me! I’d like more people to participate in this new emerging model.
Anyhow, so far, only one courageous fellow agreed with my terms: Nathanaël Schaeffer sent me his paper Efficient Spherical Harmonic Transforms aimed at pseudo-spectral numerical simulations. I wrote a review, the same way I would for a journal, and I sent it to him. He produced a revised version that took into account my criticism. I am now telling you that if you care for spherical harmonic transforms, you should definitively check out his paper. (Update: Schaeffer’s paper has now appeared in a good journal.)
So where does that leave us?
- If you are a researcher, and you would be willing to review manuscripts openly the way I did, please let the world know!- If you think your work could interest me and you want to try a different type of peer review, please send me your paper!
- If one of my papers interests you and you want to write a review and share it with me, please do! I also have software that needs reviewing.
I stress that you do not need to be affiliated with a college or have researcher as your title for this model to work. Anyone can write or review a research paper. (Admittedly, few people can write good papers or produce deep reviews but that is another story.)
I am not sure exactly how far we can go with such open peer review processes. But I think we can improve the current system significantly. To make my point stronger, I plan to write a blog post describing how I benefited from reviews and criticisms I have received through my blog and social networks. For now, please believe me: I received insights that I would never had received through the traditional peer review process.