, 1 min read
The art of paper review
I do not claim to be an expert at reviewing academic papers, but I have done my share of work. Here is my recipe:
- Reproducibility, (self-)plagiarism and presentation are easy to evaluate and I usually spend quite a bit of time on these issues. Science should be reproducible. (Panos Ipeirotis seems to agree with me.) Plagiarism can be surprisingly hard to detect, but it is also amazingly frequent, so I usually search for a few word cooccurrences in Google. Presentation is, on average, quite poor. Figures are often ugly. Poor English is frequent.- The relevance and strength of the paper is something I usually have an opinion about. Alas, it is easy to be wrong about the importance of a paper, so I usually do not have much to say unless I have directly worked on the same problems for a couple of years.- Correctness is hard to check especially if I am not a domain expert. I usually pick up on secondary details. Are the results credible? Do the authors mention some special cases that should have arisen in their analysis or experiments? I must unfortunately admit that I usually cannot be sure that the papers I have reviewed are correct. At best, I can voice an opinion about their credibility.