, 1 min read
Most scientific papers are probably wrong
According to this New Scientist article, most scientific papers are probably wrong:
John Ioannidis, an epidemiologist at the University of Ioannina School of Medicine in Greece, says that small sample sizes, poor study design, researcher bias, and selective reporting and other problems combine to make most research findings false.
In my opinion, there is no question that the weak review process most conferences enjoy and the pressure to publish lots of new results, as opposed to verifying what others have done, contribute to this. In my experience, very few people will ever try to reproduce your results, very few people will check your proofs, very few people will reimplement your algorithms.
A few years ago, I tried to publish an errata to one of my paper. It took some doing since the editors had never received such a request. Quite clearly, our tolerance for unaccurate or wrong results is quite high.
Is it a problem? If you are an engineer trying to implement a new system, and you are naïve enough to trust a few scientific papers, then you can get in big trouble. It usually isn’t much of a problem because the average engineer doesn’t read cutting edge scientific papers. It seems obvious that the problem can get more serious if medical doctors read cutting edge articles… do they?