Daniel, I am not so optimistic. I think you can cheat, cheat a lot, and never get caught.
First of all, there are ways to make it improbable to get caught:
1) Never release your data, if you ever bothered to collect some.
2) Never release your software.
3) Never flush out fully the proof of a theorem (after all, if people believe your sketches…).
4) Make it very difficult to reimplement your work.
5) Cheating by omission is not really cheating, is it? How can you prove that the authors had other data than the one they presented?
Then, even when you do get caught, there are lots of ways around it:
1) Peer-review will ensure that it is difficult to publish a paper whose main contribution is to prove that another paper was wrong. We rarely see these papers for good reasons: people hate negative results.
2) Accuse whoever can’t reproduce your results of having made a mistake or having misunderstood your work.
3) Say that you have “moved on” and that this problem is no longer interesting.
4) Say that it was the work of your collaborators and you had very little to do with it.
In short, cheaters can get by just fine. I am quite certain.
As long as we take the peer review system for anything else than an honor system, cheaters have a huge edge over honest folks, and this ensures that cheating gets built into the system.
But the minute you realize that reviewers can’t possibly be able to detect well-crafted fraud, then you start rethinking your metric.
I am not saying we should not pay tribute to those who publish lots of papers in high places. Just… be careful in how you interpret it. Maybe they careful crafted their papers for this very purpose, you know?
I would think the other aspect of peer review is that peers read the papers you publish. If you succeed in selling your work as consequential (i.e., claiming to solve an important problem and publishing in a prestigious venue), then a lot of your peers will read your work and try to build on it. That makes it pretty likely that you’ll be exposed.
Sure, that won’t happen with less ambitious publications. But it’s a lot like theft. You can probably steal a little and not get caught most of the time. But I suspect that the expected value of violating the honor system is negative, at least if you’re aiming for any kind of status in the research community–which is presumably the primary motive for cheating. After all, the pay isn’t that good!
Daniel, I am not so optimistic. I think you can cheat, cheat a lot, and never get caught.
First of all, there are ways to make it improbable to get caught:
1) Never release your data, if you ever bothered to collect some.
2) Never release your software.
3) Never flush out fully the proof of a theorem (after all, if people believe your sketches…).
4) Make it very difficult to reimplement your work.
5) Cheating by omission is not really cheating, is it? How can you prove that the authors had other data than the one they presented?
Then, even when you do get caught, there are lots of ways around it:
1) Peer-review will ensure that it is difficult to publish a paper whose main contribution is to prove that another paper was wrong. We rarely see these papers for good reasons: people hate negative results.
2) Accuse whoever can’t reproduce your results of having made a mistake or having misunderstood your work.
3) Say that you have “moved on” and that this problem is no longer interesting.
4) Say that it was the work of your collaborators and you had very little to do with it.
In short, cheaters can get by just fine. I am quite certain.
As long as we take the peer review system for anything else than an honor system, cheaters have a huge edge over honest folks, and this ensures that cheating gets built into the system.
But the minute you realize that reviewers can’t possibly be able to detect well-crafted fraud, then you start rethinking your metric.
I am not saying we should not pay tribute to those who publish lots of papers in high places. Just… be careful in how you interpret it. Maybe they careful crafted their papers for this very purpose, you know?
I would think the other aspect of peer review is that peers read the papers you publish. If you succeed in selling your work as consequential (i.e., claiming to solve an important problem and publishing in a prestigious venue), then a lot of your peers will read your work and try to build on it. That makes it pretty likely that you’ll be exposed.
Sure, that won’t happen with less ambitious publications. But it’s a lot like theft. You can probably steal a little and not get caught most of the time. But I suspect that the expected value of violating the honor system is negative, at least if you’re aiming for any kind of status in the research community–which is presumably the primary motive for cheating. After all, the pay isn’t that good!