(1) PLoS One is a journal which works based on the publish-then-filter model. Papers are checked for connectedness, but not significance, and then published. Statistics regarding each paper is collected (e.g., download statistics) and it is on this basis that the paper is assessed for importance. Nobel prize winners publish in PLoS One.
(2) Physics has already switched to an electronic model through arXiv.org where almost all physicists upload their papers. Of course, physicists still published in convention peer reviewed journals, but their work is typically available on arXiv first.
(3) Gregori Perelman solved the Poincaré conjecture in a series of papers published online without any traditional peer review. Even though no journal reviewed his work, it is widely accepted that his work is correct, so much so that he was offered one million dollars (which he turned down). He published first, and then his work was filtered and determined to be very important.
Adam Hylandsays:
I agree this would be preferable along a few lines to the current system of submit and hope you aren’t rejected for opaque reasons or relegated to a too-parochial journal.
But I keep wondering whether or not talk is cheap as well as whether or not filtration serves a function in of itself. We can imagine that the flood of academic research winnowed through an archaic mechanism results in serious deadweight loss. But at the same time I think the prestige of some of these journals stems directly from their selectivity. Sometimes this selectivity backfires (see the 1930s decision of Keynes at the helm of The Economic Journal spurring the rise of Econometrica), but refusing to publish anything but a subset of submitted work generates value through scarcity. The real challenge is replicating that selectivity while not imposing what is essentially randomization between sufficiently “good” papers to determine issue content.
Adam Hylandsays:
@otoburb
I agree. And frankly that’s exactly what is happening in a lot of fields. Works which are ex post “important” or “interesting” almost always receive citations well in excess of the average for their journal regardless of the journal.
But excluding exceptional works this process takes a great deal of time and most of the concerns from young faculty members relate to establishing tenure and relevance quickly–the mechanism to do so depends (unfortunately) on that scarcity and the presumption that without prior information a paper in a good journal is better than a paper in a bad journal due to the selection process.
I guess I’m not making normative claims (the perennial excuse of the economist!); just trying to tease out the politics.
Well, first of all, your letter basically says: we spent a lot of time, a lot of money, we say is good and we modify the structure so it fits you Journal.
Where is the scientific validity of your paper? If you would to ask someone to definitely publish something, I would for sure say: The scientific validity is unquestionable, or something of the sorts.
I do think is a bit rude, seeing that is you who wants to publish in their Journal and not the other way around.
But despair not, most well known authors were rejected by a ton of publishers. But you know they did not want to change the establishment so they could get their way, they kept searching, and today we know them as George Lucas, Rowling, etc.
For each guy who presented their own breaking results in a non-peer reviewed way, I can tell you 10 who presented in peer reviewed journals.
Stephen Hawking, Einstein, Bohr, Darwing, Crick,…. all of them presented in peer reviewed conferences and Journals and their Nobel Prizes come from works they presented over there.
You should follow the books you recommend more closely, in “Write a lot” Silvia clearly says: “If your paper is not accepted in the Journal, do not get angry, do not send hate letters, just look for another Journal to publish”
You misunderstood my post. The letter is a fictitious example meant to ridicule people who lack the ncessary intellectual maturity to publish their work.
I want to exclude from science peope who would write such a letter.
otoburbsays:
That submission sounds like there was a history of refusals from either the same journal, or a variety of similar publications.
Granted, the tone seems a bit brusque, but at least they get to their most important (what seems to be a rather sore) point.
Itmansays:
OMG, this is hilarious! Some authors (actually the paper was good and kinda ahead of time) wrote that the refereed proposed to analyze their method analytically. They (authors) instead thank the referees for the suggestion, but postpone implementation for the future.
Man, they almost literally wrote that and it was published in an ACM journal.
“Publish then filter” currently exists as blogs and wikis. I can’t see print journals going this route for the simple reason that it would increase the number of pages by at least an order of magnitude (most journals reject 80-90% of submissions, and a publish-first model will discourage authors from heeding the adage that brevity is the soul of wit. If journals go to electronic format, space is a much less important issue – but the journal still risks losing its identity (and readership) if readers have to wade through a lot of muck to find a good paper.
otoburbsays:
@AdamHyland
A social filtration system would hopefully have the capability of stamping a “prestige” badge/reputation on papers. The value through scarcity would still exist, except the determination of “scarcity” wouldn’t be a function of whether a paper was published in a selective journal, but by how many highly selective social committees (presumably roving panels of experts or editors) endorsed a paper.
One could think of a social “like” mechanism, except applied to papers with a different rating system.
Rafael Perezsays:
Big fan of your blog here.
I have been struggling (well, not really struggling) with a PhD for the last two years, and every time I read one of your posts I feel I will just not finish.
You see, as much as I love research, finding new knowledge, new ways and breaking the established paradigms, I hate writing. Sadly one of the requirements in my university to graduate is to publish a couple of journal papers, and I just can not get to write down the mess in my mind in an organised way.
As a side project, I started an small company where I can apply my research without nobody asking me about previous or related research (specially when there is no previous research) or formats or references, and well, it is flourishing (and getting some money as a side effect).
I love the pursuit of knowledge, to ask questions, find patterns, relations, reasons, causes and reactions (data/knowledge mining dude here). Yet, too many restrictions…
Anyway, this last post makes me fully realise I will never make it in academia. I have no option but to go my own way, and survive from there.
Thank you Daniel, you opened my eyes.
As potential arguments in the submission letter, you forgot to mention: “I am Mr. Bigshot and I’ll have my paper published no matter what; you decide whether it’s in your journal or elsewhere.”
However, if I may add my gain of salt, a good scientific contribution must eventually convince people of its worthiness. It may not pass a journal board on its first try, but I guess it eventually must if it is really that good. The opposite would be that a small number of editorial boards in *different* venues consistently block a valid idea to the point it gets published nowhere, ever. That looks a bit improbable to me. Hence I don’t see what the publish-filter model adds *in the long run* over the existing process. It may delay the publishing of an interesting contribution in the short term, but cannot prevent something objectively interesting/true from emerging over time.
@Rubin
(1) PLoS One is a journal which works based on the publish-then-filter model. Papers are checked for connectedness, but not significance, and then published. Statistics regarding each paper is collected (e.g., download statistics) and it is on this basis that the paper is assessed for importance. Nobel prize winners publish in PLoS One.
(2) Physics has already switched to an electronic model through arXiv.org where almost all physicists upload their papers. Of course, physicists still published in convention peer reviewed journals, but their work is typically available on arXiv first.
(3) Gregori Perelman solved the Poincaré conjecture in a series of papers published online without any traditional peer review. Even though no journal reviewed his work, it is widely accepted that his work is correct, so much so that he was offered one million dollars (which he turned down). He published first, and then his work was filtered and determined to be very important.
I agree this would be preferable along a few lines to the current system of submit and hope you aren’t rejected for opaque reasons or relegated to a too-parochial journal.
But I keep wondering whether or not talk is cheap as well as whether or not filtration serves a function in of itself. We can imagine that the flood of academic research winnowed through an archaic mechanism results in serious deadweight loss. But at the same time I think the prestige of some of these journals stems directly from their selectivity. Sometimes this selectivity backfires (see the 1930s decision of Keynes at the helm of The Economic Journal spurring the rise of Econometrica), but refusing to publish anything but a subset of submitted work generates value through scarcity. The real challenge is replicating that selectivity while not imposing what is essentially randomization between sufficiently “good” papers to determine issue content.
@otoburb
I agree. And frankly that’s exactly what is happening in a lot of fields. Works which are ex post “important” or “interesting” almost always receive citations well in excess of the average for their journal regardless of the journal.
But excluding exceptional works this process takes a great deal of time and most of the concerns from young faculty members relate to establishing tenure and relevance quickly–the mechanism to do so depends (unfortunately) on that scarcity and the presumption that without prior information a paper in a good journal is better than a paper in a bad journal due to the selection process.
I guess I’m not making normative claims (the perennial excuse of the economist!); just trying to tease out the politics.
Hey Daniel,
Well, first of all, your letter basically says: we spent a lot of time, a lot of money, we say is good and we modify the structure so it fits you Journal.
Where is the scientific validity of your paper? If you would to ask someone to definitely publish something, I would for sure say: The scientific validity is unquestionable, or something of the sorts.
I do think is a bit rude, seeing that is you who wants to publish in their Journal and not the other way around.
But despair not, most well known authors were rejected by a ton of publishers. But you know they did not want to change the establishment so they could get their way, they kept searching, and today we know them as George Lucas, Rowling, etc.
For each guy who presented their own breaking results in a non-peer reviewed way, I can tell you 10 who presented in peer reviewed journals.
Stephen Hawking, Einstein, Bohr, Darwing, Crick,…. all of them presented in peer reviewed conferences and Journals and their Nobel Prizes come from works they presented over there.
You should follow the books you recommend more closely, in “Write a lot” Silvia clearly says: “If your paper is not accepted in the Journal, do not get angry, do not send hate letters, just look for another Journal to publish”
@Leon
You misunderstood my post. The letter is a fictitious example meant to ridicule people who lack the ncessary intellectual maturity to publish their work.
I want to exclude from science peope who would write such a letter.
That submission sounds like there was a history of refusals from either the same journal, or a variety of similar publications.
Granted, the tone seems a bit brusque, but at least they get to their most important (what seems to be a rather sore) point.
OMG, this is hilarious! Some authors (actually the paper was good and kinda ahead of time) wrote that the refereed proposed to analyze their method analytically. They (authors) instead thank the referees for the suggestion, but postpone implementation for the future.
Man, they almost literally wrote that and it was published in an ACM journal.
“Publish then filter” currently exists as blogs and wikis. I can’t see print journals going this route for the simple reason that it would increase the number of pages by at least an order of magnitude (most journals reject 80-90% of submissions, and a publish-first model will discourage authors from heeding the adage that brevity is the soul of wit. If journals go to electronic format, space is a much less important issue – but the journal still risks losing its identity (and readership) if readers have to wade through a lot of muck to find a good paper.
@AdamHyland
A social filtration system would hopefully have the capability of stamping a “prestige” badge/reputation on papers. The value through scarcity would still exist, except the determination of “scarcity” wouldn’t be a function of whether a paper was published in a selective journal, but by how many highly selective social committees (presumably roving panels of experts or editors) endorsed a paper.
One could think of a social “like” mechanism, except applied to papers with a different rating system.
Big fan of your blog here.
I have been struggling (well, not really struggling) with a PhD for the last two years, and every time I read one of your posts I feel I will just not finish.
You see, as much as I love research, finding new knowledge, new ways and breaking the established paradigms, I hate writing. Sadly one of the requirements in my university to graduate is to publish a couple of journal papers, and I just can not get to write down the mess in my mind in an organised way.
As a side project, I started an small company where I can apply my research without nobody asking me about previous or related research (specially when there is no previous research) or formats or references, and well, it is flourishing (and getting some money as a side effect).
I love the pursuit of knowledge, to ask questions, find patterns, relations, reasons, causes and reactions (data/knowledge mining dude here). Yet, too many restrictions…
Anyway, this last post makes me fully realise I will never make it in academia. I have no option but to go my own way, and survive from there.
Thank you Daniel, you opened my eyes.
I forgot what was my point…
Leon’s reaction is hilarious!
@Daniel
As potential arguments in the submission letter, you forgot to mention: “I am Mr. Bigshot and I’ll have my paper published no matter what; you decide whether it’s in your journal or elsewhere.”
However, if I may add my gain of salt, a good scientific contribution must eventually convince people of its worthiness. It may not pass a journal board on its first try, but I guess it eventually must if it is really that good. The opposite would be that a small number of editorial boards in *different* venues consistently block a valid idea to the point it gets published nowhere, ever. That looks a bit improbable to me. Hence I don’t see what the publish-filter model adds *in the long run* over the existing process. It may delay the publishing of an interesting contribution in the short term, but cannot prevent something objectively interesting/true from emerging over time.
Or am I too naive?
@Hallé
Excellent point. I will answer with my next blog post.