In a great paper, Michael Mulkay talked about three models of science. The open model is the mythical “science is open”. The closed model is much closer to what you describe, essentially the Kuhnian model where science gets stuck, followed by a crisis and a revolution.
Mulkay’s third model is the interesting one: science evolves by branching. People move between disciplines, and this is where the interesting stuff happens. There’s a lot to be said for this: many classic paradigm shifts (in a Kuhnian sense) were really ideas from one field transforming another. For Mulkay, these scientific communities were relatively small (a hundred or so people at most) working in a focused area. The classic disciplines of physics, etc., are higher-level constructs, but not really scientific communities. Most funding actually happens in these smaller fields.
But of course, there isn’t a free market, quite. Just as there isn’t for, say, kitchen appliances. Some appliances are popular, other less so. Some satisfy niche demands. There are big providers who try to do everything, and many smaller ones who don’t. The modern scientist is just trying to make a contribution, and in doing so, yes, is likely to overvalue the contribution they can make. But they are generally happy to compete within the bounds of the fields they know.
I embrace the idea of branching by a small group of people. But the competition scientists currently favor is nothing like free market competition. It is more like the competition between bureaucrats. The main goal is to climb higher in the hierarchy.
There are numerous issues with you argument. The major issue being that Peer Review actually prevents the God Complex and Halo effect. The fact that no matter how influential you might be in your field, your work wont be accepted unless it is passed by an anonymous panel of reviewers, actually prevents the God Complex.
If we had a completely open system, similar to say blogging, people would just arbitrarily follow the leader. Also the reason people flock to prestigious journals or conferences because everyone cannot read all the literature even in a relatively small area, the peer review process provides a good mechanism to rank the literature.
Finally you wish that there should be fewer “Star” scientists directing lesser ones rather you want more independent scientists. There are several issues with your argument. The first one being what is the distribution of the research skill in the population who want to be scientists? While you might like it to be near uniform, it might not necessarily be the case.
Finally consider literature, even though blogging has made sure that anyone with access to internet can access a large number of audience, yet it has not lead to increase in quality of literature being published.
All good points. Please keep in mind that my concerns had to do with two things: central planning, and discouraging diversity. Thus, I don’t necessarily disagree with the points you raise. They don’t contradict my post (in my view). Let me make my point of view precise.
The major issue being that Peer Review actually prevents the God Complex and Halo effect.
Realize that what you just did by leaving a comment is effectively peer review. And sure enough, I can write whatever I want on my blog (thinking of myself as a God), but people like you will come and tear apart my theories.
So, you are absolutely correct, peer review is a great way to tame the God complex. If you have been reading this blog, you know that I am very keen on peer review. However, I don’t believe that a small set of secret reviewers can assess accurately the long term significance of a piece of work. Thinking so is falling prey to the God complex.
What I am saying is not so controversial. Journals like PLoS One do not ask reviewers to assess the long term significance of the papers: they only ask whether the paper is correct.
What I oppose is what I call “conventional peer review” where reviewers are asked to guess whether the work is important.
(…) everyone cannot read all the literature even in a relatively small area, the peer review process provides a good mechanism to rank the literature.
Oddly enough people never complain that we lack a way to rank blogs.
Anyhow, if your filters are based on the journals, then you are probably overwhelmed. In fact, you have probably been overwhelmed since before WWII.
Most people I know have far more sophisticated filters. They follow specific individuals, citation networks and keywords.
I rely exclusively on Google Scholar, and it does not filter based on journals. I have no problem narrowing down my search. I often have the opposite difficulty: I often miss important related work because it appeared in venues I don’t know, written by authors I don’t know, using keywords I don’t know.
This is not to say that curated list of papers such as those generated by journals have no value. They do. But while they were essential in the pre-Internet era, they are now only one signal out of many. And maybe not the most important signal.
Some of the best work I have ever read was presented in small workshops. In fact, it is quite common for people to notice that their most cited work appeared in minor venues. It is certainly my case: there seems to be an inverse relationship between the impact of my work (as measured by citations) and the prestige of the venue. (Before you say that this is impossible, look at the distribution of citations among papers in prestigious journals. You will find that most of them are rarely cited.)
The first one being what is the distribution of the research skill in the population who want to be scientists? While you might like it to be near uniform, it might not necessarily be the case.
I’m not assuming that the productivity of scientists is uniform. In fact, there is a substantial statistical dispersion. However, we do not know how much of this dispersion is due to the system and how much is due to innate skills.
I believe we greatly overestimate the effect of innate skills. Consider that in the early XXth century, German scientists had sometimes a substantial edge over other scientists. Then after WWII, American scientists dominated. It is quite obvious that the “great scientists” somehow appear whenever a country becomes powerful. Strange, no? Clearly, it is not the genetics that changes… but the working conditions. Good working conditions tend to generate great scientists.
There is a Matthew effect at work. By trying to select the most productive people and giving them more resources, you create a self-fulfilling prophecy. It is unclear whether this form of attribution of resources is optimal. Do you get more science by giving one million dollars to one man, or 100 thousand dollars to ten men? The answer is maybe not obvious.
Stuart Wattsays:
@Phil
The fragmentation is nothing new. Physics has many interpretations of quantum mechanics, that’s not so very different from different interpretations of climate data. The main change is a huge transformation in media and communication, which has opened all this up to external scrutiny, and which often has a political or economic agenda it uses to selectively draw on scientific results.
However, as scientists, we still have to adhere to basic rules. Creationism is not a science — and this has been tested legally. Conspiracy theories are not scientifically testable. (Perpetual motion: more interesting as the laws of thermodynamics are descriptive, not causal, so research on *why* entropy increases could actually be interesting.)
I think you nicely demonstrate why more extreme versions of relativism are basically useless.
Even in a free market, products have to be useful. Without that, they will fail. Novelty items can exist, but are inevitably marginal. I’d consider conspiracy theories et al. as novelty items, some people will buy them, they can be fun, but you’d never use one to cook your dinner.
Phil Jonessays:
The interesting thought is that the free market tends towards pluralism. As in more competing niche products.
And the implication for “science” is that a truly free market of ideas (eg. the internet) would similarly lead to fragmentation, large numbers of niches of people who believe their own incompatible theories, each of which is backed by its own canon of evidence, that its community considers trustworthy.
I think we’re seeing the beginnings of this with the proliferation of global warming denialists, perpetual motion machinists, creationists, conspiracy theorists etc. And fragmentation will continue.
Lurking behind our idea of “Science” is a picture of a consistent nature which acts as arbitrator of such disputes. But the better picture of the future might be the blind-men analysing an elephant : multiple perspectives based on small windows of observation, failing to grok an overall coherent picture.
In this sense, a “free market” of science might actually spell the end of anything that we could recognise as science at all. Leaving us with nothing but a multitude of local belief systems.
There may be a process of Natural Selection that goes on (or ought to be going on), but I’m not sure that the “truth of the way things are” is a fitness function. It might have been in the 17th C. but I don’t think it is now. “Is there a revenue stream from this idea” or “can I patent it” seems to be the modus operandi now.
But your point that the Stalinist 5-year plan isn’t any good at *that* either is well-taken.
Love the idea of garage science. Hard to publish that way though. You would have to be happy with other ways of having an impact, I’d think.
@Greg
All calculus students learn about “Green’s theorem”. Ever wondered who was Green?
He was very much a garage scientist:
http://en.wikipedia.org/wiki/George_Green
(Though they did not have garages back then.)
OK, you asked for it: I will disagree.
In a great paper, Michael Mulkay talked about three models of science. The open model is the mythical “science is open”. The closed model is much closer to what you describe, essentially the Kuhnian model where science gets stuck, followed by a crisis and a revolution.
Mulkay’s third model is the interesting one: science evolves by branching. People move between disciplines, and this is where the interesting stuff happens. There’s a lot to be said for this: many classic paradigm shifts (in a Kuhnian sense) were really ideas from one field transforming another. For Mulkay, these scientific communities were relatively small (a hundred or so people at most) working in a focused area. The classic disciplines of physics, etc., are higher-level constructs, but not really scientific communities. Most funding actually happens in these smaller fields.
But of course, there isn’t a free market, quite. Just as there isn’t for, say, kitchen appliances. Some appliances are popular, other less so. Some satisfy niche demands. There are big providers who try to do everything, and many smaller ones who don’t. The modern scientist is just trying to make a contribution, and in doing so, yes, is likely to overvalue the contribution they can make. But they are generally happy to compete within the bounds of the fields they know.
@Stuart
I embrace the idea of branching by a small group of people. But the competition scientists currently favor is nothing like free market competition. It is more like the competition between bureaucrats. The main goal is to climb higher in the hierarchy.
There are numerous issues with you argument. The major issue being that Peer Review actually prevents the God Complex and Halo effect. The fact that no matter how influential you might be in your field, your work wont be accepted unless it is passed by an anonymous panel of reviewers, actually prevents the God Complex.
If we had a completely open system, similar to say blogging, people would just arbitrarily follow the leader. Also the reason people flock to prestigious journals or conferences because everyone cannot read all the literature even in a relatively small area, the peer review process provides a good mechanism to rank the literature.
Finally you wish that there should be fewer “Star” scientists directing lesser ones rather you want more independent scientists. There are several issues with your argument. The first one being what is the distribution of the research skill in the population who want to be scientists? While you might like it to be near uniform, it might not necessarily be the case.
Finally consider literature, even though blogging has made sure that anyone with access to internet can access a large number of audience, yet it has not lead to increase in quality of literature being published.
@Bhat
All good points. Please keep in mind that my concerns had to do with two things: central planning, and discouraging diversity. Thus, I don’t necessarily disagree with the points you raise. They don’t contradict my post (in my view). Let me make my point of view precise.
The major issue being that Peer Review actually prevents the God Complex and Halo effect.
Realize that what you just did by leaving a comment is effectively peer review. And sure enough, I can write whatever I want on my blog (thinking of myself as a God), but people like you will come and tear apart my theories.
So, you are absolutely correct, peer review is a great way to tame the God complex. If you have been reading this blog, you know that I am very keen on peer review. However, I don’t believe that a small set of secret reviewers can assess accurately the long term significance of a piece of work. Thinking so is falling prey to the God complex.
What I am saying is not so controversial. Journals like PLoS One do not ask reviewers to assess the long term significance of the papers: they only ask whether the paper is correct.
What I oppose is what I call “conventional peer review” where reviewers are asked to guess whether the work is important.
(…) everyone cannot read all the literature even in a relatively small area, the peer review process provides a good mechanism to rank the literature.
Oddly enough people never complain that we lack a way to rank blogs.
Anyhow, if your filters are based on the journals, then you are probably overwhelmed. In fact, you have probably been overwhelmed since before WWII.
Most people I know have far more sophisticated filters. They follow specific individuals, citation networks and keywords.
I rely exclusively on Google Scholar, and it does not filter based on journals. I have no problem narrowing down my search. I often have the opposite difficulty: I often miss important related work because it appeared in venues I don’t know, written by authors I don’t know, using keywords I don’t know.
This is not to say that curated list of papers such as those generated by journals have no value. They do. But while they were essential in the pre-Internet era, they are now only one signal out of many. And maybe not the most important signal.
Some of the best work I have ever read was presented in small workshops. In fact, it is quite common for people to notice that their most cited work appeared in minor venues. It is certainly my case: there seems to be an inverse relationship between the impact of my work (as measured by citations) and the prestige of the venue. (Before you say that this is impossible, look at the distribution of citations among papers in prestigious journals. You will find that most of them are rarely cited.)
The first one being what is the distribution of the research skill in the population who want to be scientists? While you might like it to be near uniform, it might not necessarily be the case.
I’m not assuming that the productivity of scientists is uniform. In fact, there is a substantial statistical dispersion. However, we do not know how much of this dispersion is due to the system and how much is due to innate skills.
I believe we greatly overestimate the effect of innate skills. Consider that in the early XXth century, German scientists had sometimes a substantial edge over other scientists. Then after WWII, American scientists dominated. It is quite obvious that the “great scientists” somehow appear whenever a country becomes powerful. Strange, no? Clearly, it is not the genetics that changes… but the working conditions. Good working conditions tend to generate great scientists.
There is a Matthew effect at work. By trying to select the most productive people and giving them more resources, you create a self-fulfilling prophecy. It is unclear whether this form of attribution of resources is optimal. Do you get more science by giving one million dollars to one man, or 100 thousand dollars to ten men? The answer is maybe not obvious.
@Phil
The fragmentation is nothing new. Physics has many interpretations of quantum mechanics, that’s not so very different from different interpretations of climate data. The main change is a huge transformation in media and communication, which has opened all this up to external scrutiny, and which often has a political or economic agenda it uses to selectively draw on scientific results.
However, as scientists, we still have to adhere to basic rules. Creationism is not a science — and this has been tested legally. Conspiracy theories are not scientifically testable. (Perpetual motion: more interesting as the laws of thermodynamics are descriptive, not causal, so research on *why* entropy increases could actually be interesting.)
I think you nicely demonstrate why more extreme versions of relativism are basically useless.
Even in a free market, products have to be useful. Without that, they will fail. Novelty items can exist, but are inevitably marginal. I’d consider conspiracy theories et al. as novelty items, some people will buy them, they can be fun, but you’d never use one to cook your dinner.
The interesting thought is that the free market tends towards pluralism. As in more competing niche products.
And the implication for “science” is that a truly free market of ideas (eg. the internet) would similarly lead to fragmentation, large numbers of niches of people who believe their own incompatible theories, each of which is backed by its own canon of evidence, that its community considers trustworthy.
I think we’re seeing the beginnings of this with the proliferation of global warming denialists, perpetual motion machinists, creationists, conspiracy theorists etc. And fragmentation will continue.
Lurking behind our idea of “Science” is a picture of a consistent nature which acts as arbitrator of such disputes. But the better picture of the future might be the blind-men analysing an elephant : multiple perspectives based on small windows of observation, failing to grok an overall coherent picture.
In this sense, a “free market” of science might actually spell the end of anything that we could recognise as science at all. Leaving us with nothing but a multitude of local belief systems.
I take it that you subscribe to an “Evolutionary Epistemology of Theories” approach to understanding how science works (or ought to work.) see:
http://plato.stanford.edu/entries/epistemology-evolutionary/
There may be a process of Natural Selection that goes on (or ought to be going on), but I’m not sure that the “truth of the way things are” is a fitness function. It might have been in the 17th C. but I don’t think it is now. “Is there a revenue stream from this idea” or “can I patent it” seems to be the modus operandi now.
But your point that the Stalinist 5-year plan isn’t any good at *that* either is well-taken.