Your post is reminiscent of the Cathedral and the Bazaar, by Eric Raymond. Raymond describes the conception of Linux as a modest proposal to a niche hacker community: “let’s see if we can build an OS.” The ambitions were no greater than that, but it put them on a path that led to what we now know as Open Source. He goes on to argue that it would have failed had the ambitions been that grand.
There is a clear parallel between the modest proposal of open-source projects–which I would consider pure engineering–and concrete research. A concrete robotics researcher may propose, “let’s try to build a robot that can recognize faces,” wherein the overarching research problem is to build humanoid robots. The immediate impact could be more effective image search on something like Facebook, which has pictures mainly of people.
It is curious that you use Semantic Web as an example, because I always felt its proposal in Wired in 2001 was overly ambitious and could not be logically achieved. In the two or so years that I have been following it, the field has made little progress. Semantics are being drawn from a huge variety of other sources (info. retrieval & extraction, NLP, data & text mining), all of which are more immediately practical, and making much bigger strides as such.
Are there other areas of abstract research you can name, and what is the current state? In other words, does this parallel hold?
Alekssays:
You *are* working on solving world hunger! Farmers will be able to find information on seeds faster using better tag clouds.
I haven’t tried to end world hunger, but I have tried to cure cancer, and it’s exhausting. I work in the world’s largest cancer center on projects that seek in some small way to make progress toward curing cancer. Sometimes I find the big picture inspiring, but often I find it daunting and I have to narrow my goal to running the next simulation.
Peter Turneysays:
I believe that science progresses by tiny, incremental, evolutionary steps. What you call “concrete research” is consistent with that view. I agree that what you call “abstract research” often fails, and the reason it fails is because it tries to make giant leaps; it ignores the importance of the incremental, evolutionary approach. However, it is possible to combine an incremental approach with a “big picture” view of the goal. When we walk on a rough trail, a good strategy is to alternate between looking at the ground around our feet and looking at the distant horizon, where our destination lies. When I do research, I alternate between the narrow technical task at hand and an ambitious vision of a long-term goal. I agree with you that detailed plans only work for the very short term, but I believe researchers should have a plausible story to tell about how their short-term work contributes to some long-term goal. When I lose faith in that story, I lose interest in my research, and switch to a new project.
Kevembuanggasays:
I define a problem to be too difficult when you cannot make noticeable progress in a matter of months.
How do you know you did or did not make progress?
(a private joke as you know, sorry, I couldn’t resist 🙂 )
I think I should come with more constructive arguments about which strategy is best for research since I disparage the “academic way” and “tiny, incremental, evolutionary steps”.
The Netflix Prize (*) seem to me a good setting to make a point, there has been a LOT of progress since it began but it is obviously stalling and even if the incumbent Big Dogs succeed in crossing the 10% barrier I doubt the contrived algorithm(s) will be of much usefulness for practical purposes.
Hypothesizing to the contrary that someone come up with a breakthrough, it would be interesting to know about the methodology used.
* – I know, I know, an “accuracy” problem and the RMSE metric is silly, but though…
Kevembuanggasays:
One more shot.
Abstract research derives from a formal plan. Semantic Web is one such a plan.
I find this not a very good example (or a purposely biased one?) of “abstract research”, it is indeed a Pie-In-The-Sky project fueled by plain hubris (Tim Berners-Lee “second coming”).
I rather see this as a misdirected engineering project, betting on not-yet-existing technical solutions, ontology reconciliation never worked to full expectations and anyway it cannot for deep reasons.
The Semantic Web has been copiously derided as a bloated toy and I took my fair share in this.
There was NEVER any questioning about what the “semantic” was supposed to be, it had been assumed from the start that semantic was logic and that the “times had come” to miraculously solve all the (well known!) nasty problems which had been lingering in AI research for decades.
And it was not either up to the GOFAI state of the art, not even as strong as database SQL, look for John SOWA criticisms about the “Syntactic Web”.
Krishnansays:
I think many researchers make the switch from concrete to abstract for the following reasons
1. They have had a large success (e.g. Berners Lee and the web) and smaller concrete problems will not bring that kind of splash. Hamming states that this happened to Shannon in his piece “You and your research”
2. Funding agencies do not get it if you talk about concrete problems (all this money for just a 2% improvement on that algorithm, sheesh you must be joking)
3. Progress on concrete problems is hard, requires patience, is fraught with uncertainities and requires constant upgradation in skills. Most of the times, the returns in terms of press mentions and fame is negligible.
I think, however, that it is very easy to fool yourself with a
“plausible” story. Times and times again, I show great skills at
fooling myself. Even on the small stuff. (That is why I like
experimental work!!!)
When I do research, I alternate between the narrow technical task at hand and an ambitious vision of a long-term goal. I agree with you that detailed plans only work for the very short term, but I believe researchers should have a plausible story to tell about how their short-term work contributes to some long-term goal. When I lose faith in that story, I lose interest in my research, and switch to a new project.
So do I. Great point Peter.
I think, however, that it is very easy to fool yourself with a “plausible” story. Times and times again, I show great skills at fooling myself. Even on the small stuff.
I think I should come with more constructive arguments about which strategy is best for research since I disparage the “academic way” and “tiny, incremental, evolutionary steps”.
My blog post is indeed part of an answer to your critics.
I do not think that the Netflix prize you take an example is typical of academic research. Academic research is not so narrow because individuals will attempt to differentiate themselves. People go in vastly different directions.
As an example, even in a large department within a given university, you will almost never see two professors working on the same sets of problems.
Kevembuanggasays:
I do not think that the Netflix prize you take an example is typical of academic research.
I say the failure of applying academic research to a targeted question like the Netflix prize shows the weakness of the approach.
If it ever gets “solved” in a different manner than painfully grinding one .001% after another then (may be…) it will make a case study for another approach.
Academic research is not so narrow because individuals will attempt to differentiate themselves. People go in vastly different directions.
I am not denying nor criticizing diversity, I am criticizing a “depth first” research versus a “breadth first” and the lack of any opportunity to do the later in a collaborative manner.
Quoting one of your emails:
I still cannot publish “an idea”.
Nor anyone else of course, but this is precisely the trouble spot, as you rightly say in a comment above, Times and times again, I show great skills at fooling myself. (like anyone else!).
If there were a mean or place where you could expose the idea there would likely be a cheaper feedback on bad ideas than for the originator to arduously debunk his own fault and, furthermore, even bad ideas may give rise to fruitful ideas by people who will either “remove the flaw” or be spun to a tangential but correct inspiration.
Kevembuanggasays:
When I do research, I alternate between the narrow technical task at hand and an ambitious vision of a long-term goal.
Yes but that vision isn’t shared as well or as thoroughly than a detailed published paper and therefore cannot be debated, amended, improved, etc… except by you an only you, no collaborative work here.
The only place where this happens is informal discussions between colleagues, a small world, because colleagues are likely to share many preconceptions about their shared research interests.
I even suspect that this is actually counterproductive because it reinforce fads and fashions.
Kevembuanggasays:
While I am at it…
World hunger isn’t a “problem”, world hunger is a symptom of overpopulation, unabated inter-group competition, biased trade and monetary rules and who knows what else.
Setting out to “solve the world hunger problem” without studying the causes, process and assumed values behind “the problem” is guaranteed to bring disasters (of a different kind :-> ) rather than relief.
Please note that this is roughly the same line of criticism than the one I apply to academic research.
Your post is reminiscent of the Cathedral and the Bazaar, by Eric Raymond. Raymond describes the conception of Linux as a modest proposal to a niche hacker community: “let’s see if we can build an OS.” The ambitions were no greater than that, but it put them on a path that led to what we now know as Open Source. He goes on to argue that it would have failed had the ambitions been that grand.
There is a clear parallel between the modest proposal of open-source projects–which I would consider pure engineering–and concrete research. A concrete robotics researcher may propose, “let’s try to build a robot that can recognize faces,” wherein the overarching research problem is to build humanoid robots. The immediate impact could be more effective image search on something like Facebook, which has pictures mainly of people.
It is curious that you use Semantic Web as an example, because I always felt its proposal in Wired in 2001 was overly ambitious and could not be logically achieved. In the two or so years that I have been following it, the field has made little progress. Semantics are being drawn from a huge variety of other sources (info. retrieval & extraction, NLP, data & text mining), all of which are more immediately practical, and making much bigger strides as such.
Are there other areas of abstract research you can name, and what is the current state? In other words, does this parallel hold?
You *are* working on solving world hunger! Farmers will be able to find information on seeds faster using better tag clouds.
I haven’t tried to end world hunger, but I have tried to cure cancer, and it’s exhausting. I work in the world’s largest cancer center on projects that seek in some small way to make progress toward curing cancer. Sometimes I find the big picture inspiring, but often I find it daunting and I have to narrow my goal to running the next simulation.
I believe that science progresses by tiny, incremental, evolutionary steps. What you call “concrete research” is consistent with that view. I agree that what you call “abstract research” often fails, and the reason it fails is because it tries to make giant leaps; it ignores the importance of the incremental, evolutionary approach. However, it is possible to combine an incremental approach with a “big picture” view of the goal. When we walk on a rough trail, a good strategy is to alternate between looking at the ground around our feet and looking at the distant horizon, where our destination lies. When I do research, I alternate between the narrow technical task at hand and an ambitious vision of a long-term goal. I agree with you that detailed plans only work for the very short term, but I believe researchers should have a plausible story to tell about how their short-term work contributes to some long-term goal. When I lose faith in that story, I lose interest in my research, and switch to a new project.
I define a problem to be too difficult when you cannot make noticeable progress in a matter of months.
How do you know you did or did not make progress?
(a private joke as you know, sorry, I couldn’t resist 🙂 )
I think I should come with more constructive arguments about which strategy is best for research since I disparage the “academic way” and “tiny, incremental, evolutionary steps”.
The Netflix Prize (*) seem to me a good setting to make a point, there has been a LOT of progress since it began but it is obviously stalling and even if the incumbent Big Dogs succeed in crossing the 10% barrier I doubt the contrived algorithm(s) will be of much usefulness for practical purposes.
Hypothesizing to the contrary that someone come up with a breakthrough, it would be interesting to know about the methodology used.
* – I know, I know, an “accuracy” problem and the RMSE metric is silly, but though…
One more shot.
Abstract research derives from a formal plan. Semantic Web is one such a plan.
I find this not a very good example (or a purposely biased one?) of “abstract research”, it is indeed a Pie-In-The-Sky project fueled by plain hubris (Tim Berners-Lee “second coming”).
I rather see this as a misdirected engineering project, betting on not-yet-existing technical solutions, ontology reconciliation never worked to full expectations and anyway it cannot for deep reasons.
The Semantic Web has been copiously derided as a bloated toy and I took my fair share in this.
There was NEVER any questioning about what the “semantic” was supposed to be, it had been assumed from the start that semantic was logic and that the “times had come” to miraculously solve all the (well known!) nasty problems which had been lingering in AI research for decades.
And it was not either up to the GOFAI state of the art, not even as strong as database SQL, look for John SOWA criticisms about the “Syntactic Web”.
I think many researchers make the switch from concrete to abstract for the following reasons
1. They have had a large success (e.g. Berners Lee and the web) and smaller concrete problems will not bring that kind of splash. Hamming states that this happened to Shannon in his piece “You and your research”
2. Funding agencies do not get it if you talk about concrete problems (all this money for just a 2% improvement on that algorithm, sheesh you must be joking)
3. Progress on concrete problems is hard, requires patience, is fraught with uncertainities and requires constant upgradation in skills. Most of the times, the returns in terms of press mentions and fame is negligible.
So do I. Great point.
I think, however, that it is very easy to fool yourself with a
“plausible” story. Times and times again, I show great skills at
fooling myself. Even on the small stuff. (That is why I like
experimental work!!!)
When I do research, I alternate between the narrow technical task at hand and an ambitious vision of a long-term goal. I agree with you that detailed plans only work for the very short term, but I believe researchers should have a plausible story to tell about how their short-term work contributes to some long-term goal. When I lose faith in that story, I lose interest in my research, and switch to a new project.
So do I. Great point Peter.
I think, however, that it is very easy to fool yourself with a “plausible” story. Times and times again, I show great skills at fooling myself. Even on the small stuff.
I think I should come with more constructive arguments about which strategy is best for research since I disparage the “academic way” and “tiny, incremental, evolutionary steps”.
My blog post is indeed part of an answer to your critics.
I do not think that the Netflix prize you take an example is typical of academic research. Academic research is not so narrow because individuals will attempt to differentiate themselves. People go in vastly different directions.
As an example, even in a large department within a given university, you will almost never see two professors working on the same sets of problems.
I do not think that the Netflix prize you take an example is typical of academic research.
I say the failure of applying academic research to a targeted question like the Netflix prize shows the weakness of the approach.
If it ever gets “solved” in a different manner than painfully grinding one .001% after another then (may be…) it will make a case study for another approach.
Academic research is not so narrow because individuals will attempt to differentiate themselves. People go in vastly different directions.
I am not denying nor criticizing diversity, I am criticizing a “depth first” research versus a “breadth first” and the lack of any opportunity to do the later in a collaborative manner.
Quoting one of your emails:
I still cannot publish “an idea”.
Nor anyone else of course, but this is precisely the trouble spot, as you rightly say in a comment above, Times and times again, I show great skills at fooling myself. (like anyone else!).
If there were a mean or place where you could expose the idea there would likely be a cheaper feedback on bad ideas than for the originator to arduously debunk his own fault and, furthermore, even bad ideas may give rise to fruitful ideas by people who will either “remove the flaw” or be spun to a tangential but correct inspiration.
When I do research, I alternate between the narrow technical task at hand and an ambitious vision of a long-term goal.
Yes but that vision isn’t shared as well or as thoroughly than a detailed published paper and therefore cannot be debated, amended, improved, etc… except by you an only you, no collaborative work here.
The only place where this happens is informal discussions between colleagues, a small world, because colleagues are likely to share many preconceptions about their shared research interests.
I even suspect that this is actually counterproductive because it reinforce fads and fashions.
While I am at it…
World hunger isn’t a “problem”, world hunger is a symptom of overpopulation, unabated inter-group competition, biased trade and monetary rules and who knows what else.
Setting out to “solve the world hunger problem” without studying the causes, process and assumed values behind “the problem” is guaranteed to bring disasters (of a different kind :-> ) rather than relief.
Please note that this is roughly the same line of criticism than the one I apply to academic research.