I missed the part where he explains how to measure research productivity and test hypotheses about what optimizes it.
He does not provide a definition nor does he test hypotheses, but he points to others who do, such as Boice, Gray, and so on.
It is an opinion piece, not an experiment report.
And the part where he claims research abilities are innate.
He claims the opposite: “the key to high-level performance in all sorts of fields is deliberate practice, over many years, with appropriate feedback.”
He also writes:” (…) the common research management emphasis on highly productive researchers, often leaders of teams, commonly linked with an assumption that research performance depends on natural ability.”
Ok. The average university professor hardly publishes a single paper a year. Often, the median professor does not write a single paper a year! That is certainly true in many fields!
People accept this system because of the hidden assumption that these people, intrinsically, are not “good”. They can’t produce.
So, we have a few key professors, often in a few elite universities, who produce most of the research.
Martin and I think that you can increase the total output by looking at these unproductive fellow and assuming that they *can* produce given the right training, motivations and feedback.
That’s related to the long tail idea… most professors are in the long tail of the research system. If we could just increase their productivity by 20%, it would make a huge difference.
Here is an example.
You run a grant agency. You have a fixed budget. Say 100,000,000 dollars. Your goal is to grow the research productivity of a certain region.
What do you do?
1) Select the 10 best researchers, give them each 10,000,000 dollars. All you have to worry about is selecting the best.
It is a self-fullfilling prophecy too, so you can’t be wrong! These researchers you focus upon are very likely to be productive, after all, they were already good to begin with!
But what is the net effect of your money? The same researchers who were productive before are still productive… though maybe marginally more now. Others are no more productive.
In fact, others may be less productive because they now carry the shame of not havng been selected as part of the “highly productive researchers”.
2) Or else, you determine that research abilities can be grown. So you have 10 highly productive researchers. That is good. But maybe if your region had 100 highly productive researchers, things would be better!
So you pick the 10 best researchers and also 90 promising researchers, and you offer them each 100,000 dollars. Then you encourage all of them to become highly productive.
If you succeed, if all of them become highly productive, you’ll have made a much bigger difference that way!
I missed the part where he explains how to measure research productivity and test hypotheses about what optimizes it. And the part where he claims research abilities are innate.
(…) incentives for research reduce intrinsic motivations; and, on the other hand, offering researchers grant money encourages them to become highly productive
Not offering money to some while offering lots to others will discourage the losers. They will just give up and move on to other things. And indeed, that is what the system is often designed to do.
Also, money is required to do research. If I did not have grant money, how would I even pay for the basic stuff, like a new laptop? Trip to a conference? And so on. (I am not even going to include salaries.)
Thankfully, I do have grant money, but if I did not, for an extended period of time, I might give up on research. For these two reasons.
The other issues seem peripheral.
That success is the result of effort, training and proper feedback? This is very important!
The people working with you, do you consider that their level of expertise and skills are somehow innate? If you do, you may dramatically hurt their performance on the long run.
Good companies expect their employees to improve through consistent training and feedback.
More importantly, realizing that you can raise your own level through efforts, and keeping this in mind, can make you better!
Sorry, what I meant to say what that I missed the part where he points to others who claim that research abilities are innate. I’d missed that “natural ability” clause.
But it still feels like a bit of a straw man. The explanation could just as consistently be that it’s easier or most cost-effective for an employer to hire people who are already good than to train them yourself–especially if the training may represent a one-way commitment–and I wonder what fraction of academics and researchers are loyal to their institutions not to take the training and run.
In any case, there’s no need to appeal to explanations of natural vs. acquired ability.
I’m confused about how, on one hand, incentives for research reduce intrinsic motivations; and, on the other hand, offering researchers grant money encourages them to become highly productive. Or are you saying that grant money shouldn’t be seen as an incentive at all, but rather is something a researcher should take for granted? Or should see as a lottery?
I can see how the current star system creates an unhealthy and inefficient positive feedback loop, and that there’s value in diverting more funds to potential gems in the rough, along the lines of how Billy Beane invested in the A’s. Grant agencies should act more like rational portfolio managers. If that’s all you and Martin are advocating, then we agree. The other issues seem peripheral.
Francois Rivestsays:
Very interesting pointer. More people should read it.
On the incentive point, are you then saying that grant money shouldn’t be seen as an incentive at all, but rather is something a researcher should take for granted?
As for training and feedback, yes, good employers do that. But they also reward performance and strongly prefer employees who come in with strong qualifications. Companies may have less of a star system than academia, but most believe in using incentives to improve productivity.
I think we’re talking past each other. You and Martin seem convinced that the main problem is that universities and research labs don’t believe that research ability is determined by anything other than natural ability. Why isn’t it possible that they simply prefer to get their researchers pre-trained. That’s what many employers do, even good ones.
Simon Sperosays:
It’s hard to measure research productivity, and it is thus impossible to know which researchers to fund in order to developer better Scientometrics.
Fortunately, there is a simple escape from this apparent paradox; by switching funding away from researchers and towards administration, it becomes easy to ensure that no unmeasured productivity occurs accidentally.
On the incentive point, are you then saying that grant money shouldn’t be seen as an incentive at all, but rather is something a researcher should take for granted?
At least in Canada, most grants given to researchers are not incentives. My salary is the same whether I get $1 million or no grant whatsover.
Of course, if you like to travel, then grant money can be used as free travel money, I guess. But I hate to travel.
Overwhelmingly, in Canada, grant money is used for scholarships given out to graduate students.
Right now, I have two Ph.D. students, and I had a post-doctoral fellow for a number of years. That’s where my money went. None of it went into my pocket.
I think we’re talking past each other. You and Martin seem convinced that the main problem is that universities and research labs don’t believe that research ability is determined by anything other than natural ability. Why isn’t it possible that they simply prefer to get their researchers pre-trained. That’s what many employers do, even good ones.
Ok. But once you have your researchers in place, how do you improve their productivity? Suppose you are a country like Canada or Australia… sure, you can try to recruit the best Ph.D. students, the best professors… but I submit to you that there is a limit.
Even in an average Canadian university, if we open a professorship, we will get at least 200 applicants. So, there is already a lot of intense scrutiny when you recruit.
If you are going to improve things further, you need to work beyond the recruitment phase. Now, if you believe that abilities are innate, then there is no point… you can only select the best, and after that, there is nothing else you can do.
What Martin says is that a country like Australia could substantially improve its research output by being more nurturing to its current set of researchers, instead of always focusing on recruitment.
Surely you’re not arguing that money is the main incentive for academic researchers? At least in my experience, the main incentive–and area of competition–for academics is status.
Precisely. In fact, too much money can divert attention from research! And I mean that! If you give too much money to a researcher, he’ll have to become a manager… and he’ll stop doing bona fide research.
But, it plays in reverse. If you are too poor, even the small things a researcher must do will become too expensive. That is precisely the reason why we are not totally overwhelmed by Indian and Chinese researchers (yet).
In any case, I agree that, after you go through the effort of selecting the best, it is a good strategy to nurture them. In fact, a nurturing environment can even serve to increase retention. At least that my attitude as a manager, and it’s served me well.
But given that the median college professor does not even publish a single paper a year, there is obviously a great need for more nurturing.
I think I finally see your point–that, if you believe research abilities are innate, then nurturing researchers serves no purpose. I thought you and Martin were arguing the converse: that those who don’t nurture researchers necessarily believe that research abilities are innate.
I think it is unconscious. People may not realize that they are acting under the assumption that research abilities are innate.
Put it another way… if a researcher produces good results… we should not turn him into a superhero… imagine him as a superior being. Instead, we should look around and wonder what other researchers could pick up to become like him…
We should try to figure out what makes him tick and duplicate it!
Surely you’re not arguing that money is the main incentive for academic researchers? At least in my experience, the main incentive–and area of competition–for academics is status.
In any case, I agree that, after you go through the effort of selecting the best, it is a good strategy to nurture them. In fact, a nurturing environment can even serve to increase retention. At least that my attitude as a manager, and it’s served me well.
I think I finally see your point–that, if you believe research abilities are innate, then nurturing researchers serves no purpose. I thought you and Martin were arguing the converse: that those who don’t nurture researchers necessarily believe that research abilities are innate.
You worry about the best researchers becoming managers, and yet the best managers are those who nurture others and bring out the best in them. Yes, management can also involve a lot of administrivia, but that part can be outsourced. The mentorship part cannot.
I’m not persuaded that the problem is our failure to document best practices. There are good write-ups on how to be a researcher. But people learn these skills best through mentorship.
And universities often do incent such mentorship indirectly: joint papers may reflect more mentorship on the part of a senior researcher, more leg work on the part of a junior one. Where this system probably breaks down is that we don’t see such mentorship among researchers who are peers in seniority but not status. And I suspect that comes back to the competitive individualist culture of academia.
I missed the part where he explains how to measure research productivity and test hypotheses about what optimizes it.
He does not provide a definition nor does he test hypotheses, but he points to others who do, such as Boice, Gray, and so on.
It is an opinion piece, not an experiment report.
And the part where he claims research abilities are innate.
He claims the opposite: “the key to high-level performance in all sorts of fields is deliberate practice, over many years, with appropriate feedback.”
He also writes:” (…) the common research management emphasis on highly productive researchers, often leaders of teams, commonly linked with an assumption that research performance depends on natural ability.”
Ok. The average university professor hardly publishes a single paper a year. Often, the median professor does not write a single paper a year! That is certainly true in many fields!
People accept this system because of the hidden assumption that these people, intrinsically, are not “good”. They can’t produce.
So, we have a few key professors, often in a few elite universities, who produce most of the research.
Martin and I think that you can increase the total output by looking at these unproductive fellow and assuming that they *can* produce given the right training, motivations and feedback.
That’s related to the long tail idea… most professors are in the long tail of the research system. If we could just increase their productivity by 20%, it would make a huge difference.
Here is an example.
You run a grant agency. You have a fixed budget. Say 100,000,000 dollars. Your goal is to grow the research productivity of a certain region.
What do you do?
1) Select the 10 best researchers, give them each 10,000,000 dollars. All you have to worry about is selecting the best.
It is a self-fullfilling prophecy too, so you can’t be wrong! These researchers you focus upon are very likely to be productive, after all, they were already good to begin with!
But what is the net effect of your money? The same researchers who were productive before are still productive… though maybe marginally more now. Others are no more productive.
In fact, others may be less productive because they now carry the shame of not havng been selected as part of the “highly productive researchers”.
2) Or else, you determine that research abilities can be grown. So you have 10 highly productive researchers. That is good. But maybe if your region had 100 highly productive researchers, things would be better!
So you pick the 10 best researchers and also 90 promising researchers, and you offer them each 100,000 dollars. Then you encourage all of them to become highly productive.
If you succeed, if all of them become highly productive, you’ll have made a much bigger difference that way!
I missed the part where he explains how to measure research productivity and test hypotheses about what optimizes it. And the part where he claims research abilities are innate.
(…) incentives for research reduce intrinsic motivations; and, on the other hand, offering researchers grant money encourages them to become highly productive
Not offering money to some while offering lots to others will discourage the losers. They will just give up and move on to other things. And indeed, that is what the system is often designed to do.
Also, money is required to do research. If I did not have grant money, how would I even pay for the basic stuff, like a new laptop? Trip to a conference? And so on. (I am not even going to include salaries.)
Thankfully, I do have grant money, but if I did not, for an extended period of time, I might give up on research. For these two reasons.
The other issues seem peripheral.
That success is the result of effort, training and proper feedback? This is very important!
The people working with you, do you consider that their level of expertise and skills are somehow innate? If you do, you may dramatically hurt their performance on the long run.
Good companies expect their employees to improve through consistent training and feedback.
More importantly, realizing that you can raise your own level through efforts, and keeping this in mind, can make you better!
This is worth repeating again and again.
Sorry, what I meant to say what that I missed the part where he points to others who claim that research abilities are innate. I’d missed that “natural ability” clause.
But it still feels like a bit of a straw man. The explanation could just as consistently be that it’s easier or most cost-effective for an employer to hire people who are already good than to train them yourself–especially if the training may represent a one-way commitment–and I wonder what fraction of academics and researchers are loyal to their institutions not to take the training and run.
In any case, there’s no need to appeal to explanations of natural vs. acquired ability.
I’m confused about how, on one hand, incentives for research reduce intrinsic motivations; and, on the other hand, offering researchers grant money encourages them to become highly productive. Or are you saying that grant money shouldn’t be seen as an incentive at all, but rather is something a researcher should take for granted? Or should see as a lottery?
I can see how the current star system creates an unhealthy and inefficient positive feedback loop, and that there’s value in diverting more funds to potential gems in the rough, along the lines of how Billy Beane invested in the A’s. Grant agencies should act more like rational portfolio managers. If that’s all you and Martin are advocating, then we agree. The other issues seem peripheral.
Very interesting pointer. More people should read it.
On the incentive point, are you then saying that grant money shouldn’t be seen as an incentive at all, but rather is something a researcher should take for granted?
As for training and feedback, yes, good employers do that. But they also reward performance and strongly prefer employees who come in with strong qualifications. Companies may have less of a star system than academia, but most believe in using incentives to improve productivity.
I think we’re talking past each other. You and Martin seem convinced that the main problem is that universities and research labs don’t believe that research ability is determined by anything other than natural ability. Why isn’t it possible that they simply prefer to get their researchers pre-trained. That’s what many employers do, even good ones.
It’s hard to measure research productivity, and it is thus impossible to know which researchers to fund in order to developer better Scientometrics.
Fortunately, there is a simple escape from this apparent paradox; by switching funding away from researchers and towards administration, it becomes easy to ensure that no unmeasured productivity occurs accidentally.
(For those who reject this model, http://www.academicproductivity.com/ has a lot of helpful discussions)
On the incentive point, are you then saying that grant money shouldn’t be seen as an incentive at all, but rather is something a researcher should take for granted?
At least in Canada, most grants given to researchers are not incentives. My salary is the same whether I get $1 million or no grant whatsover.
Of course, if you like to travel, then grant money can be used as free travel money, I guess. But I hate to travel.
Overwhelmingly, in Canada, grant money is used for scholarships given out to graduate students.
Right now, I have two Ph.D. students, and I had a post-doctoral fellow for a number of years. That’s where my money went. None of it went into my pocket.
I think we’re talking past each other. You and Martin seem convinced that the main problem is that universities and research labs don’t believe that research ability is determined by anything other than natural ability. Why isn’t it possible that they simply prefer to get their researchers pre-trained. That’s what many employers do, even good ones.
Ok. But once you have your researchers in place, how do you improve their productivity? Suppose you are a country like Canada or Australia… sure, you can try to recruit the best Ph.D. students, the best professors… but I submit to you that there is a limit.
Even in an average Canadian university, if we open a professorship, we will get at least 200 applicants. So, there is already a lot of intense scrutiny when you recruit.
If you are going to improve things further, you need to work beyond the recruitment phase. Now, if you believe that abilities are innate, then there is no point… you can only select the best, and after that, there is nothing else you can do.
What Martin says is that a country like Australia could substantially improve its research output by being more nurturing to its current set of researchers, instead of always focusing on recruitment.
Surely you’re not arguing that money is the main incentive for academic researchers? At least in my experience, the main incentive–and area of competition–for academics is status.
Precisely. In fact, too much money can divert attention from research! And I mean that! If you give too much money to a researcher, he’ll have to become a manager… and he’ll stop doing bona fide research.
But, it plays in reverse. If you are too poor, even the small things a researcher must do will become too expensive. That is precisely the reason why we are not totally overwhelmed by Indian and Chinese researchers (yet).
In any case, I agree that, after you go through the effort of selecting the best, it is a good strategy to nurture them. In fact, a nurturing environment can even serve to increase retention. At least that my attitude as a manager, and it’s served me well.
But given that the median college professor does not even publish a single paper a year, there is obviously a great need for more nurturing.
I think I finally see your point–that, if you believe research abilities are innate, then nurturing researchers serves no purpose. I thought you and Martin were arguing the converse: that those who don’t nurture researchers necessarily believe that research abilities are innate.
I think it is unconscious. People may not realize that they are acting under the assumption that research abilities are innate.
Put it another way… if a researcher produces good results… we should not turn him into a superhero… imagine him as a superior being. Instead, we should look around and wonder what other researchers could pick up to become like him…
We should try to figure out what makes him tick and duplicate it!
Surely you’re not arguing that money is the main incentive for academic researchers? At least in my experience, the main incentive–and area of competition–for academics is status.
In any case, I agree that, after you go through the effort of selecting the best, it is a good strategy to nurture them. In fact, a nurturing environment can even serve to increase retention. At least that my attitude as a manager, and it’s served me well.
I think I finally see your point–that, if you believe research abilities are innate, then nurturing researchers serves no purpose. I thought you and Martin were arguing the converse: that those who don’t nurture researchers necessarily believe that research abilities are innate.
You worry about the best researchers becoming managers, and yet the best managers are those who nurture others and bring out the best in them. Yes, management can also involve a lot of administrivia, but that part can be outsourced. The mentorship part cannot.
I’m not persuaded that the problem is our failure to document best practices. There are good write-ups on how to be a researcher. But people learn these skills best through mentorship.
And universities often do incent such mentorship indirectly: joint papers may reflect more mentorship on the part of a senior researcher, more leg work on the part of a junior one. Where this system probably breaks down is that we don’t see such mentorship among researchers who are peers in seniority but not status. And I suspect that comes back to the competitive individualist culture of academia.