Daniel Lemire's blog

, 19 min read

Expert performance and training: what we really know

18 thoughts on “Expert performance and training: what we really know”

  1. Schooling is super important. However, what I have noticed how, in general, poor the explanations are. I haven’t paid much attention while I was at school, But now that I have a son I started to notice this. For example, math teaching is probably horrible almost everywhere. Mostly, you learn algorithms, they rarely try to explain why things work one or another way.

  2. @Leo

    I have been generalized pleased by the mathematical education my sons are getting.

    I agree that they do not spend much time explaining why things work… but I am fine with kids learning to execute algorithms before they learn to design them…

    At least, the mathematical curriculum was designed by people who understand mathematics…

    I am a lot more concerned with how they teach languages. I am not sure they know what they are doing. In fact, I am pretty sure they do not.

  3. Alejo says:

    It’s funny you mention marathon and genetics. One of the highest (if not the highest) normalized scores of IAAF rankings was by Paula Radcliffe. British, blond, super pale. Her record is still high up there unbeaten for a decade and nobody is getting close. Also she is quite vocal anti-doping so that’s not an explanation.

    My unscientific theory is this is a complexity scenario, not just nature vs nurture. The best seem to have a good base of both but they excel at improving or minimizing weaknesses and maximizing strengths while keeping all in balance.

    There are mind tools for each weakness. For example, to cope a tendency to procrastinate there’s GTD, for absent minded mistakes there’s the checklist method, drowsyness there’s caffeinated drinks. Or more controversial, the use of nootropics.

    The guy with the highest measured IQ runs a farm. He has the potential but no results. I’ve seen this pattern very often with super-intelligent people, almost every case. It’s like they can’t keep focused or can’t deal with the anxiety. Maybe their brains work too hard to keep pace, or overheat.

    Great post on a fascinating subject!

  4. One thing I realize as I learn skills is that things are not as linear as they seem. It’s true that trivial practice will not not get you much far, that you need the right kind of practice, and meeting the proper people, and so on. Amongst the skills I have learned, for any given amount of practice, there are some I have become very good at, others I have not improved on much.

    What I feel is at work here is a kind of virtuous cycle. You try something: you’re good enough at it to get a certain reward. You then get the will to make the proper, often harder effort that will get you to progress. This effort does not seem so hard to you, because you’re enjoying most of it. You get progress and feel rewarded by it. Then you get on the next step. Each step is exponentially higher than the last, but each time you get exponentially happier about your progress, so you can put up the resources to succeed at the exponentially greater work that is needed.

    This means if you don’t get exponentially happier about your progress at each step, you don’t have the fuel (so to speak) to climb up the next step.

    tl;dr: innate skills and practice are, by themselves, insufficient for elevating a skill to world-class level. You need a mix of support, practice and self-appreciation that all reinforce each other to get you to the next step, which is always N>1 times as tough to get to as the last.

  5. @Alejo

    One of my research collaborators runs a small farm. I am sure he has a very high IQ.

  6. @Benoit

    You need a mix of support, practice and self-appreciation that all reinforce each other to get you to the next step

    I agree. The hard work does not come out of nowhere. It has to be sustained.

  7. Anton says:

    I must say that you should be interested in activity you trying to master. It must give you pleasure, or else you will not be able to «work hard» on it.

    I took «word hard» in quotes because it looks so for an observer. For involved person it is not work, it is style of life.

    I would like to mention philosophical Russian cartoon series «Kikoriki», episode «Pair macrame». Unfortunately, this episode was not translated to English; nevertheless, it worth watching:
    http://www.youtube.com/watch?v=JsSF0F4O00g

    In short, the protagonist was introduced to macrame, and finds it extremely boring; he wants to be a poet instead! Nevertheless, soon he become best at macrame, outperforming his teacher. He hates macrame, he hates himself, and the whole situation looks surrealistic.

  8. @Anton

    I agree. But what I mean by hard work is long hours of practice. To some, these long hours can be enjoyable.

  9. Alejo says:

    @Daniel

    Oh, I meant no disrespect. My family had a farm when I was young (weekend hippie farmers). I failed to express the anecdote in context, clearly. Sorry.

  10. @Alejo

    I did not take it as a lack of respect. It does seem true that farmers have lower IQs than medical doctors:

    http://www.iqcomparisonsite.com/occupations.aspx

  11. Paul says:

    I’ve been learning to play the boardgame “Go”, and it’s been a great way to think about learning/training since there are objectives measures absent from things like research and programming.

    What I’ve found is the following:

    1) There’s a local club with people from all walks of life: math professors to warehouse workers. There isn’t a noticeable correlation between skill or learning rate and the intellectual nature of the person’s job.

    2) The slowest to improve seem to be the ones most upset by a loss or lack of progress. The best players are the most eager to review a finished game to find mistakes.

    3) The biggest predictor for my own improvement rate seems to be the frequency with which I think about the game while not playing it. At the fastest improvement I’ll find myself mulling over a position in bed or while walking. When life gets busy or other interests fill up my idle thoughts, improvement stalls.

  12. Atul Mehta says:

    There are observed differences in the performance of students based on genetics and social backgrounds. But are school grades an accurate measure of a person’s potential? Has schooling today become so old school that we are churning out lots of rote learners and ranking them using an outmoded factory model of education? There’s an active debate among education reformers on TED: http://www.ted.com/playlists/24/re_imagining_school. Ray Kurzweil thinks that personal intelligence levels will become less relevant in the future where he sees augmented intelligence as part of evolution. Maybe we’ve already started living in an age of primitive augmented intelligence — anyone can do an Internet search and acquire more knowledge about a specific topic of interest. Sure there are few polymaths but anyone can be an automath which is an important part of skill development along with practice and discipline. With all due respect to James Flynn, IQ does measure intelligence to some degree. But does it accurately portray critical thinking and abstract reasoning skills, and does it accurately measure intelligence? Let’s say someone took an IQ test and scored 135 without prep, and then went through some rigorous prep work for a couple of months and then scores a 165.

    Given the impact of advances in genetics on medicine over the last decade, I posed some questions on the subject to three experienced doctors who knew so little about it that it was shocking. On the other hand, a car salesman who had researched the topic on his own had a much better perspective!

  13. Tal says:

    The problem with arguments over whether practice or talent are responsible for expert performance is that both extremes are patently silly, yet once you get away from the extremes, it’s essentially impossible to actually answer the question quantitatively. As you rightly point out, it would be absurd to suppose that talent is all that’s needed to achieve world-class performance in virtually any domain. But it’s also patently obvious to almost everyone (save perhaps Anders Ericsson and Malcolm Gladwell) that no amount of practice will make you a basketball, chess, or violin star if you don’t have some basic aptitude for those things.

    When dealing with normal variation (i.e., apportioning variance in a phenotype across the general population to various genetic vs. environmental sources), we’re in a much better situation, because we can conduct large twin studies. Contrary to what you suggest, there is no glaring problem with twin studies: there are a number of different designs in common use (the most common is to compare identical to fraternal twins rather than twins raised together vs. apart), and concerns about identical twins being raised in more homogenous environments explain at best a small fraction of heritability estimates. The reality is that the vast majority of quantitative traits that have been studies show a high degree of heritability. As a rule of thumb, you won’t go too far wrong if you assume that, in the general population, most personality and cognitive traits are roughly 50% heritable, give or take 20%.

    Note that traits like work ethic and engagement are no exception. When people argue that innate talent is not enough, as you do above, they often forget that hard work, interest, and all of the other things that usually fall under the “you also need to have…” clause are also highly heritable. Of course this is not to deny in any way that attaining world-class expertise requires thousands of hours of practice in most domains. It’s simply to point out that the ability and proclivity to practice thousands of hours is itself in part driven by genetic factors. So, in essence, hard work is to a large extent already bundled into that nebulous “talent” thing people talk about.

    The problem with world-class experts, unfortunately, is that by definition there are only a few of them, so it’s unlikely that anyone will ever be able to estimate the heritability of world-class expertise in any domain (as opposed to variation in the broader population, where sampling is not an issue). Absent quantitative estimates, we’re left with no meaningful way to answer a question like “how much of success is due to talent vs. practice?” Anecdotes are not helpful, unfortunately, because they suffer from exactly the same fundamental indeterminism. In your Von Neumann example, for instance, we don’t actually have any idea to what degree Von Neumann’s work ethic was part and parcel of his genetic endowment (which, effectively, is talent), and what part was acquired via some other channel. It’s theoretically possible that all of his work ethic was due to a lucky combination of genes, and that, barring some major accident, there is literally nothing that would have kept Von Neumann from becoming who he became. Unfortunately, we can’t know, so absent new empirical evidence (and the McNamara et al study is arguably as close as we’ve come to having good quantitative estimates of the heritability of expertise), there’s just not much we can say other than “it’s both, of course”.

  14. @Tal

    The point of my blog post was to argue precisely what you have granted: attaining world-class expertise requires thousands of hours of practice in most domains.

    It does not particularly matter to me whether laziness is an inherited trait. It is a second order issue.

  15. Alejo says:

    Achievement isn’t “normal”
    Very relevant talk
    90 seconds

    https://www.youtube.com/watch?v=q6bNUG7nKHY

  16. @Alejo

    I agree that it is very relevant.

  17. Carson Chow says:

    I don’t think the dispute is that hard work is important. It’s that the number 10,000 hours is not a magic number. Some people could reach world class expertise with fewer hours and some may never reach it no matter how long they train. Mastery requires both some innate ability and lots of practice.

  18. @Carson

    I agree regarding the 10,000 hours. I would be very surprised to learn that anyone took this precise threshold as something important. it is a good round number that can serve as an estimate. (E.g., if you take a Chess master, you might expect that he practiced around 10,000 hours or more).

    As for nobody disputing the role of hard work… I am not so sure. When one writes that practice explains less than 1% of the performance difference, one effectively implies that innate abilities trump everything else.

    I agree with everything else you wrote.