Daniel Lemire's blog

, 29 min read

Old people are not very sharp, are they?

19 thoughts on “Old people are not very sharp, are they?”

  1. Anonymous says:

    Have you just called managers stupid? 😉

  2. int says:

    This is good point. For now I don’t worry about that, because my goal in life is to solve every problem in projecteuler.net:)

  3. John Cook says:

    Along these lines, I recommend David Snowdon’s book “Aging with Grace.” It’s about half science, half human interest.

    It’s about a study of Alzheimer’s disease in nuns. Turns out nuns are great for epidemiology studies because a lot of confounding variables have been eliminated — they all live in very similar circumstances — and they keep very good records.

  4. @Anonymous

    I think that it can be difficult to come back to research or programming after having been a manager for too long.

  5. Paul says:

    The converse also seems true, that too much research and programming gets you out of the managing zone. When I’ve had a couple weeks of deep, uninterrupted coding at work, I’m always amazed at how tongue tied I’ve become after.

  6. gwern says:

    > For example, we know that retirement significantly degrades your cognitive functions.

    Retirement is pretty heavily confounded, don’t you think?

    If cognitive activity really made a huge difference to fluid intelligence, then we would see in the scads of cognitive training experiments in the elderly strong evidence for universal improvement across cognitive domains – the training tasks like WM training would improve not just WM but math skills, Gf, verbal fluency, etc.

    Instead, what we tend to see is the same thing as in the n-back studies: inconsistent gains, mostly on the trained tasks, no proof of gains on the latent variables one actually cares about, and – shades of stereotype threat! – considerable evidence that passive control groups inflates the score improvements. (Response bias is a sword which cuts many ways.)

    The experiments have been run, and they are not reassuring.

    > Old people convinced that they have rotten brains should not be expected to be sharp…

    Old people with rotten brains should not be expected to be convinced their brains should be sharp.

  7. Studies show (no pun intended) that brain doesn’t age much. In fact, as you mention here, some loss in speed is well compensated by experience. For example, I have heard that in verbal tests older people are doing better.

    However, this seems to be only true for people who strain their brains. Again, somewhat surprisingly, it seems that a workout is probably an essential exercises for the brain.

    To summarize: to prevent early cognitive decline, one must think hard and work out hard as well.

  8. @gwern

    Retirement is pretty heavily confounded, don’t you think?

    And the researchers can account for that, naturally.

    There is a fair amount of work dating from way back showing the negative causal effects of retirement.

    the training tasks like WM training would improve not just WM but math skills, Gf, verbal fluency, etc.

    Right. So people who do word puzzles to stay sharp might very well be wasting their time, except maybe if they want to get better at word puzzles.

    However, it does not follow that intellectual activity does not matter. It very much does, I believe.

    That is why you should never use cross-sectional comparisons. You need longitudinal studies.

    It is totally silly to take hundreds of people at one time and benchmark them… the old people are probably nothing like the young people will be (e.g., we tend to be much more intellectually stimulated than we were decades ago).

  9. gwern says:

    > And the researchers can account for that, naturally.

    They may be able to exploit some natural experiments, sure.

    > There is a fair amount of work dating from way back showing the negative causal effects of retirement.

    So maybe you should be citing that to make your case?

    > However, it does not follow that intellectual activity does not matter. It very much does, I believe.

    If intellectual activity mattes, why do the experiments not seem to show this? Given that people cannot be assigned to R&D positions, what could be more direct, more externally and ecologically valid, than taking a sample of seniors and having them do suites of demanding tasks like Cogmed?

    > That is why you should never use cross-sectional comparisons. You need longitudinal studies.

    The longitudinal studies have their own problems; specifically, most of these tests have practice-effects where doing a long session for researchers means years later you still do it better than if you had never done it before (similar to how people increase their scores by a question or so on matrix IQ tests from pre-test to post-test, regardless of whether the intervention worked). See “Influence of Age on Practice Effects in Longitudinal Neurocognitive Change” http://www.apa.org/pubs/journals/releases/neu-24-5-563.pdf , Salthouse 2010. (Extracting latent variables is also important for checking this part: in psychology, what you measure at the beginning is only sometimes what you measure at the end; interventions trying to increase intelligence very often fall prey to this problem.) And the longitudinal studies (like Schaie’s _Intellectual Development in Adulthood: The Seattle Longitudinal Study_) still show considerable decline in WM and other forms of speed.

    (The usual exception is verbal skills, but I’m not sure that’s necessarily a good thing for people trying to do science or research in general: is it really good to still be glib and eloquent even as your actual ability to think and understand and learn falls off a cliff? When I first began looking at the charts showing performance decreases/increases over age, I wondered if this explained why some scientists and physicists especially seem to embrace lunatic ideas in their dotage.)

  10. @gwern

    So maybe you should be citing that to make your case?

    “We investigate the effect of retirement on cognition empirically using cross-nationally comparable surveys of older persons in the United States, England, and 11 European countries in 2004. We find that early retirement has a significant negative impact on the cognitive ability of people in their early 60s that is both quantitatively important and causal. ”

    http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2958696/

    The human capital framework suggests that retirement may cause an increase in cognitive decline, since after retirement individuals lose the market incentive to invest in cognitive repair activities. Our empirical results, based on an instrumental variable strategy to deal with the potential endogeneity of retirement, confirm this key prediction.

    http://www.sciencedirect.com/science/article/pii/S0014292112000463

  11. @gwern

    If intellectual activity mattes, why do the experiments not seem to show this? Given that people cannot be assigned to R&D positions, what could be more direct, more externally and ecologically valid, than taking a sample of seniors and having them do suites of demanding tasks like Cogmed?

    You mean compare people who sit at home all day with people who go to the office? Well, yes, we have evidence that going the office will keep you sharper.

    The experiments you allude to are training the working memory on the hypothesis that this will improve the brain. As you well know, this does not work.

    I personally fail to see the appeal in working memory training. I am not at all surprised that it fails.

    You seem to infer from this very specific failure that you could put someone in a box for all his life, and he would just be as mentally sharp as someone who spend his life at Harvard.

    Surely, you can see that is a wide gap in your argument.

    “This one very specific form of mental training does not help. Therefore, not using my brain at all is just fine.”

  12. gwern says:

    > The experiments you allude to are training the working memory on the hypothesis that this will improve the brain. As you well know, this does not work.

    No. The relevant experiments here go far beyond simple WM exercises – when I say suites, I mean suites. Verbal, reaction time, simple choice, complex choice, simple WM, complex WM, etc. Cogmed, Lumosity et al do a *lot* of things.

    (This diversity is a serious problem but for a different reason: because research groups change the training paradigm from study to study by the same group or even just invent their own new variant, the literature is a *complete* mess and difficult to meta-analyze, and worse, it means researchers will never ever admit that the null might be true; they’ll simply say that ‘maybe task X didn’t work, but next time we’ll try task X with adaptive difficulty! Or combined with task Y! and our post hoc analysis shows that task X helped poor Hispanic school-children who performed worst on the pre-test so it’s still evidence for cognitive training’.)

    > You seem to infer from this very specific failure that you could put someone in a box for all his life, and he would just be as mentally sharp as someone who spend his life at Harvard.

    I would say that people who are not mentally sharp will be pushed into or seek out a box, and vice versa for the Harvard; whether the immediate causal effect of Harvard is a large enough effect size to be worth talking about, I wouldn’t know. Remember, the heritability of intelligence *increases* with age, which is best explained by self-selection of environment.

    > http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2958696/

    I was expecting something like a regression-discontinuity in which a national pension plan instantaneously increased the retirement age, but I’m not sure what this is… This paper claims to have an IV but I don’t see how a cross-country regression of official retirement age vs performance remotely qualifies as an IV. Populations do not have the same cognitive performance levels and you wouldn’t expect them to have the same cognitive performance trends nor the same health trends (just compare longevity across countries…) nor would you expect the job markets to treat people identically (does Spain have the exact same need for highly cognitive professions as the UK?), and you would not expect retirement ages to be set at random independent of how people age – in fact, if retirement ages *were* set at random, that would be serious malpractice on the part of the legislators and actuaries, who are not taking into account factors like the physical deterioration of their country’s old people.

    > http://www.sciencedirect.com/science/article/pii/S0014292112000463

    Fulltext: http://www.ire.eco.usi.ch/personale/paper-mazzonna-181897.pdf This is, you will note, based on a subset of the data the above paper is based on, and seems to be much the same. (Also note the small effect sizes.) As natural experiments or IV analyses go, these are not particularly convincing, as they acknowledge.

    > “This one very specific form of mental training does not help. Therefore, not using my brain at all is just fine.”

    It would help if one could show anything worked.

  13. @gwern

    Let us see. The brain appears to behave like a database of sort… it keeps storing more information and it has progressively trouble recovering the information as fast, or at all in some instances.

    People also move slower, get poorer perception… if they are like me, they lose their hair.

    But then, they keep getting better, or, at least, they do not get worse in things that matter… like academic research : you cite
    Timothy Salthouse… this man does not look young (he got his PhD in 1974). I mean, I read some of his work, and it sounds just as good as what younger folks produce. (And yes, full professors are definitively often more productive than assistant professors…)

    Old people also get better (or not worse) in important linguistic tasks, and in world knowledge.

    They are slower at working out some reasoning problems, maybe because they have to sort out over more memories, slower at recalling some memories… again maybe because they have many more…

    And then you conclude that the brain is just rotting away.

    Yes. Older people are slower. They are more forgetful. They are probably a bit more easily confused…

    But from these observations, I come to different conclusions…

  14. @gwern

    There are many other, older, studies on the adverse effects of retirement. Depression rates increase. Social network decreases. If you claim that retirement does not affect negatively one’s intellectual life, I feel that you have the burden of the proof.

    Retirement has some benefits, but it has some clearly defined downsides.

    It would help if one could show anything worked.

    I think that the absence of strong transfer effects is an orthogonal issue.

    If you keep doing mathematics 8 hours a day, you will keep on being good. Granted, when you reach 80, you might be more easily confused and slower… but if you have done math. all your life, you will still be quite good, objectively speaking.

    You might suck at playing Go however. Well, if you play Go every day up until you are 80, you will be good at Go… though maybe not at math…

    However, if you stop doing any kind of intellectual work at 40, chances are good that you will be poor at mathematics when you are 80. You might also be poor at Go.

  15. gwern says:

    > But then, they keep getting better, or, at least, they do not get worse in things that matter…

    Maybe you should *look* at the graphs in the papers you linked to get an idea of the magnitude of the losses with time. These are not small cognitive losses like the estimates for retirement. These are multiple-standard-deviations even on very easy questions (look at the numeracy questions included in the second link, and think about how much loss there must have been for there to be any variation from person to person rather than a massive ceiling effect where the average score is ~100%).

    > like academic research : you cite Timothy Salthouse… this man does not look young (he got his PhD in 1974). I mean, I read some of his work, and it sounds just as good as what younger folks produce. (And yes, full professors are definitively often more productive than assistant professors…)

    Research productivity tends to be highest in relative youth; it varies from field to field but I think the highest is medicine in the 40s. Nowhere in the 50s or 60s. See the papers by Simonton, Jones, Stroebe and other people who’ve looked into that.
    (As for Salthouse, who knows? He may be a lucky exception, he may be more of a coauthor reviewing his younger colleagues’ work, he may be stuck in a rut applying the same techniques to the same ol’ problems, etc.)

    > Old people also get better (or not worse) in important linguistic tasks, and in world knowledge.

    Those exceptions hardly balance out all the other losses… Words aren’t that important, or I should say, for people trying to still contribute to research etc, words are much less important than intelligence. Being silvertongued won’t make any important discoveries. And the ‘world knowledge’ here, IIRC, is closer to Jeopardy trivia questions than something that really matters.

    > They are slower at working out some reasoning problems, maybe because they have to sort out over more memories, slower at recalling some memories… again maybe because they have many more…

    What memories do you need to recall or sort through, exactly, when trying to (and increasingly for the elderly, failing to) solve a Raven-style matrix problem which is just some squares, triangles, and circles on a 3×3 grid?

    > And then you conclude that the brain is just rotting away.

    Look at neuronal counts by age. Look at brain imaging studies about white-matter loss. The losses are visible at every level you want to look at, from personality factors like Openness to cognitive performance on classic psychology tests to low-level neurobiological data.

    (I’m not happy about this either – I’m going to get old too.)

    > There are many other, older, studies on the adverse effects of retirement. Depression rates increase. Social network decreases.

    Yeah, maybe. Not clear how much that matters. If they mattered all that much to cognition, then they should show up.

    > If you claim that retirement does not affect negatively one’s intellectual life, I feel that you have the burden of the proof.

    The burden has been met by your own citations; you presumably read them, right? You read the background sections where the reverse causality arguments were laid out, you looked at the graphs, you saw how small the best models’ coefficients were?

    > However, if you stop doing any kind of intellectual work at 40, chances are good that you will be poor at mathematics when you are 80. You might also be poor at Go.

    This seems like a triviality, and seems to have little to do with your core arguments which are, as far as I can tell, that old people do not suffer severe cognitive declines and to the extent that they do, it is causally the fault of bad beliefs & poor life-choices.

  16. @gwern

    Research productivity tends to be highest in relative youth; it varies from field to field but I think the highest is medicine in the 40s. Nowhere in the 50s or 60s.

    In Physics the mean age of Nobel Prize winning achievements since 1980 is 48 years old. (http://www.pnas.org/content/108/47/18910.full)

    If we focus on post 1965 results, the mean age of a great discovery or invention appears to be at around 40 and it is roughly a bell curve (Jones, Age of great invention, 2010). It does fall to zero at around 65, but it also roughly zero at 18.

    This seems like a triviality, and seems to have little to do with your core arguments which are, as far as I can tell, that old people do not suffer severe cognitive declines and to the extent that they do, it is causally the fault of bad beliefs & poor life-choices.

    I do not think I wrote anything so drastic.

    It makes a whole lot of sense for people to stop investing effort in staying intellectually in the best possible shape. I would not call it “poor choices”.

    In fact, I might very well decide to retire early from intense professional activities and move into a more relaxed life style. Nothing wrong with that.

  17. gwern says:

    > In Physics the mean age of Nobel Prize winning achievements since 1980 is 48 years old. (http://www.pnas.org/content/108/47/18910.full)

    And is lower if you don’t choose that subgroup or field.

    > If we focus on post 1965 results, the mean age of a great discovery or invention appears to be at around 40 and it is roughly a bell curve (Jones, Age of great invention, 2010). It does fall to zero at around 65, but it also roughly zero at 18.

    Discovery can be expected to be a multi-factor thing: intelligence and knowledge and time to achieve it. Two blades of a scissor. I think Jones is uncertain whether the increasing age of achievement reflects the increasing amount of material which must be studied to make a discovery or whether it reflects institutional factors (ever longer PhD degrees, time wasted on grants and seeking ever scarcer positions and grants, etc); if it’s the former, it’s a little disturbing because that implies that there could be a sort of Scientific Stagnation Singularity, where it takes so much time to reach the research frontier that by the time people have mastered all the prerequisites and gained some experience, aging has robbed them of much of the ability to make any major new discoveries or paradigm-shifts. I think I estimated it at I do not think I wrote anything so drastic.

    Your entire post is loaded with language attempting to minimize any biological change with age, minimize any measurement of decreased performance (such as by claiming cross-sectional studies are biased due to cohort differences, and longitudinal are better, although in part one is just trading biases there), and explain away such decreases as due to retirement or stereotype-threat-like effects. Some excerpts:

    “Of course, we know that our brains incur *some* damage over time, so some decline of some of our abilities appears *likely*. However, it is *probably* not as simple as “we lose brain cells over time”… And we can compensate in many ways for a *moderate* decline… In effect, if the hardware gets *slightly* slower, we can compensate with better software, and with new peripherals. However, my belief is that a *good* share of the age-related cognitive decline is psychological, or caused by cognitive disuse. For example, we know that retirement *significantly* degrades your cognitive functions…Of course, it stands to reason that if retirement can have a *large* effect, so can other similar life style changes….We know that this effect is real and *strong*. In fact, *the effect is so strong* that removing the stereotype threat can be enough to eliminate age-related differences in specific experiments:…*even the slightest hint* that you are attempting to measure a decline in their cognitive functions could ensure that you *will indeed measure a strong decline*…Entertaining the idea that you are getting dumber might just be *a self-fulfilling prophecy*.”

    In reality, the decreases with time are large, real, due to measurable biological damage (sometimes of grotesque magnitude; look at brain volume and neuron count with age), the retirement effect is mostly reverse causation and the direct causal effect is both unlikely and of small magnitude, the intellectual activity thesis is largely ruled out by the failure of cognitive training programmes to improve the targeted latent variables, changes with age are most certainly not due to self-fulfilling prophecies, and I didn’t want to get into it but priming studies are the dregs of social psychology and the least likely to replicate (as shown by the recent Reproducibility Project results) and stereotype threat in particular has huge problems (publication bias, and experimenter demand effects in no-stakes tests).

  18. @gwern

    Your entire post is loaded with language attempting to minimize any biological change with age, minimize any measurement of decreased performance (such as by claiming cross-sectional studies are biased due to cohort differences, and longitudinal are better, although in part one is just trading biases there), and explain away such decreases as due to retirement or stereotype-threat-like effects.

    The latter two are in my blog post, but the cross-sectional criticism is not part of my blog post. Maybe you got confused. How old are you? 😉

    In reality, the decreases with time are large, real, due to measurable biological damage (sometimes of grotesque magnitude; (…)

    I do not have any problem with this scientific point of view, as you stated it.

    However, my impression from working with much older colleagues is that many make it to a relatively advanced age while remaining remarkably smart and clever.

    I do not observe this “damage of grotesque magnitude” in others. And I still feel “smart enough” myself. If I did observe problems, I would surely militate for compulsory early retirement. Instead, I do the opposite. I view many of my older colleagues as contributing just as much or more than the younger ones.

    Of course, I would not, say, want to serve in the military at 60. But I would feel quite at ease teaching at 70.

  19. gwern says:

    > many make it to a relatively advanced age while remaining remarkably smart and clever.

    One would hope that they know when to retire, yes.

    > I do not observe this “damage of grotesque magnitude” in others. And I still feel “smart enough” myself.

    The people in question could be fortunate enough to be declining slower, and of course I imagine they are starting off from unusually high levels of ability in the first place.

    But also, you would not necessarily observe or feel this for a long time since introspection is notoriously error-prone and unreliable; people often cannot detect even large decreases in cognitive performance (eg chronic sleep deprivation has this problem – if people *felt* anywhere near as badly as they *performed*…) As I’ve heard a number of middle-aged people who used to enjoy computer/video games remark, of kids’ play: ‘watching this game makes me feel old’.