, 6 min read
Enough with the bogus medical studies!
Every week, we hear about how eating such and such food gives cancer, or how working out can save you from a heart attack. If you have been reading these studies with attention and changing your life to follow their recommendations… you have been wasting your time and you have possibly harmed yourself.
Feynman famously described a common phenomenon which he called cargo-cult science. True science is a fantastic tool that has taken us, in a couple of centuries, from superstitious fools to a highly advanced civilization. Almost all major advances in the last two centuries can be linked with science. So people want to appear to be doing good science… but they are much less concerned about actually doing good science, when they even know what it is.
What is science? As Feynman wrote, it is the belief in the ignorance of experts. It is the idea that our intuition, our good sense… are not enough to tackle the world. We need objective measures. We often need mathematics. We often need hard logic. We need repeatable experiments capable of falsifying theories.
We need these things because we are easily fooled. Science acts as some sort of “software update” for the brain. You train your brain to rely on objective, verifiable facts, and hard logic… instead of impressions. One of the core ideas behind science is that authority-based arguments should be rejected. So if Professor X from Harvard University or MIT says something… it is not scientific to believe her because she is from Harvard, or because she is a professor. It is similarly unscientific to believe something because it has appeared in a prestigious journal. It is not even scientific to believe something because of a “consensus”. (Hint: there was a widespread consensus that the aeroplane was “impossible” before the Wright brothers came along.)
Let me take this further. Believing something because impressive people say it must be so is the illustration of pre-scientific thought. And this means that every time you hear that a study by impressive people proved something new, you should doubt… you should doubt automatically, without even thinking.
How do we determine what is true? That’s one of the core lessons of the scientific revolution: it is much harder than you might expect. We are feeble creatures that very easily fall prey to bogus arguments. We must relentlessly seek incontrovertible evidence. There is no shortcut.
The real question is not how to make your mind about what is true… that is easy, your brain will eagerly accept many things as true given a chance. The hard part is to keep on doubting. We are just not wired for constant doubt, we are “believers”.
Many people do not really understand what science is… and this includes fully trained “scientists” who receive government research grants and train other scientists. What they do understand, however, is how to speak and act like a scientist. It looks like science, but it does not give the same result. That’s what Feynman calls cargo-cult science. When it comes to medicine, we invest a lot of money in medical research. Several times more than any other type of research. Secondly, lots of real people can end up suffering needlessly if the research is wrong.
Is medical research wrong? Yes. But that’s not the most tragic part. The real problem is that much of it is pure cargo-cult science. And well funded at that. Cargo-cult science does not merely lead to the wrong ideas… it actively sets us back.
What are the problems?
- Most work is based on observational studies
Just watching and taking notes is not science. It is valid scholarship, as long as you report it as such. But it gets reported as “science” instead which is misleading.
There is nothing wrong with historians who tell us that Vikings once traveled to North America. It is interesting. It is scholarship. But it is not science. It is an observation. Don’t get me wrong. Scholarship is precious. We know, for example, that women live longer than men. Cataloging such facts can help us… but it is not science. Our ancestors knew exactly where the stars were. They knew a lot about the seasons. Their high priests observed, cataloged… but they did not know about science.
When you hear that exercise has been shown to reduce your risk of cancer… that’s (usually) a flat out lie. What actually happens is that scientists take a group of people. Then they ask them whether they exercise. Then they ask whether they have cancer. And then they run the statistics. “Oh! See, the people who say they exercise also less often say that they have cancer.” So far so good, but you have merely observed, you have not done any science. You have done nothing that is above what scholars routinely did in the pre-scientific era.
All you can show with observational studies is an observation. Nothing else. It can be a lot, but it is not science.
It could be that when Joe complains about his knee, it rains. That does not show that complaining about one’s knee brings the rain. Nor does it show that the rain brings pain to one’s knee. It is an observation, not science.
- Small effects are greatly exagerated
Even when scientists actually do science… they do not use the proper tools to avoid being fooled or fooling others. In fact, they often appear to seek to fool us.
Suppose that you are treating patients for cancer. The average survival rate is 9 months. All patients die within 3 years. You give them some pill and the survival rate increases to 10 months, but all patients still die within 3 years. It is progress, but a small progress. If the pill costs 100k$ and makes you twist in pain for 3 weeks, maybe it is no progress at all.
However, it is not how most studies will present it. They will choose a short window. Say a year. Maybe 60% of all patients die within a year. However, with the extra month offered by the pill, only 40% of all patients die within the year. Then the study will conclude that the survival rate has been increased by 50%. I kid you not.- They report selective effects
Without even looking at the numbers, I can tell you that people with high-risk occupations have a lower cancer mortality. Why? Because they die younger. So, there you go… I got a cancer prevention technique: choose a high-risk occupation. Of course, this reasoning is silly. But that’s often how medical studies are presented.
So how to keep cancer free? You will hear things like “eating well, exercise, keeping your weight low, eating broccoli, avoiding red meat, getting tested repeatedly”. But if you actually look at the numbers, mostly, there are two factors that have an effect worth noticing on your risk of having cancer: your age and whether you smoke. Your risk of having cancer grows up exponentially with age. If eating bacon increases your cancer risk, it is by a tiny amount. What about early diagnostic? Again, a nice idea, but, in general, it is a waste of time to get regularly tested for cancer. You won’t live an extra day.
What about heart attacks? Age and gender are basically the things you need to watch, assuming you are not smoking. Again, your risk grows up exponentially with age. Also, men are three times more likely to have a heart attack than women. Why? We don’t know. Smoking is often considered like a really bad habit. It is true that it doubles your risk of cardiovascular diseases. However, aging from 40 to 90 multiplies your risk of cardiovascular diseases by 100.
So, how do you stay healthy? By staying young. Sadly, we don’t know how to do prevent aging. Yet. Demonizing hamburgers is a dangerous distraction.
Note: I am not a medical professional. Consult your doctor if you have questions. While you are at it, ask her if she knows about cargo-cult science.