Daniel Lemire's blog

, 1 min read

Information is blood, coverage is intelligence

We need to learn the lesson of the Web: coverage is more important than accuracy. We only think accuracy and precision are very important, because we fool ourselves into thinking that our brains are accurate and precise tools. Your brain is provably sloppy. It is terrible at remembering objectively what happened. But human brains are the best computers in the world as far as coverage goes: I can play chess, program a computer, love my wife, and take care of my children. Views and object-oriented abstractions have taught people to minimize the amount of data available to end-users. Security is important, but current approaches are misguided. Information flow is like blood: you want to prevent it from leaking out, but if you limit the flow, you get sick and may die. We need to handle bad data more carefully: your body careful picks what it discards. When your arm hurts, you do not shed it off. To a large extend, databases are managed by overzealous surgeons.

Business Intelligence is not the art of drilling down repeatedly on 5% of a company’s data. It should be about making sense of all of the data. And not just the clean data.