Daniel Lemire's blog

Is software a neutral agent?

, 5 min read

We face an embarrassing amount of information but when we feel overwhelmed, as Clay Shirky said, “It’s not information overload. It’s filter failure.” Unavoidably, we rely heavily on recommender systems as filters. Email clients increasingly help you differentiate the important…

We know a lot less than we think, especially about the future.

, 3 min read

The inventors of the airplane, the Wright brothers, had little formal education (3 and 4 years of high school respectively). They were not engineers. They were not scientists. They ran a bicycle repair shop. At the time of their invention, there was quite a bit of doubt as to whether airplanes were…

How will you die? Cancer, Alzheimer´s, Stroke?

, 6 min read

Before the 1950s, many of us suffered from poliomyelitis and too many ended up crippled. Then we developed a vaccine and eradicated the disease. Before the second world war, many people, even the richest, could die of a simple foot infection. Then we mass-produced antibiotics and got rid of the…

The powerful hacker culture

, 3 min read

In my post the hacker culture is winning, I observed that the subculture developed in the software industry is infecting the wider world. One such visible culture shift is the concept of “version update”. In the industrial era, companies would design a phone, produce it and ship it. There might…

No more leaks with sanitize flags in gcc and clang

, 2 min read

If you are programming in C and C++, you are probably wasting at least some of your time hunting down memory problems. Maybe you allocated memory and forgot to free it later. A whole industry of tools has been built to help us trace and solve these problems. On Linux and MacOS, the state-of-the-art…

How close are AI systems to human-level intelligence? The Allen AI challenge.

, 6 min read

With respect to artificial intelligence, some people are squarely in the “optimist” camp, believing that we are “nearly there” as far as producing human-level intelligence. Microsoft co-founder’s Paul Allen has been somewhat more prudent: While we have learned a great deal about how to…

Narrative illusions

, 2 min read

Our brain contains lots of neurons and can do great things. I can read, write and speak fluently in two languages made of tens of thousands of words. Millions of human beings can do that, and much more. But our brains have also clear limits. For example, despite the fact that I have advanced…

Being shallow is rational

, 4 min read

Pundits often lament who people have become shallow. They no longer sit down to read books cover from cover. Instead of writing thoughtful 2-page emails, they write a single line. Sometimes they do not even write it themselves, as they often delegate to an artificial intelligence like Google’s…

Could virtual-reality make us smarter?

, 2 min read

When the web initially took off, there were major concerns that it was “dumbing us down”. There are similar concerns with e-books making us dumber. I am quite sure that when we first started to use the written word, there were related concerns: “not having to remember all of your thoughts…

Setting up a “robust” Minecraft server (Java Edition) on a Raspberry Pi

, 22 min read

My kids are gamers, and they love Minecraft. Minecraft sells its client software, but the server software is freely available. Since it is written in Java, it can run easily on Linux. Meanwhile, you can order neat little Raspberry Pi Linux computers for less than $50. So, putting two and two…

What thirty years of technology looks like: the early high-school years

, 5 min read

Thirty years ago, I was attending a high school. My oldest son is preparing to attend a similar high school. There was a meeting at my son’s future school, and I paid attention to how things changed. We went to visit the school and hear the principal give a talk. I would say that a good half,…

Declarative programming is a dead-end for the lowly programmer

, 3 min read

Most programmers focus on software execution. We want to understand what the computer is actually doing. Java, C, JavaScript, PHP, Python… all these languages make it easy to build any software you like. In these languages, even at the very beginning, you are truly in charge. They are like…

What the heck is interesting research?

, 1 min read

I have long advocated that the world would be better off if more people did research. But what the heck is research? In simple terms, I know of two types of interesting research… The engineering path: Pick an unsolved problem that has some importance. Propose a viable solution. For example, no…

Ranged random-number generation is slow in Python…

, 2 min read

A colleague has been running simulations using a library written in Python. She was having serious performance problems… Her application is parallelizable, but Python does not make parallelization easy. She could switch to another language, but that’s expensive. Further investigations reveal…

Biology and computing are more alike than you think…

, 2 min read

Biological systems are seemingly of infinite complexity. We still don’t know what a heart attack is, not really. Despite billions spent, we still don’t know what Alzheimer’s is. We don’t what aging is, not really. We don’t know how genes work, we can’t reprogram you at will. We study…

Artificial intelligence is mostly a matter of engineering?

, 3 min read

Unless you live under a rock, you should know by now that AlphaGo, an artificial intelligence, has beaten a world champion at the game of Go. After Tic Tac Toe, Checker and Chess, Go was the last conventional board games where computers could not beat the best human beings. We have made…

Web plasticity

, 4 min read

When the Internet started out, people wanted to use it to chat and discuss. So we did as human beings had always done. We created tribes. In this manner, we got newsgroups, posting boards and so forth. This worked very well at first. But just like real-world tribes, Internet groups degenerated into…

It is easy to lose sight of why we do things…

, 4 min read

I like to write and publish research papers. I think it can be tremendously useful. A research paper serves as a reference describing what was done and what was achieved. Others can learn from these records, verify them, and build upon it. It turns out that the form is less important than the…

Statistics is overrated: the rise of data science

, 2 min read

With the industrial and scientific revolutions, we saw the rise of enormous bureaucracies collecting reliable numbers. For the first time in history, we could ask about the total production of silver in England and get a meaningful answer. But data is rarely complete. We most often only have…