Daniel Lemire's blog

, 2 min read

To solve hard problems, you need to use bricolage

People who think that they can design efficient solutions in the abstract, effectively believe in Oracles. That is, they somehow believe that from their desk, and using only their mind, they can anticipate all the implementation issues that will come up after hours of programming. They somehow believe that before they even start building the software, they can know everything there is to know about a practical problem.

In a talk about a neat software component he designed, Bruce Haddon observed that there is no way that the final structure and algorithmic behavior of this component could have been predicted, designed, or otherwise anticipated. Haddon observed that computer science serves as a source of core ideas: it provides the data structures and algorithms that are the building blocks. Meanwhile, he views software engineering as a useful set of methods to help design reliable software without losing your mind. Yet he points out that neither captures the whole experience.

That’s because much of the work is what Haddon calls hacking, but what others would call bricolage. Simply put, there is much trial and error: we put ideas to together and see where it goes.

It is common to be dismissive of bricolage (or hacking). However, I think it is a grave mistake to discourage it.

It is common that my students will ask me “how do I do X?” and my answer is “try something, anything.” Though I don’t put it in words, I am encouraging them to use bricolage. I may need to urge them 2 or 3 times before they try it. And you know what happens? Often the student actually solves the problem!

I believe that a common process is as follows:

  • You are given a problem. You lack the necessary information to solve it. For example, maybe you are in Europe and you want to go to India but you have no map.- You could try to build a simplified model of the problem and solve that. For example, you might decide that the world is round and that you simply have to sail West. There is no harm done unless hubris takes over and you conclude hastily that the problem is effectively solved. Something you did not know that you did not know might get in the way (e.g., there is an extra continent called America in your way).- You could try something, anything. Chances are that it will fail. If it does, chances are that you will learn something. Something that might not have been obvious. For example, you may decide to go South along the coast of Africa. In the process, you may discover cities ripe for plunder.

Most problems become reasonably easy once you have all the relevant information. The really difficult problems are such that you lack critical information. And, hence, almost all hard problems require bricolage.