, 2 min read
Theory lags practice
Schools teach us theory so that we can be more productive workers. You learn grammar so that you can be a better writer. You learn about computer science, so that you may be a better programmer. You learn about electromagnetism so you can do electronics. You learn about thermodynamics so you can design engines.
It is tempting therefore to believe that theory precedes practice. We invented grammar first, and then the written language. We came up with the thermodynamics and then we invented engines. We discovered electromagnetism and then we could build electric circuits. We studied computer science and then we started programming.
But this is backward. Watt was certainly a knowledgeable technician when he designed the engine. Flowers was a skilled electrical engineer when he computerized the post office in the 1930s. However, clean principles only emerge after the fact.
There is a big difference between putting theory in practice, and trying something out while lacking the theory. My claim is that most inventions are of the latter sort, even if academics much prefer to imagine that the former plays a central role.
I believe that Torvalds put it best:
Don’t ever make the mistake [of thinking] that you can design something better than what you get from ruthless massively parallel trial-and-error with a feedback cycle. That’s giving your intelligence much too much credit. (Linus Torvalds)
As you invent something new, the existing theory is lacking. The principles do not quite apply. It is only after the fact that someone smart can finally derive clear principles.
The real-world is a complicated place. Our minds are limited to relatively simple models. We evolved as tinkerers. Sure, we are smart tinkerer in that we can trade ideas and designs, but our fundamental R&D strategy remains tinkering.
That is why we invented the written language long before scholars wrote grammars. That is why Watt invented the engine long before scientists conceived thermodynamics. We built electric circuits before scientists founded electromagnetism. We hacked computers together and then founded computer science. We toyed with uranium (and got sick) long before we could build an atomic bomb.
There is an important practical implication to my claim: if you want to invent something new, you need to be willing to go where our theory no longer applies. Sadly, this means you will get it wrong at times through no fault of your own.
Further reading: Problem solvers and theory builders by John D. Cook, quoting from Mathematics without Apologies by Michael Harris.
Update: Yoshua Bengio (U. Montréal) wrote on Facebook:
In my field of research (machine learning, and especially deep learning & neural nets), [theory lags practice] is a truth I have experienced first-hand.