, 6 min read
Why virtual reality (VR) might matter more than you think….
I have heard it claimed that the famous novelist William Gibson uttered his famous quote, “the future is already here “it’s just not very evenly distributed”, for the first time after experiencing virtual reality, decades ago.
We are fast arriving at a point where virtual-reality will be dirt cheap, and it will work really well.
A core issue right now, and that might surprise you, is that most people, including those who have tried virtual-reality goggles, cannot really say what virtual-reality is.
The naive answer is that virtual reality provides immersive three-dimensional world view. That is, when thinking about virtual reality, people think about the display. And they could be excused for doing so by the fact that the physical devices appear to be focus so much on displaying pixels. We have goggles with embedded screens, and so forth.
But, actually, I submit to you that the display is not entirely essential. Of course, you need perception for an experience to make sense, but you could have virtual reality without any light whatsoever. You would probably have to focus on sounds, touch, and smell.
Virtual reality also does not need to be realistic. It is not at all obvious that the more realistic the representation, the better it is. You could have great experiences with a cartoonish worldview. That would side-step the uncanny-valley issue. I actually suspect that some of the best applications of virtual reality will not involve photo-realistic worldviews.
What actually matters with virtual reality is that it engages your whole body. That’s the crucial point. When you use a computer, your fingers (mostly three of them on each hand) do most of the work. I can sit in my campus office working, and because the lights are automated, it might go dark just as I am finishing off a sentence… because I am hardly moving at all when I work in a traditional manner with my computer.
If you were paralyzed, virtual reality would not help you in the least. At a minimum, for virtual reality to make any kind of sense, you must be able to move your head around. It is not so with traditional computing where as long as you can move your arms and use your fingers, your head can be mostly stationary.
I believe that it explains in part how virtual reality affects our perception regarding the flow of time. Virtual reality is somewhat tiring, compared with sitting at a desk, so fifteen minutes of interaction in virtual reality feel (as far as the body is concerned) as tiring as hours sitting at a desk. Thus, time is somewhat accelerated in virtual reality.
But I also theorize that virtual reality affects how you think in a less trivial manner. It favors embodied cognition. An athlete or a chef has a particular type of intelligence where the space around them becomes an extension of their own mind.
It is easy to dismiss such ideas as verging on the mysticism. Yet it is undeniable that we think differently when our bodies are involved. I have now reached a point where I set a clear separating line between in visu meetings and videoconferences. They are drastically different experiences, resulting in very different cognitive outcomes. For example, I believe that it makes no sense to conduct job interviews using video conferencing. And I say this as a nerd that avoids social interactions whenever possible.
That is, the view that we are brains in a jar is hopelessly naive and wrong. The idea that we “think with our brains” is, in my view, only true as a first approximation. There is a continuum between our brain cells and the objects around us. A spider without a web is a useless animal. The spider uses its web as an extension of itself, to measure distances, track directions, and even as a perception device. Human beings do not have physical webs coming out of their hands, but we are simply much more advanced spiders, with the ability to create our own webs, like the world-wide-web.
I believe that many of the paradigm shifts that we have encountered as intellectuals come about through changes that have little to do with pure reason and a lot to do with our bodies and their perception:
- Museums often present very little textual information. Mostly, you get to see, and often touch, artifact. It is through the presentation of inanimate objects that people acquire a feeling of how things were many centuries ago. Try, as an experiment, to view a three-dimensional representation of the object on a screen. It is not the same! The idea that you should collect and display objects to convey information is not entirely trivial, and yet we take it for granted today.
- Though we might credit much of the rise of statistics to the formal mathematical results introduced by famous mathematicians… I believe that we should rather credit authors such as Playfair for introducing the modern-day line graph (in 1786!). If that’s all you had, you could still study effectively inequalities and climate change. But plots are much less rational than it appears: if you were to present line charts to people and ask them to describe what they see, they would have a hard time elaborating beyond a first-level interpretation. And the provided linguistic description would not allow others to understand what was in the graph. There is more in a graph than we can tell. In some sense, it is also easier to lie with statistics than with a plot: try plotting your own weight over the last few months… and compare the result with whatever statistical rationalization you might come up with. Lying with a plot requires a more deliberate attitude. I believe that there is a deeper story to be told about the relationship between the emergence of science and the scientific method: it seems clear that the line graph preceded science. I believed that it might have played an important role.
- The industrial revolution came about after we got to experience automatons, these popular toys from the Victorian era (and earlier) where one could see gears moving underneath. The physical reality of these devices and the fact that you could, as a kid, look at them and eventually hold the gears in your hands, probably made a huge difference.
- The early computers were programmed using plugs and cards… but soon we imported the keyboard into computing… the keyboard is an obvious cognitive extension first created to help us make music more precisely. Without the keyboard we would not have modern-day programming, that much is certain. Isn’t it amazing how we went from musical instrument to software programming?
All of these examples illustrate how altering our environment even in a minute way allowed us to think better.
My theory is that there are entire threads of thoughts, that we cannot have yet, that we cannot even imagine, but that virtual reality will enable.
There are still massive challenges, however. One of them is affordance. For example, many virtual-reality games and systems use the concept of “teleportation” to move you from one point to another. In my view, this is deeply wrong: it uses your hand as a pointing device, just as you would do in conventional computing. Grabbing and moving objects, interacting with objects in general, is awkward in virtual reality. I don’t think we know how to enter text in virtual reality. There is also a bandwidth issue. The screens of current virtual-reality goggles have a relatively low resolution which makes reading small fonts difficult, and reading in general is unpleasant. Interactions are also at a relatively large scale: you cannot use fine motor control to flip a small switch. Everything has to be large and clunky.
Still. I think that chances are good that new world-changing paradigms are made possible by virtual reality. It should allow us to build better webs as the spiders that we are.