In almost every sci-fi movie worth re-watching, it seems that us humans are always less technologically advanced, dumber and only serve as a mere speed bump into an alien race eliminating humans to take over our planet and suck Earth dry of its resources. We’re always the weaker ones in alien wars. Well, what if we’re not? Tom Scott imagined a scenario where everyone else in the universe was afraid of humans. It’s fantastic.
In a previous post I wrote about how an immersive virtual reality (IVR) system would have to work and how many portrayals in popular television and movies get it wrong. I established that such a system would necessarily be invasive, having to connect itself to every neuron in the peripheral nervous system. Warning: What follows is a rather technical discussion rife with medical and engineering terms.
The crude cables shown in The Matrix, though definitely invasive, do not begin to capture it. To reiterate, every retina cell, every cochlear hair cell, every olfactory sensory neuron, every proprioceptive (joint position) neuron, and every other neuron in peripheral nervous system would have to be freely modulated by such a system. Every channel of sensation would have to be controlled. Anatomically, each of these channels is kept separate until it is highly processed by the brain (separately), and only after each are processed (separately) do specialized areas of the brain integrate them. As a consequence of the channels being processed in parallel, there is no one convergent area that could be targeted or manipulated for the purposes of an IVR system. Thus it has to be the complete peripheral nervous system, i.e. all the neurons comprising the 24 cranial nerves and 62 spinal nerves.
How might this be accomplished? My own idea, while still fanciful and well beyond current technologies, does have a certain conceptual soundness to it. That is, it would work in theory. I call it a peripheral nervous system graft. Rather than a surgically-installed device, the PNS graft would sprout and grow into the body as a synthetic organism. I imagine an genetically-engineered virus that would ‘infect’ peripheral neurons selectively and transform them into syneurons. The syneurons would function identically to and maintain the same connectivity as the cell from which they were born, but would bud and form new connections with a central hub. As the central processor of the device, the hub would implement the control loop that I described previously. I picture it residing somewhere in the abdomen.
Thus, sensory input from the real world would be blocked and simulated reality from the hub would be fed in instead. Likewise, motor output from the brain would be captured and applied to the simulated environment (to a virtual body) and the proxy software would generate whatever motor output it was programmed to do for the real body.
Imagine hanging out with your friends in a virtual club while your proxy does a work-out routine with your body in the real world. You could have virtual feasts on simulated food and return to the real world with an empty stomach. Sound good? Here is where one’s imagination can take off.
There is much more to say about devices such as these, but what is particularly interesting to me are the ethical, moral, and societal implications. My novel delves into just a few of them. It is true that, while theoretically sound, the above is just speculation on my part today, but as we see increased use of mobile computing (smart phones and tablets) and augmented-reality devices (Google Glass), we will begin to grapple with many of these issues.
Last Wednesday I discovered how easy it is to sell a paperback with a print-on-demand service, so I formatted my manuscript and cover and submitted it to CreateSpace. This morning (Tuesday) I just got my proof copy in the mail! The quality is really good, indistinguishable from a paperback sold at a regular book store. One of the best parts is that there was no money up front except for the proof copy itself. Once I modify and approve it, I can have it up for sale on Amazon within like 48 hours.
Imagining how a future technology might work and the implications of such is one of the reasons I enjoy reading and writing sci-fi. That being said, I am often annoyed by wishful and frankly impossible portrayals of technology in many popular movies, shows, and novels. In particular, portrayals of virtual reality technologies are rife with problems. You might just call me a nitpicker, but I’m most interested in what might actually happen in the future and how humans as a species will respond to it.
One blatant example comes to mind: the Holoband in the Battlestar Galactica spin-off series, Caprica. The device is put on and removed as readily as a pair of glasses, and transports the user into a fully-immersive virtual reality. By fully-immersive, I mean it is like being transported to another reality, with its accompanying sights, sounds, smells, sensations, and the ability to move around as you would in reality. The characters using the Holobands are shown climbing stairs, walking through large spaces, riding in aircraft, eating, drinking, getting shot or stabbed, getting in fist fights, having sexual encounters, etc. The problem is that this not only impossible, but absurdly impossible. (Don’t take this as a diss to Caprica. I actually really enjoyed the series and was disappointed it was cut short.)
Let’s talk about immersive virtual reality. To start, I’ll need to launch into a bit of a biology lesson. I’m a neurologist, so you might find the following a bit dense, but I’ll try to make it as interesting and accessible as possible. (It’s okay to skim.)
Inputs and Outputs
Each of us as living things with brains can be viewed as a control loop. The environment around us acts on us (light hits our retinas, sound hits our eardrums, physical objects touch us, etc.), our brain processes this information (sensory input), translates it into a response (motor output), and our bodies act on the environment in turn, thus completing the loop/circuit/cycle. As a requirement, any immersive virtual reality system would have to completely hijack this control loop.
On the sensory input side we have many channels:
- olfaction (smell)
- gustation (taste)
- rotational/linear acceleration sense (vestibule, saccule, and utricle in the inner ear)
- somatic sensation (joint position sense, touch, pain, temperature, etc.)
- visceral sensation (all sensory input from the heart, stomach, intestines, and other internal organs)
Each of these channels can be further broken down into subchannels. To illustrate the complexity, taste, for example, might be the simplest of them all. It can be broken down into specific sensory neurons conveying sweet (sugars, ketones, aldehydes), sour (acid), salty (sodium), savory/umami (glutamate), and a family of 25 taste receptors attuned to chemicals we perceive as bitter. A final receptor that detects lipids (fats) is also believed to exist. Do you agree it’s complex now?
The motor output system consists of all of the motor neurons that emerge from the brainstem and spinal cord, connecting to (innervating) the muscles of the body. (There are neuroendocrine outputs from the brain also, but are not important to this discussion.) The motor nerves can be divided into two groups: somatic (voluntary) and visceral (dealing with internal organs).
Immersive Virtual Reality (IVR) Device
Now that we’ve covered some basic neurobiology, let’s further define how a virtual reality device would behave before we talk about design requirements. For simplicity, let’s say that the device gives the user the experience of being transported to another place in their own body with the ability to walk/talk/interact as he or she would in the real world. The experience would be indistinguishable from the real world or close to it. While using the device, the user’s body would stay in a resting state, preferably sitting or lying on a bed.
For the above immersive virtual reality device to work it must:
- capture and replace the user’s sensory input streams from the real world with the simulated virtual ones
- capture the user’s motor output streams and simulate appropriate movements of a virtual body
- replace the user’s motor output streams with those that maintain the real body in a safe resting state
- provide an exit switch in the form of a motor output to leave the virtual world
Before expounding on items one and two, let’s just address the last two quickly. If one is transported to another reality, we don’t want that person to walk around, bumping into things in the real world, as would happen if their motor outputs were allowed to reach the muscles in their real body. On the other hand, we can’t just block the impulses and paralyze the person either, as they would stop breathing and die. For my novel, What a Piece of Work is Man, I came up with the idea of a proxy, essentially a program that controls the body while the user is in the virtual space. It would keep the body in a safe and inert position, and even perform such tasks as exercising, eating, and using the bathroom during prolonged visits to the virtual world. My point is that it is not a trivial problem to keep one’s real body safe while they are in an immersive virtual reality.
Exiting the virtual reality is the other problem. How would the device know when you want to leave? The easiest solution would be for it to monitor your motor outputs for a trigger action or movement. This could take the form of a gesture or verbal command. Dorothy’s tapping her feet together three times and saying “there’s no place like home” would qualify. The point is there has to be a way to readily exit the virtual control loop.
Capturing and Replacing Human Input and Output Streams–the Nitty Gritty
This is the most difficult part of the whole design and the place where Caprica and The Matrix get it all wrong. Slipping on a Holoband or ramming in a series of wires into your spinal cord will not do the trick. When I talk of capturing the input and output streams of the human brain, I mean literally recording from millisecond to millisecond the exact electrical responses of each of the millions of neurons in each sensory channel. For an immersive virtual reality device to work, this must happen. There are no shortcuts. There is no convergence of the neurons to any one place in the brain, nor can you ‘fudge’ it in any way. And don’t talk to me about using EEGs and functional MRIs. I read EEGs (electroencephalograms) as part of my living, and have kept up with the medical literature on fMRI. Believe me, they will never provide anywhere near the resolution that would be required for this application.
At risk of getting even more technical, the only way to capture the input and output streams with the necessary fidelity is to have a sensor on or within every neuron. To replace the input and output streams with those corresponding to a virtual reality, the same is true: an actual physical device would have to be present on/within and assume control of every neuron in the peripheral nervous system. Topping it off, to create a coherent IVR experience, each of these millions of sensor/controllers would have to connect to a central processing unit that would run the simulation. That’s some seriously advanced and more importantly, invasive, technology.
How could this be physically possible? This post is already too long, so I’ll get to it in the next one…
Eliezer Yudkowsky has done some great work in the areas of probability theory, decision theory, and artificial intelligence. (Those are important topics for the coming decades as computers become more powerful, so dig in if you’re interested.) I found recently that he has written an especially clever and well-thought-out short story called Three Worlds Collide. This is probably the best portrayal of mankind making first contact with another intelligent race that I’ve ever encountered. He gives it away for free, so check it out!
It’s out! Thanks so much to all who have helped through this journey.
Just finished the trailer for my upcoming novel. I hope to have it available for Kindle in a few weeks.