Reality Check

“[Virtual reality] defines our agenda with machines as being primarily cultural and sensual, as opposed to power-oreiented.”

with Jaron Lanier

When virtual reality became a cultural obsession and took the national spotlight, Jaron Lanier stood center stage. The diverse scope of possibilities created through full sensory immersion into computer-generated worlds caught the collective imagination, and Jaron became the hero of cyberspace. He began his journey into virtual reality after quitting high school, when he engineered his own education in computer science by spending time with mentors such as Mantilz Minsky at MIT. After a stint performing as a street musician in Santa Cruz, Jaron began programming electronic sound effects into video games. He quickly became a pioneer in computer programming, and soon he started the first VR company out of his home-VPL Research-which produced most of the world ‘s VR equipment for many years. He is the co-inventor of such fundamental VR components as the interface gloves and VR networking.

Jaron coined the phrase “virtual reality” and founded the VR industry. He appears regularly on national television shows, such as “Nightline ” and “60 Minutes. ” His work with computer languages and VR was twice chosen for the cover of Scientific American, and it also appeared on the cover of the Wall Street Journal, in a piece entitled “Electronic LSD. ” But music is his first love. Since the late seventies, he has been an active composer and performer in the world of new classical music. He writes chamber and orchestral music, and is a pianist and a specialist in unusual musical instruments. Jaron has the largest collection of exotic instruments from around the world that I’ve ever encountered, and the most remarkable thing is that he can play them all. He has performed with artists as diverse as Philip Glass, Ornette Coleman, Terry Riley, Barbara Higbie, and Stanley Jordan.

Jaron has a powerful presence. His large eyes, which alternate between dreamy reflectiveness and focused intensity, peer out from behind long, brown dreadlocks. He appears gentle and relaxed, although he gets very animated when he starts talking about something that excites him. His nervous system is unusually balanced with a blend of artistic sensitivity, sharp scientific mindfulness, and great imagination. Referring to the unique neurochemistry. that must contribute to Jaron’s genius, Timothy Leary once said that he would like a cerebral spinal fluid transfusion from Jaron S brain. Jaron currently divides his time between New York and California. He has an album out on PolyGram, Instruments of Change, and two books in press, one from Harcourt/Brace and the other from MIT Press. Amid a sea of exotic musical instruments and a tangle of electronic equipment, we interviewed Jaron at his Sausalito home on February 3, 1993.

DJB

David: Jaron, what was it that originally inspired you to develop Virtual Reality technology?

Jaron: Not that question! Oh no! (laughter) There are three different things that got me involved with Virtual Reality - or really, four. One of them had to do with the philosophy of mathematics, another had to do with direct action politics and the frustrations thereof. The third had to do with musical instrument design, especially a really fantastic thing called a theremin and the fourth had to with the psychology of early childhood, specifically my own. Do you want to hear about all four of them? (laughter)

David: Well, how about how they all came together?

Jaron: Well, that’s the most mysterious of all because, of course, life is experienced mostly in anticipation and retrospect and only at rare moments in the present. So, I anticipated wondrous things without really believing them and then suddenly found they’d happened, but I can hardly even remember when they did. But somehow this whole Virtual Reality thing has been taken seriously by the world and has become a field in it’s own right.

David: I first became acquainted with your computer work many years ago when it was featured on the cover of an issue of Scientific American. What was that about?

Jaron: That was probably the first of two different Scientific American covers. It relates to one of those four tracks, which is the philosophy of mathematics. I studied math in graduate school. If you’ve ever been around a math school, you notice a very strange phenomenon which is that people who have already learned some new math thing refer to that thing as trivial, while the people who are learning it for the first time refer to it as abominably difficult. Same people, same items - just different moments in time.

So the question is; What’s going on? Is this just a sort of a hubris, or is there something particularly unusual about the process of learning math and understanding it? I was suspicious that the way that math was communicated and the way it was understood were completely different, that math had to be understood somewhat visually or perhaps in something of a process way - playing with images and shapes inside one’s imagination.

Math was taught and expressed in notation systems and I thought, maybe the notation system is the problem. But then as soon as you think that, you have to ask yourself, what of mathematics is left once you take away the notation system? And that’s a very hard question because the traditional view of most mathematics almost equates it with at least some kind of notation system.

David: But isn’t the dynamic pattern left, which the notation system is expressing?

Jaron: It depends on the type of math. Some types of math are largely a by-product of notation, like algebra - at least, that’s my interpretation. There are some kinds of math which are a little ambiguous. If you’re talking about the mathematics of dynamic flows and that sort of thing, indeed the flows exist without the math, but a lot of abstract math might not exist effectively without some kind of notation. It’s just not clear - it’s a hard problem.

So, I became interested in the idea of having computer pictures of mathematical structures, pictorial interpretations that would be modified in real time to express the movement of mathematical ideas. I was fortunate to get hired as part of an NSF project - I was just a kid in my teens - and I worked with a group of people doing a college level curriculum in mathematics using pictures which these days would not be too radical or exciting, but which in those days was pretty unusual.

That project had mixed results, but I became very interested in the process of programming. I started to think, well, maybe you can look at computers and computer programming as a simplified mirror of the problem of mathematics and mathematical notation, but where the situation is much less ambiguous. There are some similarities. In computers, you have a notation (a programming language), that tells the computer what to do, and then there’s what the computer actually does. Unlike math, there’s no trouble defining the reality of what the computer actually does, as a separate matter from the notation.

So you can ask yourself, is there some better way to understand what the computer is doing than using the usual programming notation? And if you could come up with something that was an interactive visual representation of what the computer’s doing, would that give you any inspiration (getting back to the original question) towards being able to express math ideas in a better way?

David: Was that an original thought? It sounds like a really unique way of looking at it.

Jaron: No, although I didn’t know of his work at the time, the first person to think about visual programming languages, as far as I know, was a particularly brilliant guy namedIvan Sutherland who also invented the head-mounted display and computer graphics in general and who is a very important seminal figure in twentieth-century culture and science. The idea of a visual version of math, on the other hand, has fascinated many people, like G. Spencer Brown.

So I became immersed in the world of programming language design and I was very fortunate to be able to spend a lot of time and be somewhat taken under the wing of some of the people who’d invented the standard programming languages in use. I started developing a big “virtual” programming language and it was an off-shoot of that work, which was oriented towards children, which was on the cover of Scientific American in 1984.

The early team included some wonderful and brilliant minds, like Chuck Blanchard, Young Harvill, Tom Zimmerman, and Steve Bryson. The biggest problem we had, in that early work, was that the screen was too small a window - a frustratingly tiny