Augmented Reality (AR)
Development Tools & Platforms
Field Of View
Head-Mounted Display (HMD)
Head Related Transfer Function (HRTF)
Heads-up Display (HUD)
Mixed Reality (MR)
Place Illusion (PI)
Plausibility Illusion (Psi)
Six Degrees of Freedom (6DOF) Tracker
Spatial Augmented Reality
Up and Down
Virtual Reality (VR)
- Agency is the capacity of some agent (typically a user) to act and affect a virtual world. The user’s sense of presence is amplified by a greater sense of agency, as is the plausibility of the virtual world.
- The quality of an object that allows a user to perform a particular action. For instance, a doorknob is an affordance for opening or closing a door. James J. Gibson introduced the term to refer to the “action possibilities” embodied in an object or environment. Unlike objects in the real world, where affordances aren’t separate from the physical characteristics of objects, virtual objects usually need to have their affordances programmed as behaviors. Therefore, objects in virtual worlds often don’t have the appropriate affordances – walls can be walked through, doors can’t be opened, and so on. Because true haptic feedback is usually an impossibility in current VR, the most basic affordance of physical objects – that they can be felt by touching them – is missing, and has to be made up for using other means.
- The standard anatomical position of the human body is defined as standing upright, feet together, arms to the side, and head, eyes and palms of the hands facing forwards. This is different from the T-pose used as the starting point for rigging an animated figure.
- Coined by Myron Kreuger in the 1970s, artificial reality refers to a graphical world that generates responses to a participant’s actions to create the illusion that those actions are taking place within that world. This was usually implemented as a kind of magic mirror, in which the user would see their figure represented in a virtual world displayed on a screen. Thus, artificial reality generally refers to a third person perspective on a virtual world.
- One of the two principal modalities used in human-computer interaction and in virtual worlds. Refers to all information received through hearing, including sound, music and speech. Often overlooked or ignored in the creation of virtual worlds, usually to the detriment of those worlds. High-quality audio can make lower-quality visual experiences seem better (see synaesthesia), and since audio resources are usually less processor-intensive than visual resources, putting more emphasis on audio is often a useful strategy.
- Augmented reality refers to any augmentation of the physical environment by computer graphics, whether viewed on a screen or on an immersive display, and whether the graphics are spatially registered to the environment or simply overlaid. Currently, this has led to a proliferation of mobile augmented reality apps in which fiducials are used to register graphics on the camera view of a mobile phone. Google Glass, a one-eye wearable augmented reality device being developed by Google X Labs, is poised to bring AR to a much wider audience. However, we are most interested in what might be called “true” augmented reality, in which a stereoscopic virtual overlay is superimposed on, and registered to, the physical world, implemented via the use of a immersive see-through display (or as an overlay on a video feed), creating the illusion that virtual objects are actually embedded in the user’s physical surroundings. AR concepts were pioneered by Steve Mann, who began building wearable AR devices in the 1980s. Ultimately, AR may be implemented as a virtual retinal display, in which the display is scanned directly onto the retina of the user’s eye, a technique pioneered by University of Washington’s HIT Lab.
- A visible representation of a user’s body in a virtual world. An avatar can resemble the user’s physical body, or be entirely different, but typically it corresponds to the user’s position, movement and gestures, allowing the user to see their own virtual body, as well as for other users to see and interact with them. Since VR is typically a first person experience, many VR applications dispense with any representation of the user whatsoever, and therefore the user is simply disembodied in virtual space – a state of being first described in H.G. Wells The Invisible Man.
- Binocular vision occurs when the images perceived by both eyes are combined into a single coherent image. Binocular vision provides stereopsis, one of the primary means of depth perception. Because each eye individually has a field of view of about 120 degrees, but only overlaps with the other eye by about 60 degrees, binocular vision provides an extremely wide field of view of about 180 degrees. Finally, it provides binocular summation, increasing the ability to detect fine detail and faint objects. For stereopsis to occur, both eyes must work together to combine two individual views (differing from one another because of binocular disparity) into a single depth-enhanced image. Estimates of the percentage of the population that lacks true binocular vision (due to disorders such as strabismus or amblyopia) range as high as 10 percent.
- A method of simulating a full three-dimensional soundscape, either by recording with a microphones embedded in a dummy head, or by synthesizing an aural soundscape by using head-related transfer functions (see below).
- Biosensors are sensors that can detect and measure brainwaves, nerve and muscle activity, skin conductance, and pulse and heart rate.
- CAVE is a recursive acronym for Cave Automatic Virtual Environment. A CAVE is an immersive projection room, usually a cube, in which continuous stereoscopic 3D projections of a virtual world fill multiple surfaces, often including the floor and ceiling. A single user’s head position is tracked, and the projections are anamorphically distorted so that the 3D view is correct from that one position. Other users can experience the virtual world at the same time, but they will see distorted views of that world.
- Virtual worlds often lack detail and complexity. In the real world, we are hardly ever presented with smooth and undifferentiated elements – almost every surface has detail and texture. At nearly every moment of our day, we are faced with an overwhelming amount of visual material, and yet we rarely feel overwhelmed. The fact that there is too much to take in at any given moment is one of the cues that we are in the real world. Detail and texture can be used to amplify a sense of immersion. And since fine detail attracts the eye, it can be used in virtual worlds to direct attention.
- In vision, convergence is the point at which the left and right lines of sight meet; the eyes rotate to converge on this point, which is generally also the point of focus and attention. In stereoscopic imaging, convergence refers to the particular elements in the left and right image that have no horizontal displacement (parallax), and appear exactly at the distance of the screen plane.
- A term coined by William Gibson in his novel Neuromancer denoting a shared multisensory virtual universe made up of the world’s networked computer systems.
- Depth cues are the features in an environment or image that facilitate our ability to perceive the world in three dimensions. Monocular depth cues provide depth information when viewing a scene with one eye, or when viewing a flat image. The primary monocular depth cues include perspective, motion parallax, occlusion, lighting and shadow, relative size, aerial perspective and texture gradients. Binocular depth cues, primarily stereopsis, require two eyes. The virtual reality designer needs to have a thorough understanding of depth cues, which can be combined in endless ways to create novel spatial experiences.
- Most virtual reality experiences never make it past the demo stage. A demo is essentially a proof of concept – it shows that the technology works, but doesn’t necessarily do much more than that. There is a real need for VR applications that go beyond mere demos, and prove that virtual reality is a viable medium for art, entertainment and education.
- VRPN: http://www.cs.unc.edu/Research/vrpn/
- VR Juggler: http://vrjuggler.org
- Unity: http://www.unity3d.com/
- Max/MSP/Jitter: http://cycling74.com/
- Processing: http://www.processing.org/
- Cinder: http://libcinder.org
- Open Frameworks: http://www.openframeworks.cc
- UDK/Unreal Engine: http://www.unrealengine.com
- The laws that govern the behavior of system within a virtual world. These laws can mimic the dynamics of the real world (for instance, gravity can be simulated) or they can be wholly imaginary and non-realistic. This is one of the strengths of VR; real and unreal dynamics can be combined in countless ways, giving a direct immersive experience of imaginary worlds governed by self-consistent but “impossible” natural laws.
- A point of reference in a visual field, used to deduce the geometric properties of the scene so that virtual objects can be superimposed onto the scene with the correct spatial coordinates. Typically, a fiducial marker is a unique two-dimensional printed pattern placed in a scene and recognized by a digital camera using techniques of computer vision. Fiducial markers are a key component of many current augmented reality applications.
- The extent of the visible world seen by a person or a camera. For a head-mounted display, the virtual camera’s visual field should be matched to the visual field of the display. Our real-world visual field is extremely wide – with both eyes, it covers nearly 180 degrees – but it doesn’t have hard boundaries (unlike most displays), and each eye’s field only partially overlaps with the other. Since it’s difficult to make a virtual reality display that produces an image large enough to fill the entire visual field, the most important consideration is to set the optical parameters of the virtual camera to match the optical size of the display surface. For example, if the display fills 100 (horizontal) degrees of the field of vision, then the virtual camera should be set to a focal length that capture 100 degrees horizontally.
- A desktop VR system utilizing a stereoscopic monitor and 3D glasses. The user’s head position is tracked, and the display updates the view of a three-dimensional scene or object, creating the illusion that the display encloses a volume of space analogous to a fishtank.
- Virtual Reality is premised on the idea of a first-person view – a requirement that a user’s position and their viewpoint be identical. While intrinsic to VR, this assumption has nonetheless prevented VR from developing an expressive grammar of the kind that we take for granted in cinema. In cinema, the camera position is almost never the exact equivalent of any single character’s viewpoint (see Third Person).
- Frame rate takes on a special importance in VR, as very slight delays in system response can cause discomfort and disorientation (see latency). Generally, the frame rate for VR applications should be 30fps or greater, preferably closer to 60fps.
- In most virtual reality experiences in which the user is standing, the virtual world will have a ground plane that coincides with the actual floor. If there is a virtual floor that doesn’t coincide with the actual floor, the user is likely to feel disoriented or uncomfortable. This is not a problem if there is no floor at all, for instance in a flying simulation.
- Gyroscope sensors are built into many current devices, such as smartphones and tablets. These 3-axis MEMs (micro-electro-mechanical-systems) gyroscopes consist of miniature vibrating elements that detect tiny changes in vibration forces when the phone is rotated. Typically, smartphones integrate this gyroscope data with data from a built-in accelerometer, which can sense gravity, and therefore can keep track of which way is up. This means that pitch (looking up and down) and yaw (tilting left and right) are effectively sensed absolutely – because of gravity, they are locked to the real world. However heading (turning left and right) is only sensed relatively, from one moment to the next, because nothing changes gravitationally when the user turns. Most smartphones include a compass sensor, and this data can be used to provide absolute coordinates for heading.
- The history of handheld stereoscopic 3D displays dates back to the 1830s, when Charles Wheatstone built the first stereoscope, later improved by David Brewster and Oliver Wendell Holmes. The Holmes stereoscope was extremely popular in the second half of the19th century, and could be said to be the VR of its day. In 1939, the View-Master was introduced, and became the most widely used stereoscope for most of the 20th century. The first stereoscopic head-mounted display (HMD) was built by Ivan Sutherland in 1965, but it wasn’t until the 1980s that any handheld stereo VR display appeared. The BOOM (Binocular Omni-Orientation Monitor), invented by FakeSpace in 1988, was one of the first handheld (or head-coupled) virtual reality displays.
- More recently, many handheld VR displays have been built by MxRLab, designed around mobile devices such as the iPhone, iPad, and Android phones. Handheld VR turns out to be a very different experience from HMD-based VR. One is perhaps more aware of the artificiality of the virtual world, since you’re always holding the device in your hands; however, the ease with which one enters and exits the virtual space creates a strong impression of the virtual world coexisting with physical space, as though two worlds are superimposed on one another.
- Haptic technologies provides tactile feedback so that a user can touch and feel virtual objects. Although haptics is still at a much earlier stage of development than computer graphics, it can be extremely useful in amplifying the plausibility of a virtual objects. The two primary means of implementing haptic feedback are vibration and force. Vibration can create a sensation of touching a virtual object, but the object provides no resistance. With force feedback, the object seems to have an actual physical presence that impedes user movement. Obviously, vibration is much easier to implement than force feedback, and this technology has been incorporated into such devices as phones, game controllers and touchpads. Force feedback is generally implemented through servo systems and actuators, and thus requires a physical device to be present in the position of the virtual object, which limits its applicability. Another technique, known as passive haptic feedback, uses fixed physical props that match the virtual objects exactly; for instance, placing a physical table in the same position as a virtual table so that the user actually feels the appropriate resistance.
- A device that is held up to the head, rather than worn as a head-mounted display. The most well known head-coupled display is FakeSpace’s Binocular Omni-Orientation Monitor, or BOOM. See Handheld Displays.
- A display device worn on the head. HMDs range from lightweight eyeglasses to full-blown helmets. Early HMDs include a 1962 patent by Morton Heilig, and the Ultimate Display, created by Ivan Sutherland in 1965. Practical low-cost HMDs were demonstrated by NASA (the VIEW and VIVED systems) in the 1980s. HMDs can be monocular (such as Google Glass) or binocular (most virtual reality HMDs). For augmented reality systems, HMDs are see-through (either through optics or video). For VR, the HMD incorporates a head-tracking device that can report the position and orientation (the pose) of the head so that the display can be updated appropriately.
- In a head-mounted or head-coupled display, the display devices is tethered to the user’s head, allowing the viewer to turn in any direction while keeping the display directly in front of his/her eyes. Contrast this to a fixed display, where, unless the user is facing the display, it is lost from view.
- A virtual or real camera is matched to the motion of the user’s head, so that their view of the virtual or telepresent scene responds the same way a real environment would react; that is to say, it appears to be a stable, all-encompassing environment. When the user turns to the left, the world pans to the right, and vice versa. Thus, the view no longer seems like an external image with an arbitrary relationship to the user’s visual field – a window – but instead it becomes identical to the visual field itself.
- The virtual camera becomes, in effect, synonymous with our eyes. We use this camera to substitute a virtual world for the real one, becoming a kind of visual prosthesis – it has to match the view from our eyes in every way possible. Because we have binocular vision, the camera has to be stereoscopic. When we turn our heads, the camera has to turn with it. The camera’s focal length is no longer arbitrary – it now has to correspond to our visual field.
- A transformation of an audio signal that modifies the signal to reflect the shape of the listener’s head, thereby allowing the listener to locate the sound in three-dimensional space.
- An overlay of data or information that is superimposed over the user’s field of view (see above). In a head-mounted display (see above) the HUD remains centered in the user’s view when s/he turns.
- The sensation of elevation can be extremely powerful in VR, and one can experience normally impossible points of view, leading to a sensation of vertigo when one looks down. Slight changes of camera height can make a huge difference in virtual worlds. The user’s height in the virtual world can be very different from their actual height; for instance, a user might be physically seated, but with a standing height as they travel through the virtual world.
- To approach the horizon – to reach the limits of the observable world, and to realize that there is absolutely nothing beyond – can be a terrifying experience. Unless the goal is to create a sense of existential dread in the user, one should probably prevent them from reaching the edge of the virtual environment.
- See mixed reality.
- In VR, the isolation of a user’s sensory apparatus from their physical surroundings, along with the substitution of a multi-sensory computer-generated virtual environment. More generally, there are two primary senses of immersion, which might be described as literal (actually being immersed in a liquid or an environment) and conceptual (a deep mental involvement). Although in discourse about VR we generally mean the first, one’s sense of presence can obviously be enhanced by the second.
- ○Intention is a key aspect of interactivity, and is especially important in VR applications. In general, the responsiveness of the system should reflect the exact intentions of the user. For instance, the more accurately a user is tracked, the closer their viewpoint will be to their intended one. This principle can also be reflected in interaction design; input options should encourage specific intended outcomes, which requires that users are well-informed about the consequences of their interactions.
- Because virtual reality is a medium that attempts to replicate one’s experience in the physical world, users are likely to have an expectation that they will be able to interact with that virtual world in the same ways they do on the outside. So, in designing interactivity for virtual worlds, the designer should generally attempt to provide as much “natural” interaction as possible. Rather than using menus and icons, interaction should, as much as possible, be with the objects in the virtual world.
- See proprioception.
- A delay in response such that the result of some action comes measurably later than the action itself. When a tracker updates its position and/or orientation, there is a lag (typically measured in milliseconds) between the tracker’s motion and its reflection by the system. Ideally, latency should be less than the frame rate (that is, less than the time it takes to display one frame). When latency from a head tracker approaches 100ms (?), the disjunction between one’s head motions and the motions of the virtual world start to feel out of sync, causing discomfort and disorientation. Large latencies are one of the causes of simulator sickness.
- Lighting – the illumination, shading and shadowing of three-dimensional objects – is a crucial component of visual experience, setting mood and defining space. However, it is often overlooked in the design of virtual environments, which often settle for a kind of “default” lighting, where everything is more or less equally illuminated. There are a number of reasons for this. Because multi-source lighting and cast shadows tend to be fairly expensive to calculate, virtual worlds are often designed with a single directional light source. Like audio, lighting often tends to be considered as an afterthought, without the careful consideration that usually goes into modeling and interaction design. Finally, because of the limited dynamic range of current displays, it is difficult or impossible to deploy the full range of lighting effects that we experience in the physical world, where objects in sunlight can be millions of times brighter than objects in deep shadow, and dimly lit environments retain much of their detail.
Not everything in a virtual world needs to be equally illuminated. While directional light sources cast the same light on everything in a scene (like sunlight), point light sources and spotlights can be used to highlight areas and objects, and to throw other parts of the scene into shadow. And although cast shadows are expensive, they are a crucial depth cue, and they are one of the most important components in making a scene feel “real”.
- In Paul Ingram’s scheme of a Reality-Virtuality Continuum (see reality-virtuality continuum), Mixed Reality occupies the middle zone between real environments at one end and purely virtual environments at the other. Mixed Reality includes Augmented Reality and Augmented Virtuality, and in fact refers to any environment in which physical and virtual objects co-exist and interact with each other.
- A mechanical platform that simulates the effects of being in a moving vehicle. Movement is programmed to be synchronized with a visual display, adding kinaesthetic effects to the visual experience.
- Virtual reality has typically been a solitary experience, perhaps largely for technical and financial reasons, but this is about to change. With the introduction of high quality low cost VR gear such as the Oculus Rift, more and more virtual experiences will be designed for multiple simultaneous users. There are many complex issues in multi-user design for virtual environments that have to be tackled, including networking and player-to-player interaction and communication.
- In most VR worlds, we want to be able to move around and explore. But in most present VR systems, we can’t simply get up and walk around – the room isn’t large enough, or we’re tethered to a tracking system, or we’re seated, or our position isn’t even being tracked. So in most cases, we have to come up with navigation techniques, methods for giving the user the ability to move their avatar around in the virtual world. The most common scheme is to use a joystick on a gamepad to move forward, back, left or right. Other methods include pointer-based navigation (pointing at a location takes you there), gaze-directed navigation (if you look at one spot long enough, you start to move towards it), and torso-steering (leaning forward and to the left or right steers you in that direction).
- Occlusion refers to the property of near objects to block our view of objects that are farther away. Like stereoscopic vision, occlusion sets up a hierarchy of relative depth relationships that is usually unambiguous, and if the two systems contradict each other, a conflict is created that is impossible to resolve, often leading to discomfort and eyestrain.
- An oculus is a circular opening at the apex of a dome, or a circular window.
- Parallax is the apparent shift in relative position of objects seen from two different points of view. The closer an object is, the more of a shift it has in relationship to farther objects. Parallax is the key component of stereopsis, as our visual system interprets parallax shifts as depth perception.
- Peripheral vision is the area of the visual field that surrounds the area of central vision or the area of sharp focus (the fovea). Our peripheral region is less sharp as central vision, and perception of color and shape is worse, but detection of motion is excellent. As we move through the world, the visual flow is greatest in the peripheral region, and so the illusion of moving through a virtual space is strongest when the display is wide enough to include our peripheral vision. Because our left and right peripheral vision don’t overlap at all, we have no stereoscopic perception in the periphery. It is important to remember that peripheral vision isn’t just to the left and right, but also above and below the central region. In fact, much of our interactions with objects (for instance, typing at a keyboard) takes place in the lower periphery, and as we move through space, this area also helps us avoid bumping into obstacles. The upper peripheral region gives us a sense of the height of the ceiling, and whether we are outdoors or in.
- The perceptual sense of actually being in the place depicted by the virtual environment. Contrast with plausibility illusion. These two terms were introduced by Mel Slater in order to break down the term presence into something more meaningful.
- The sense that characters, objects and events in a virtual world will have the appropriate responses to my presence and actions. Contrast with place illusion. These two terms were introduced by Mel Slater in order to break down the term presence into something more meaningful.
- Portals, in real life, are simply openings such as doorways, but portals are also a mainstay of science fiction, in which a doorway or other opening connects two spatially separated locations, allowing instantaneous travel between different places or realms. Screens can also be seen as metaphorical portals, in that they allow imaginative travel. In this sense, virtual reality is itself a kind of portal, one that transforms the metaphor of screen portals into an actuality. Thinking of VR as a portal highlights the importance of transitions, especially of the moment when the user is transported from the physical world to the virtual one. Special attention should be paid to this moment, but unfortunately it is usually an cumbersome sequence in which the user awkwardly dons an HMD, adjusts it for comfort, and only then is immersed in a virtual world. One interesting idea is to build a virtual replica of the actual room, so that when the user first puts on the HMD they find themselves in the same world they just left, and only then transition to another environment.
- At any given moment, an object’s rotation and translation, taken together, describe the pose of the object.
- Presence is a psychological term referring to one’s sense of being in a virtual environment. Many factors can contribute to (or subtract from) a sense of presence. In general, the sense of presence increases the more the user forgets where they “really” are, and forgets that the world they are inhabiting is only an image on a screen. The virtual world becomes a place. Presence can also be measured by how “real” the memory of the virtual world seems afterwards.
- The sense of the relative position, orientation, and muscular effort, of the parts of the body.
- While absolute realism may seem to be the ultimate objective in the design of virtual worlds, it is rarely achievable, and in many cases it is actually undesirable. Realism is merely one alternative, and should be considered a variable that can be used to sculpt the user’s experience. Useful for training purposes. Redirected walking – not notice or not care.
- Concept introduced by Paul Milgram in 1994. Ranges from pure Reality at one end to pure Virtual Reality at the other, passing through a zone of Mixed Reality, which includes both Augmented Reality and Augmented Virtuality.
- The number of times per second that a display device redraws an image. Unlike frame rate, which can be variable, the refresh rate is fixed, so if the frame rate is lower than the refresh rate, identical frames are redrawn. Although the frame rate can be reported as being faster than the refresh rate, in actuality the refresh rate places an upper limit on the frame rate. A refresh rate of at least 120Hz is necessary to avoid the perception of flicker.
- Although there is plenty of repetition in the world, it is rarely precise – real-world environments are characterized by repetition with minor variation. Virtual worlds often suffer from exactly repeating textures and objects. Unless the objective is to create a sterile, uninhabited world, minor variations and imperfections should be introduced whenever possible.
- Although resolution is specified in absolute values, it is mostly meaningless without taking into account the angular field of view. A more meaningful measurement of resolution for VR would describe each pixel in terms of degrees of field of view.
- As a VR designer, you are taking over your user’s entire sensorium. You are thus responsible for their well-being. You have unprecedented control over their emotional state. You can create a sense of calm and balance; conversely, you can create a sense of claustrophobia or confusion.
- Brightly-lit virtual worlds tend to foreground any limitations of resolution – the user can see every single pixel clearly. By subduing lighting and bathing parts of the scene in shadow, the designer can emphasize the content of the virtual environment, as pixels mass together to form volumes and voids.
- Simulator sickness is generally thought to be caused by a conflict of cues between the visual senses and the vestibular and proprioceptive (body) sensing system. In other words, the user is receiving conflicting cues between what s/he sees and the cues (or lack thereof) that their body is receiving about how it or the world is moving or being moved. This is known as the cue conflict theory. There are a number of symptoms, primarily eyestrain, ataxia (disequilibrium), and vection (illusory motion). Simulator sickness doesn’t affect everyone, but some people will experience it to some degree no matter how “perfect” the VR experience is. It is certainly made worse by badly calibrated systems, and by lag in which motion by the user results in any noticeable delay in the update of the visual display. Other factors include binocular display and wide field of view (these can be turned off to avoid simulator sickness, but at the cost of lessened immersion), and age (children are the most susceptible) and experience (VR novices tend to experience simulator sickness more). The longer a user is inside a virtual world, and the less control that user has over their movements, the more likely they are to experience simulator sickness.
- A head-mounted display has to be updated with information about what the user is doing. First, we need to know what direction they’re facing and when they turn to look in another direction. Next, we need to know where they are, and when they change their position and move somewhere else. The first of these – direction – is the rotation of the viewer’s head. The second – position – is the translation of the viewer. Because we live in a three-dimensional world, rotation and translation each provide three degrees of freedom – x, y and z – and together they add up to six degrees of freedom, a requirement of any virtual reality system. At any given moment, the rotation and translation together describe the pose of the virtual camera.
- 6DOF trackers follow the current position and orientation of a point in space, and provide numeric coordinates to a computer system. They are typically used to track the pose of a user’s head and hands, or other objects.
- It should be noted that we’re used to looking around the world even when we’re not changing our position, but the inverse is hardly true (whether or not we’re moving through the world, we can always look around). Therefore, even without position sensing, it is possible to build a fixed-position virtual reality display that supports lookaround (rotation).
- Also known as video mapping. A technique in which projected images are perfectly registered to physical objects and surfaces, creating a kind of hybrid virtual/physical entity.
- All head-mounted and head-coupled displays are stereoscopes. In a stereoscope, the stereo image pair is displayed side-by-side, so that when the device is held up close, the two images match the positions of the left and right eyes, and lenses allow the eyes to focus on the images up close. A primary advantage this arrangement is that the image – which is, after all, being displayed on a relatively small surface – can be magnified to fill the visual field of view (FOV), creating the compelling illusion that the user is actually inside the image.
- The perceptual process whereby the left and right images of a stereo pair are perceived as a single three-dimensional view.
- Synaesthesia is a neurological condition in which information received through some sensory channel triggers an experience in another sensory channel. For example, words, numbers or sounds might seem to have colors or smells attached to them. There is some evidence that, before our neural pathways are fully formed, we are all synaesthetic to one degree or another. Synaesthesia is a valuable concept for virtual reality, in that at present we are trying to stimulate all the senses, primarily through vision and sound.
- While inside a virtual world, the means by which the user retains control over the system that is generating that virtual world. This can be accomplished by providing the user with virtual controls (such as a mouse and keyboard) that perform the same actions as the actual devices.
- The sense of being present at a remote location, implemented via remote cameras and telerobotics. The user (or teleoperator) typically wears a head-mounted display, with a tracker that controls the movements of a remote robot-mounted camera. Thus, when the user turns their head, the remote camera does as well, creating the illusion that the user is seeing out of the robot’s “eyes”. Telepresence is thus similar to VR, with the difference being that the user experiences a remote physical location instead of a virtual environment.
- View of other users and/or avatars. A true third person perspective in VR is a view of your own avatar from another location – essentially an out-of-body experience.
- The transition from the user’s actual surroundings to a virtual environment is a key part of the experience, and is often ignored. One strategy is to first place the user in a virtual replica of their actual surroundings, and then transition to another virtual environment. Because one can be instantly ‘teleported’ from one virtual world to another, it is important to consider the transition between the two worlds, or the designer risks disorienting the user.
- We tend to think of peripheral vision as being off to the sides, but peripheral vision extends into the vertical as well. And one of the things that you can do in virtual reality that you can’t do with any other image-based media is to look straight up and down. Because it is so unusual to have this capability, it is one of the signs that convinces us that virtual reality is “real”. Looking straight up into the sky can be an exhilarating experience, watching virtual rain or snow come straight at you. Looking down can be equally compelling, especially looking down over the edge of a cliff, which can trigger a kind of dream vertigo, where you feel the danger of falling but know that you’re actually safe.
- In perspective projection, the point at which parallel lines converge. See horizon line.
- The nursery was silent. It was empty as a jungle glade at hot high noon. The walls were blank and two dimensional. Now, as George and Lydia Hadley stood in the center of the room, the walls began to purr and recede into crystalline distance, it seemed, and presently an African veldt appeared, in three dimensions, on all sides, in color reproduced to the final pebble and bit of straw. The ceiling above them became a deep sky with a hot yellow sun. George Hadley felt the perspiration start on his brow. — Ray Bradbury, The Veldt, 1951
I watched him throw the surveillance circuit switch. The projection came off smoothly. One second I was reclining on leather upholstery, the next, I was standing in an analog videophone booth. Since it wasn’t an empathy coupling, I wasn’t imprisoned in the back of some ID unit’s mind. Instead, I was there – in a pseudo-physical sense. — Daniel F. Galouye, Simulacron-3, 1964
The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such room would be fatal. With appropriate programming such a display could literally be the Wonderland into which Alice walked. — Ivan Sutherland, The Ultimate Display, 1965
And then without harbinger of any kind the two evolved Terrans disappeared; the grassy plain, the monument, the departing dog–the entire panorama evaporated, as if the method by which it had been projected, stabilized, and maintained had clicked to the off position. He saw only an empty white expanse, a focused glare, as if there were now no 3-D slide in the projector at all. The light, he thought, that underlies the play of phenomena which we call ” — Philip K. Dick, The Three Stigmata of Palmer Eldritch, 1965
- In visualization, complex scientific or mathematical data is graphically represented in a form that allows that data to be understood and analyzed. When viewed as formulas, algorithms or matrices of data, complex relationships can be difficult to apprehend, and so by experiencing them visually we can gain insight into underlying structures and unpredictable surface features. This process is enhanced if we can actually modify the data interactively, learning more about it through a process of cause-and-effect feedback. It can even be enhanced further if we can make the data immersive, such that we engage our entire body and sensory apparatus, using skills that we’ve acquired through a lifetime of interacting with the physical world. Thus, visualization and virtual reality are natural partners, and VR makes possible landscapes of data that would be impossible to experience through other means.
- Because there is rarely a one-to-one correspondence between physical space and the space of the virtual environment, a user doesn’t normally travel by actually walking, but through some means of transport, using a joystick or other device. This is precisely analogous to the way we use vehicles in the real world – the distances are too great to travel by foot, so we get into a car or airplane.
- Because of the powerful magnification factors of many of the lenses used for VR displays, an element of pincushion distortion is introduced into the image – that is, coordinates nearer the corners of the image are pulled outward, causing straight lines to appear to warp away from the center of the image. The solution is to apply a reciprocal barrel distortion to the image in software, in which coordinates nearer the center of the image are pushed outward. When the two warping distortions are combined, straight lines appear straight again. This is generally done as a post-processing step, by running a pixel shader on the rendered scene. This is an expensive operation, and, while it can be applied to applications running on desktop platforms, current mobile platforms don’t have the processing power to do this kind of post-processing operation at an acceptable frame rate.
- Normally, we think of the view from a virtual camera (or any camera) as a window into another world. Because windows have frames, this virtual view is something that exists inside our visual field, and is separate from it. In a virtual reality application, the view is no longer a window; the virtual camera becomes synonymous with our eyes. We use the camera to substitute a virtual world for the real one, and it becomes a kind of visual prosthesis.