 |
AN INVESTIGATION OF CURRENT
VIRTUAL REALITY INTERFACES
By
Christopher M. Smith
Lynellen D.S.P. Smith
Steven Yang
A Final Project
Submitted to the Faculty of
Mississippi State University
in Partial Fulfillment of the Requirements
for CS4663/6663
in the Department of Computer Science
Mississippi State, Mississippi
April 25, 1995
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Visual Aspects of Virtual Reality. . . . . . . . . . . . . . . . . . . . 1
Types of Visual Display . . . . . . . . . . . . . . . . . . . . . . 2
LCD Flicker Lens . . . . . . . . . . . . . . . . . . . . . . . 2
Head Mounted Displays. . . . . . . . . . . . . . . . . . . . . 3
LCD display HMD. . . . . . . . . . . . . . . . . . . . . . . . 3
Projected HMD. . . . . . . . . . . . . . . . . . . . . . . . . 3
Small CRT HMD. . . . . . . . . . . . . . . . . . . . . . . . . 4
Single Column LED HMD. . . . . . . . . . . . . . . . . . . . . 4
Binocular Omni-Orientation Monitor (BOOM). . . . . . . . . . . 4
Advantages and Disadvantages. . . . . . . . . . . . . . . . . . . . 5
Graphic Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Depth Cueing. . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Lighting Models . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Shading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Radiosity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Ray Casting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3-D Audio. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Evolution of 3D Sound . . . . . . . . . . . . . . . . . . . . . . . 11
Realistic Sound . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Tactile and Force Feedback . . . . . . . . . . . . . . . . . . . . . . . 15
Force Feedback. . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Motion Platforms . . . . . . . . . . . . . . . . . . . . . . . 16
Gloves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Exoskeletons . . . . . . . . . . . . . . . . . . . . . . . . . 17
Butlers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Tactile Feedback. . . . . . . . . . . . . . . . . . . . . . . . . . 18
Texture. . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Tracking Devices. . . . . . . . . . . . . . . . . . . . . . . . . . 19
Mechanical Trackers. . . . . . . . . . . . . . . . . . . . . . 21
Electromagnetic Trackers . . . . . . . . . . . . . . . . . . . 21
Ultrasonic Trackers. . . . . . . . . . . . . . . . . . . . . . 22
Infrared Trackers. . . . . . . . . . . . . . . . . . . . . . . 23
Inertial Trackers. . . . . . . . . . . . . . . . . . . . . . . 24
Interaction Devices . . . . . . . . . . . . . . . . . . . . . . . . 25
Gloves . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3D Mice. . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Joysticks. . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Human Factors in Virtual Environments. . . . . . . . . . . . . . . . . . 28
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Abstract
Virtual Reality hype is becoming a large part of
everyday life. This paper explores the components of actual
virtual reality systems, critiqueing each in terms of human
factors. The hardware and software of visual, aural, and
haptic input and feedback are considered. Technical and
human factor difficulties are discussed and some potential
solutions are offered.
Introduction
Virtual reality is a new technology for simulation,
design, entertainment, and many other pursuits. Simulation
applications range from testing a non-existent object before
it is created, to military training, to practicing a
maneuver for the next shuttle mission. The purpose of our
paper is to identify weaknesses in virtual reality
interfaces. To accomplish this task, we have divided the
typical virtual reality interface into four specific areas:
audio, visual, tactile, and navigation. We will point out
the limitations of current solutions to problems in these
areas, possible areas of improvement, and those problems
that remain completely unsolved at this point.
Visual Aspects of Virtual Reality
The main tradeoffs in this area are image detail versus
rendering speed, and monoscopic versus stereoscopic vision.
In most applications of virtual reality, visual feedback is
required. In fact, visual cues are perhaps the most
important feedback required in the virtual reality system.
To achieve reality, the pictures sent to the display have to
be real-time to avoid discontinuity. Therefore, the
trade-off between the rendering time and the graphic
resolution for the 3-dimensional graphic scene and the
2-dimensional graphic scene are investigated from both the
software and hardware perspective.
Types of Visual Display
LCD Flicker Lens
The LCD (liquid crystal display) flicker lens has the
appearance of a pair of glasses. A photosensor is mounted
on these LCD glasses with the sole purpose of reading a
signal from the computer. This signal tells the LCD glasses
whether to allow light to pass through the left lens or the
right lens. When light is allowed to pass through the left
lens, the computer screen will be showing a left eye scene,
which corresponds to what the user will see through his left
eye. When light passes through the right lens, the scene on
the computer screen is a slightly offset version of the view
from the left eye. The glasses switch between the two
lenses at 60 Hertz, which causes the user to perceive a
continual 3D view via the mechanics of parallax (Blanchard
and Tsuneto).
Head Mounted Displays
Head mounted displays place a display screen in front
of each of the viewer's eyes at all times. The view, the
segment of the virtual environment generated and displayed,
is controlled by orientation sensors mounted on the
"helmet". Head movement is recognized by the computer, and
a new perspective of the scene is generated. In most cases,
a set of optical lens and mirrors are used to enlarge the
view to fill the field of view and to direct the scene to
the eyes (Lane). Four types of Head Mounted Displays (HMDs)
will be discussed below.
LCD display HMD
This type of HMD uses LCD technology to display the
scene. When a liquid crystal pixel is activated, it blocks
the passage of light through it. Thousands of these pixels
are arranged in a two dimensional matrix for each display.
Since liquid crystals block the passage of light, to display
the scene a light must be shone from behind the LCD matrix
toward the eye to provide brightness for the scene
(Aukstakalnis and Blatner 1992).
Projected HMD
This type of HMD uses fiber optic cables to transmit
the scene to the screen. The screen is similar to a cathode
ray tube (CRT) except the phosphor is illuminated by the
light transmitted through fiber optic cables. Ideally, each
fiber would control one pixel. But due to the limitation in
cost and manufacturing, each fiber controls a honeycomb
section of pixels (Lane).
Small CRT HMD
This type of HMD uses two CRTs that are positioned on
the side of the HMD. Mirrors are used to direct the scene
to the viewers eye. Unlike the projected HMD where the
phosphor is illuminated by fiber optic cables, here the
phosphor is illuminated by an electron gun as usual (Lane).
Single Column LED HMD
This type of HMD uses one column of 280 LEDs. A mirror
rapidly oscillates opposite from the LEDs, reflecting the
image to the user's eye. The LEDs are updated 720 times per
oscillation of the mirror. As the LED column updates for
each column of the virtual screen, the mirror redirects the
light to the viewers eye, one column at a time, to form the
image of the entire virtual screen (Aukstakalnis and Blatner
1992).
Binocular Omni-Orientation Monitor (BOOM)
The binocular omni-orientation monitor is mounted on a
jointed mechanical arm with tracking sensors located at the
joints. A counterbalance is used to stabilize the monitor,
so that when the user releases the monitor, it remains in
place. To view the virtual environment, the user must take
hold of the monitor and put her face up to it. The computer
will generate an appropriate scene based on the position and
orientation of the joints on the mechanical arm
(Aukstakalnis and Blatner 1992).
Advantages and Disadvantages
LCD flicker lens' are light weight and cordless. These
two features makes them easy to wear and to remove.
Unfortunately, the user would have to stare only at the
computer screen in order to see the 3-D scene. Since the
field of view is limited to the size of the computer screen,
the surrounding real world environment can also be noticed.
This does not provide an immersive effect.
The LCD display HMD is lighter than most HMDs. As with
most HMDs, it does provide an immersive effect, but the
resolution and the contrast is low. The problem associated
with low resolution is inability to identify objects and
inability to locate the exact position of objects. Since
the crystals are polarized to control the color of a pixel,
the actual polarizing of the crystal creates a small delay
while forming the image on the screen. Such a delay may
cause the viewer to misjudge the position of objects (Bolas
1994).
The projected HMD provides better resolution and
contrast than LCD displays. This HMD is also light weight.
Higher resolution and contrast means that the viewer is able
to see an image with greater detail. The downside of this
type of HMD is that it is expensive and difficult to
manufacture (Bolas 1994).
The CRT HMD is in many ways similar to the projected
HMD. This type of HMD is heavier than most other types
because of added electronic components. These electronic
parts also generate large amounts of heat. The user wearing
this type of HMD may feel discomfort due to the heat and the
weight of the HMD (Bolas 1994).
Single Column LED HMDs allow the user to interact with
a virtual world and the real world simultaneously. This
type of display can be used to create a virtual screen that
seems to float in the real world.
One of the common problems of HMDs is that the cable
connecting the HMD and a computer restricts the mobility of
the user. The user can only move as far as the cable
allows. If the cable is not properly managed, one could
trip over it or become entangled in it. Another problem
occurs when switching frequently between a virtual world and
the real world. Continually wearing and removing the HMD is
tedious and tiresome.
Some of the problems associated with HMDs can be solved
by using a BOOM display. The user does not have to wear a
BOOM display as in the case of an HMD. This means that
crossing the boundary between a virtual world and the real
world is simply a matter of moving the eyes away from the
BOOM.
Graphic Techniques
Depth Cueing
Depth Cueing is a technique to provide a 3-D
perspective of a scene. Basically, it adds depth to a 2-D
picture. There are two ways to add such perspective. The
first method is to change the color of pixels. For example,
a light source located farther away looks dimmer than one
located nearer the viewer. When the scene is generated, the
red, green, and blue color values of the light source
located farther away can be reduced by the same amount to
represent its distance. The second method is to add color
to the scene. For example, fog may exist between a light
source and the viewpoint. The color value corresponding to
that light source in the fog is accumulative from the
viewpoint. The deeper the light is in the fog, the whiter
its light. In this manner the user can judge the depth of
the scene and navigate through the virtual world with less
effort (Foley et al. 1992).
Lighting Models
Lighting is essential in creating a realistic 3-D
image. In an unlit atmosphere, a sphere looks the same as a
disk. In other words, a 3-D scene becomes a 2-D scene. As
a light shines on a sphere, some of the light is reflected
diffusely and thereby gives an indication of three-
dimensionality. The other reflected light allows the viewer
to see the sphere's specular reflection and shadow, which
provide further spatial information. Due to the complexity
of the lighting formula, computing the intensity of light at
each pixel reduces the screen update rate (Foley et al.
1992). When the update rate falls too low, flickers are
noticeable and can cause visually induced motion sickness.
Motion sickness occurs whenever the body receives
conflicting information from the eyes and the brain (Bolas
1994).
Shading
Shading is used in conjunction with lighting. As the
lighting model is computed, the intensity of the light at
each vertex of a polygon is calculated. Two types of
shading can be used to render a scene. The first is flat
shading, where the entire polygon takes on the value of the
color of one of the vertices. The second is Gouraud
shading, where pixel color within the polygon is linearly
interpolated from the color at each vertex to create a
smoothly varying shade. Since linear interpolation requires
little computational power, it is often used to render
realistic scenes quickly (Foley et al. 1992).
Radiosity
Radiosity is most often used for rendering a static
scene. This technique pre-calculates the light interaction
among the objects in the 3-D environment. The calculation
of light interaction for a complex environment usually takes
a long time. However, upon completion of the calculation,
the user can view this environment from any angle, on the
fly, with no further exhaustive computation (Foley et al.
1992).
Ray Casting
Ray casting is a real-time technique for rendering a
scene. A ray is traced from the viewpoint through a pixel
of the screen to intersect an object defined in the virtual
space. The color of the closest object intersected is
recorded for that pixel of the screen. If an object is
moved, ray casting can trace the new position of the object,
thus creating a dynamic scene (Foley et al. 1992).
Distortion correction is needed in most of HMDs. The
optic lenses used in the HMD magnifies the image displayed
on the screen. This magnification causes the image to be
distorted radially. Since this distorted image is not what
one would see in the real world, the viewer may feel nausea
in the virtual world (Watson and Hodges).
3-D Audio
The main research area in audio is the simulation of
sound origin. "It has been demonstrated that using sound to
supply alternative or supplementary information to a
computer user can greatly increase the amount of information
they can ingest" (Aukstakalnis and Blatner 1992). This is
no less true in the virtual world. Although a virtual
environment provides a multitude of visual cues, a person
can't see what is behind him/her. However, in reality
someone can hear what is behind them and it should be so in
a virtual world.
The main problem with producing sound is that it is
impossible to replay previously recorded sound in a manner
that moves a sound from behind you to in front of you as you
turn your head. Crystal River Engineering has developed a
process for producing a sound such that it seems like it is
coming from a particular direction. Since these sounds are
computed and produced in real time, there is not a problem
with playback.
In addition to visual output, a complete virtual world
must incorporate a three dimensional sound field that
reflects the conditions modeled in the virtual environment.
This sound field has to react to walls, multiple sound
sources, and background noise, as well as the absence of
them. This requires massive computational power and speed
because hearing is a complex system which uses the shape of
the outer ear and microsecond delays in the arrival of sound
to the two ears to determine position and location of the
source of the sound.
Evolution of 3D Sound
The evolution of 3D sound can be traced through time
and technology starting with monophonic sound. "Mono", the
latin word for "one", sends one signal to every speaker. It
appears that all of the sounds of the environment are coming
from each individual speaker. If there is only one speaker,
then all sounds seem to be coming from that point.
Stereophonic sound is the next step in the development
of realistic sound production. It allows for the sounds to
seem as if they are coming from anywhere between two
speakers. This is accomplished by delaying the signals
between the two speakers by a few microseconds. The smaller
the delay, the closer to the center the source appears to be
located.
Surround sound, used in many theaters, uses the idea of
stereo but with more speakers. They delays can be set so
that a sound can seem to move from behind the listener to in
front of the listener. An example problem with this system
is that a plane taking off behind the listener will appear
to go by the listener's elbow instead of overhead (Crystal
River Engineering 1994).
A solution to the problem of creating a three
dimensional sound field comes from production of sound which
is tuned to an individual's head. When sound reaches the
outer ear, the outer ear bends the sound wavefront and
channels it down the ear canal. The sound that actually
reaches the eardrum is different for each person
(Aukstakalnis and Blatner 1992). To resolve this problem,
the computer must create a sound that is custom designed for
a particular user. This is done by placing small
microphones inside the ear canal, then creating reference
sounds from various locations around the listener. Then the
computer solves a set of mathematical relationships that
describe how the sound is changed from being produced to
being received inside the ear canal. These mathematical
relationships are called Head Related Transfer Functions
(HRTFs) (Crystal River Engineering 1994).
To simulate a virtual sound environment, a computer
must first determine the position of the source relative to
the listener. It also must calculate the effects of the
environment. For example, to simulate an echo due to a
wall, the computer must first determine the source's
location relative to the subject and the wall, then place
another source the at the appropriate distance and location
on the opposite side of the wall (Tonnesen and Steinmetz).
Realistic Sound
An additional computational burden is the production of
background noise. This is very important if the person in
the virtual environment wishes to be immersed in a
"believable" world. However, since the noise is background,
it does not need to take advantage of 3D sound technology.
This limits the interactivity of the user with the virtual
environment. In the real world, a person can pick out
sounds from the background. This ability is commonly called
the "cocktail party affect", because of the ability of a
person to focus on different conversations from the
background noise. This can only be done in a 3D acoustic
field (Steinmentz and Lee) and background noise in the
virtual world does not use a 3D field.
Some researchers suggest the use of prerecorded sounds
so that all computational power is devoted to determining
the location and direction of the source. This, however,
can not work in a 3D sound field. Although the sounds can
be accurately placed in a 3-D sound field, the listener can
not interact with the environment--he/she can only observe
it. In a 3D acoustic field played through headphones, when
the listener turns around, the sounds that were behind the
listener should now be in front. However, with
prerecorded/playback methods, the sounds that were behind
the listener are still behind the listener (Aukstakalnis and
Blatner 1992).
HRTF measurements can not accurately simulate the
acoustical environment when used alone. The problem lies in
trying to nonintrusively make measurements. When the
microphone is placed in the ear canal, it changes the
acoustical track, thereby changing the HRTF. Also, this
method does not attempt to take into consideration the
middle or inner ear (Steinmetz and Lee).
A realistic sound environment has great potential to be
an interface for visually impaired or blind people. For
example, a virtual environment can be created where the
objects in it are application software. Then the users can
learn their way around the environment, much like they can
learn their way from home to the store without ever having
to see.
Tactile and Force Feedback
One of the biggest complaints about top of the line
virtual environment packages is the "lack of tangibility."
Although the area of tactile feedback is only a few years
old, it has produced some impressive results. These
solutions are critiqued below. There is no interface
currently built that will simulate the interactions of
shape, texture, temperature, firmness, and force.
Being able to produce a realistic interface means
having to produce tactile and force feedback to correspond
to the objects in the virtual world. Dr. Fred Brooks of the
University of North Carolina at Chapel Hill is noted for
introducing the problem of "shin-knockers" (Brooks 1995).
This was originally in reference to modeling of a submarine.
"How are you going to let the person know when he knocks his
shin on a pipe that is sticking out in his way?"
The area of touch has been broken down into two
different areas. Force feedback deals with how the virtual
environment affects a user. For example, walls should stop
someone instead of letting him/her pass through, and pipes
should knock a user in the shin to let him/her know that
they are there. Tactile feedback deals with how a virtual
object feels. Temperature, size, shape, firmness, and
texture are some of the bits of information gained through
the sense of touch.
Force Feedback
There are several different types of devices that allow
a user to "feel" certain aspects of the virtual environment.
Motion platforms for simulators and simulated rides, force
feedback gloves, exoskeletons, and butlers are all forms of
force feedback.
Motion Platforms
The motion platform was originally designed for use in
flight simulators which train pilots. A platform is bolted
to a set of hydraulic lift arms. As the motion from a
visual display changes, the platform tilts and moves in a
synchronous path to give the user a "feeling" that he/she is
actually flying. This has a serious limitation in that it
can only go so far. If the user's visual cues are that the
plane is upside down, the hydraulics can not simulate this.
However, it does give the middle ear sensations that
correspond to the visual scene, making the simulation more
realistic.
Gloves
For interaction with small objects in a virtual world,
the user can use one of several gloves designed to give
feedback on the characteristics of the object. This can be
done with pneumatic pistons which are mounted on the palm of
the glove, as in the Rutgers Master II (Gomez, Burden and
Langrana 1995). When a virtual object is placed in the
virtual hand, the users hand can close around it. When the
fingers would meet resistance from the object in reality,
the pressure in the pistons is increased, giving the
sensation of resistance from the virtual object.
Exoskeletons
Exoskeletons are also employed to simulate the
resistance of objects in a virtual world. An exoskeleton is
basically a robotic arm strapped to a person. At the
University of Utah, researchers have developed a robotic arm
which has 10 degrees of freedom. The robot continuously
updates the force to each of its ten joints, and can make it
appear that the 50 pound arm is weightless. "However, when
the operator touches something, the virtual forces become
actual forces felt through the exoskeleton" (Lane and
Smith). This would make the operator's arm stop when it hit
a virtual wall or feel the weight of a virtual object.
Butlers
The butler is a robot that basically gets in the way
whenever you try to move through an object. If a user
reaches out his/her hand to touch a wall, desk, or any other
virtual object, the butler robot will place a real object at
the location where the virtual object is supposed to be.
This technique is currently being researched at the
University of Tokyo by Susumu Tachi (Tachi 1995). The
butler being worked on "provides mechanical impedance of the
environment, i.e., inertia, viscosity and stiffness" (Tachi
1995). The major drawback of the butler robot is that it
can only present these properties for a single point at a
time.
Tactile Feedback
The butler robot under development can give the
impression of stiffness and viscosity, but it can't present
the information needed by a human to know what the object
feels like. The temperature and texture are totally unknown
to the user. It is possible to display the temperature by
heat resistive wires sown into the lining of a glove.
Texture
The texture of a surface is probably the hardest
feature of tactile feedback to simulate. The closest
documented attempt is the Sandpaper system. This system,
developed by a research group which includes members from
MIT and UNC, can accurately simulate several different
grades of sandpaper (Aukstakalnis and Blatner 1992). Other
systems, like the Teletact Commander, use either airfilled
bladders sown into a glove, or piezo-electric transducers to
provide the sensation of pressure or vibrations. These
systems have problems with the unreliability of compressors
and interference between the piezo-electric transducer
electromagnetic fields and the electromagnetic field used by
a Polhemus tracking system (Stone 1993).
Any attempt to model the texture of a surface faces
tremendous challenges because of the way the human haptic
system functions. There are several types of nerves which
serve different functions, including: temperature sensors,
pressure sensors, rapid-varying pressure sensors, sensors to
detect force exerted by muscles, and sensors to detect hair
movements on the skin. All of these human factors must be
taken into consideration when attempting to develop a
tactile human-machine interface.
Navigation
Tracking Devices
The purpose of a tracking device is to determine the x,
y, and z position, and the orientation (yaw, pitch, and
roll) of some part of the user's body in reference to a
fixed point. Most types of virtual reality interaction
devices will have a tracker on them. HMDs need a tracker so
that the view can be updated for the current orientation of
the user's head. Datagloves and flying joysticks also
usually have trackers so that the virtual "hand" icon will
follow the position and orientation changes of the user's
real hand. Full body datasuits will have several trackers
on them so that virtual feet, waist, hands, and head are all
slaved to the human user.
When designing or evaluating a virtual reality system
that will receive tracking information, it is important to
pay attention to the latency (lag), update rate, resolution,
and accuracy of the tracking system. Latency is the "delay
between the change of the position and orientation of the
target being tracked and the report of the change to the
computer" (Baratoff and Blanksteen). If the latency is
greater than 50 milliseconds, it will be noticed by the user
and can even cause nausea or vertigo. Update rate is the
rate at which the tracker reports data to the computer, and
is typically between 30 and 60 updates per second.
Resolution will depend on the type of tracker used, and
accuracy will usually decrease as the user moves farther
from the fixed reference point (Baratoff and Blanksteen).
Six-degree-of-freedom tracking devices come in several basic
types of technology: mechanical, electromagnetic,
ultrasonic, infra-red, and inertial.
Mechanical Trackers
A mechanical tracker is similar to a robot arm and
consists of a jointed structure with rigid links, a
supporting base, and an "active end" which is attached to
the body part being tracked (Sowizral 1995), often the hand.
This type of tracker is fast, accurate, and is not
susceptible to jitter. However, it also tends to encumber
the movement of the user, has a restricted area of
operation, and the technical problem of tracking the head
and two hands at the same time is still difficult.
Electromagnetic Trackers
An electromagnetic tracker allows several body parts to
be tracked simultaneously and will function correctly if
objects come between the source and the detector. In this
type of tracker, the source produces three electromagnetic
fields each of which is perpendicular to the others. The
detector on the user's body then measures field attenuation
(the strength and direction of the electromagnetic field)
and sends this information back to a computer. The computer
triangulates the distance and orientation of the three
perpendicular axies in the detector relative to the three
electromagnetic fields produced by the source (Baratoff and
Blanksteen).
Electromagnetic trackers are popular, but they are
inaccurate. They suffer from latency problems, distortion
of data, and they can be thrown off by large amounts of
metal in the surrounding work area or by other
electromagnetic fields, such as those from other pieces of
large computer equipment. In addition, the detector must be
within a restricted range from the source or it will not be
able to send back accurate information (Sowizral 1995), so
the user has a limited working volume.
Ultrasonic Trackers
Ultrasonic tracking devices consist of three high
frequency sound wave emitters in a rigid formation that form
the source for three receivers that are also in a rigid
arrangement on the user. There are two ways to calculate
position and orientation using acoustic trackers. The first
is called "phase coherence". Position and orientation is
detected by computing the difference in the phases of the
soundwaves that reach the receivers from the emitters as
compared to soundwaves produced by the receiver. "As long
as the distance traveled by the target is less than one
wavelength between updates, the system can update the
position of the target" (Baratoff and Blanksteen). The
second method is "time-of-flight", which measures the time
for sound, emitted by the transmitters at known moments, to
reach the sensors. Only one transmitter is need to
calculate position, but the calculation of orientation
requires finding the differences between three sensors
(Baratoff and Blanksteen).
Unlike electromagnetic trackers that are affected by
large amounts of metal, ultrasonic trackers do not suffer
from this problem. However, ultrasonic trackers also have a
restricted workspace volume and, worse, must have a direct
line-of-sight from the emitter to the detector. Time-of-
flight trackers usually have a low update rate, and phase-
coherence trackers are subject to error accumulation over
time (Baratoff and Blanksteen). Additionally, both types
are affected by temperature and pressure changes (Sowizral
1995), and the humidity level of the work environment
(Baratoff and Blanksteen).
Infrared Trackers
Infrared (optical) trackers utilize several emitters
fixed in a rigid arrangement while cameras or "quad cells"
receive the IR light. To fix the position of the tracker, a
computer must triangulate a position based on the data from
the cameras. This type of tracker is not affected by large
amounts of metal, has a high update rate, and low latency
(Baratoff and Blanksteen). However, the emitters must be
directly in the line-of-sight of the cameras or quad cells.
In addition, any other sources of infrared light, high-
intensity light, or other glare will affect the correctness
of the measurement (Sowizral 1995).
Inertial Trackers
Finally, there are several types of inertial tracking
devices which allow the user to move about in a
comparatively large working volume because there is no
hardware or cabling between a computer and the tracker.
Inertial trackers apply the principle of conservation of
angular momentum (Baratoff and Blanksteen). Miniature
gyroscopes can be attached to HMDs, but they tend to drift
(up to 10 degrees per minute) and to be sensitive to
vibration. Yaw, pitch, and roll are calculated by measuring
the resistance of the gyroscope to a change in orientation.
If tracking of position is desired, an additional type of
tracker must be used (Baratoff and Blanksteen).
Accelerometers are another option, but they also drift and
their output is distorted by the gravity field (Sowizral
1995).
Interaction Devices
Virtual reality and virtual environments go far beyond
typical interfaces in the realism of the visual metaphor.
Point and click with a table-top mouse is wonderful in some
situations, but not nearly sufficient for an immersive
environment. So instead of a keyboard and mouse,
researchers are developing gloves, 3D mice, floating
joysticks, and voice recognition. This paper will not
attempt to cover voice recognition because it is such a
large domain.
Gloves
For sensing the flexion of the fingers, three types of
glove technology have arisen: optical fiber sensors,
mechanical measurement, and strain gauges. The Dataglove
(originally developed by VPL Research) is a neoprene fabric
glove with two fiber optic loops on each finger. Each loop
is dedicated to one knuckle and this can be a problem. If a
user has extra large or small hands, the loops will not
correspond very well to the actual knuckle position and the
user will not be able to produce very accurate gestures. At
one end of each loop is an LED and at the other end is a
photosensor. The fiber optic cable has small cuts along its
length. When the user bends a finger, light escapes from
the fiber optic cable through these cuts. The amount of
light reaching the photosensor is measured and converted
into a measure of how much the finger is bent (Aukstakalnis
and Blatner 1992). The Dataglove requires recalibration for
each user (Hsu). "The implications for longer term use of
devices such as the Dataglove--fatigue effects,
recalibration during a session--remain to be explored"
(Wilson and Conway 1991).
The Powerglove was originally sold by Mattel for the
Nintendo Home Entertainment System but, due to its low
price, has been used widely in research (Aukstakalnis and
Blatner 1992). This Powerglove is less accurate than the
Dataglove, and also needs recalibration for each user, but
is more rugged than the Dataglove. The Powerglove uses
strain gauges to measure the flexion of each finger.
A small strip of mylar plastic is coated with an
electrically conductive ink and placed along the length
of each finger. When the fingers are kept straight, a
small electrical current passing through the ink
remains stable. When a finger is bent, the computer
can measure the change in the ink's electrical
resistance (Aukstakalnis and Blatner 1992).
The dexterous hand master (DHM) is not exactly a glove
but a exoskeleton that attaches to the fingers with velcro
straps. A mechanical sensor measures the flexion of the
finger. Unlike the Dataglove and Powerglove, the DHM is
able to detect and measure the side-to-side movement of a
finger. The other gloves only measure finger flexion. The
DHM is more accurate than either of the gloves and less
sensitive to the user's hand size, but can be awkward to
work with (Hsu).
The main strength of the various types of gloves is
that they provide a more intuitive interaction device than a
mouse or a joystick. This is because the gloves allow the
computer to read and represent hand gestures. Objects in
the environment can therefore be "grasped" and manipulated,
the user can point in the direction of desired movement,
windows can be dismissed, etc (Wilson and Conway 1991).
"Gestures should be natural and intuitive in the particular
virtual environment. Actions should be represented visually
and be incremental, immediate, and reversible to give a
person the impression of acting directly in an environment"
(Dennehy). Wilson and Conway (1991) say that a basic set of
command gestures for gloves has been developed, but that
more work is needed to expand the set beyond the current
simple mapping. Another area of improvement is feedback
for the user to aid hand-eye coordination and proprioceptive
feedback to let a user know when an object has been
successfully grasped (Wilson and Conway 1991).
3D Mice
There are several brands of 3D mice available, all with
basically the same technology: A mouse or trackball has been
modified to include a position and orientation tracker of
some kind (Aukstakalnis and Blatner 1992). This modified
mouse is fairly familiar and intuitive to users--simply push
the mouse in the direction you want to move. However, these
mice are not very useful for interactions other than
navigation and selection of objects (Hsu).
Joysticks
The final category of interaction device is the wand or
floating joystick. Basically, this device works exactly the
same as a conventional joystick, but it is not attached to a
base that sits on a table top. Instead, the joystick is
equipped with an orientation tracker so the user simply
holds it in their hand and tilts it. Most flying joysticks
also have some buttons on the stick for "clicking" or
selecting, similar to a mouse (Hsu).
Human Factors in Virtual Environments
Kay Stanney (1995) has written an excellent critique of
the areas that still need to be researched in order to make
virtual environments a safe place to work. These include
health concerns such as "flicker vertigo" which can induce a
seizure, auditory and inner ear damage from high volume
audio, prolonged repetitive movements which cause overuse
injuries (for example, carpal tunnel syndrome), and head,
neck, and spine damage from the weight or position of HMDs.
Safety factors also need to be considered. For example,
when a user's vision is restricted by an HMD, they are
likely to trip and fall over cables or other real world
objects. Also, how safe is the user from harm in the event
of system failure? Hands and arms might be pinched or over
extended if a haptic feedback device fails; the user might
be disoriented or harmed if the computer crashes and
suddenly dumps the user into reality, disrupting the sense
of "presence".
Conclusion
Virtual reality holds promises of being the "ultimate
human-computer interface". This would incorporate a
natural, intuitive interface between a human and a machine
generated work environment.
An intuitive interface between man and machine is
one which requires little training . . . and proffers a
working style most like that used by the human being to
interact with environments and objects in his day-to-
day life. In other words, the human interacts with
elements of this task by looking, holding,
manipulating, speaking, listening, and moving, using as
many of his natural skills as are appropriate, or can
reasonable be expected to be applied to as task (Stone
1993).
This paper has overviewed the technology currently in
use, in addition to the open areas of research. Visual
display devices, graphics display techniques, 3D audio,
haptic feedback, navigation, and interaction devices are all
in need of more development. Large areas of concern about
the health and safety of the user are still in focus, not to
mention the unsolved technical problems standing in the way
of an intuitive immersive environment. As the public market
for virtual reality grows in the coming years, more money
will be spent on quality interface improvements and some of
these problems may be solved.
References
Aukstakalnis, Steve, and David Blatner. 1992. Silicon
Mirage The Art and Science of Virtual Reality.
Berkeley, California: Peachpit Press, Inc.
Baratoff, Gregory, and Scott Blanksteen. Unknown.
Encyclopedia of Virtual Environments. World Wide Web URL
http://www.cs.umd.edu/projects/eve/eve-articles/I.D.1.b.TrackingDevices.html
Blanchard, Jim, and Reiko Tsuneto. Unknown. Stereoscopic
viewing. Encyclopedia of Virtual Environments. World
Wide Web URL:
http://www.cs.umd.edu/projects/eve/eve-main.html
Bolas, Mark T. 1994. Human factors in the design of an
immersive display. IEEE Computer Graphics and
Applications January: 55-59.
Brooks, Fred. 1995. Panel Discussion on March 13, 1995 at
the IEEE Virtual Reality Annual International Symposium
in Research Triangle Park, North Carolina.
Dennehy, Michael. Unknown. Encyclopedia of Virtual
Environments. World Wide Web URL:
http://www.cs.umd.edu/projects/eve/eve-articles/I.D.2.c.DirectManipulation.html
Foley, J., A. van Dam, S. Feiner, and J. Hughes. 1992.
Computer Graphics Principles and Application, second
edition. Reading, Massachusetts: Addison-Wesley.
Gomez, Daniel, Grigore Burdea, and Noshir Langrana. 1995.
Integration of the Rutgers Master II in a Virtual
Reality Simulation. In IEEE: Proceedings of the Virtual
Reality Annual International Symposium in Research
Triangle Park, NC, March 11-15, 1995, IEEE Computer
Society Press. Washington: IEEE Computer Society Press,
198-202.
Hsu, Jack. Unknown. Encyclopedia of Virtual Environments.
World Wide Web URL:
http://www.cs.umd.edu/projects/eve/eve-articles/I.D.1.a.ActiveInteraction.html
Lane, Corde, and Jerry Smith. Unknown. Encyclopedia of
Virtual Environments. World Wide Web URL:
http://www.cs.umd.edu/projects/eve/eve-articles/I.C.ForceTactile.html
Lane, Corde. Unknown. Displays. Encyclopedia of Virtual
Environments. World Wide Web URL:
http://www.cs.umd.edu
/projects/eve/eve-main.html
Sowizral, Henry. 1995. Tutorial: An Introduction to Virtual
Reality. Virtual Reality Annual International
Symposium.
Stanney, Kay. 1995. Realizing the Full Potential of Virtual
Reality: Human Factors Issues That Could Stand in the
Way. In IEEE: Proceedings of the Virtual Reality Annual
International Symposium in Research Triangle Park, NC,
March 11-15, 1995, IEEE Computer Society Press.
Washington: IEEE Computer Society Press, 28-34.
Steinmetz, Joe, and Glen Lee. Unknown. Encyclopedia of
Virtual Environments. World Wide Web URL:
http://www.cs.umd.edu/projects/eve/eve-articles/III.A.2.Auditory.html
Stereo is dead. The AudioReality story. [1994]. Palo Alto,
California: Crystal River Engineering.
Stone, Robert J. 1993. Virtual reality systems. Edited by
R.A. Earnshaw, M.A. Gigante, and H. Jones. Virtual
Reality: A tool for telepresence and human factors
research. London: Academic Press.
Tonnesen, Cindy, and Joe Steinmetz. Unknown. Encyclopedia of
Virtual Environments. World Wide Web URL:
http://www.cs.umd.edu/projects/eve/eve-articles/I.B.1.3DSoundSynthesis.html
Tachi, Susumu. 1995. Whither Force Feedback?. In IEEE:
Proceedings of the Virtual Reality Annual International
Symposium in Research Triangle Park, NC, March 11-15,
1995, IEEE Computer Society Press. Washington: IEEE
Computer Society Press, 227.
Watson, Benjamin A., and Larry F. Hodges. Unknown. Using
Texture Maps to Correct for Optical Distortion in
Head-Mounted Displays. World Wide Web URL:
http://www.cc.gatech.edu/gvu/virtual/VirtualEnvironments.html"
Wilson, Michael, and Anthony Conway. 1991. Enhanced
Interaction Styles for User Interfaces. IEEE Computer
Graphics and Applications March: 79-89.
|
