An Objective Surrogate for Presence: Physiological Response


Michael Meehan
Computer Science Department, Sitterson Hall
University of North Carolina
Chapel Hill, NC 27599-3175 USA
meehan@cs.unc.edu
Phone: (919) 962-1979
Fax: (919) 962-1799
 Summary
This paper discusses the following:
    - Significant experimental support for an objective surrogate for presence: physiological response. 
    - Reported, behavioral, and physiological presence vary with the number of exposures to a virtual environment.
Abstract
    This talk discusses our investigation into an objective presence measure. In our experiment, we investigated the correlation among a post-experiment presence questionnaire, human peripheral response (electrodermal activity and skin temperature), and a behavioral-presence questionnaire. Our findings show that the measures do correlate and have use as a potential objective surrogate for subjective presence measures. We also found a significant decrease in presence-evocation over multiple exposures to the same virtual environment.

Introduction
    Since the mid 1980’s, researchers have been laboring to make virtual environments (1) work, (2) work well, and (3) do demonstrably useful things [Brooks, 2000] . One of the keys to making a virtual environment work well is inducing presence in the users: make users feel they are  in the VE so that the experience is more like a place visited and not just a series of pictures seen [Slater, 1995b].
    The major findings in this field include that presence is enhanced by incorporating a virtual representation of the user into the environment (a 'virtual body') [Slater, 1993b; 1994b; 1995b], by the presence of haptic and tactile feedback corresponding to the virtual environments [Ho, 1998], [Dihn, 1999], by portraying the environment with a wider field of view [Hendrix, 1996], by having users move with actual limb motion [Slater, 1995b] [Usoh, 1999], more realistic physical simulation [Uno, 1997], audio and olfactory stimuli corresponding to events in the VE [Dihn, 1999], by environment portrayal in stereo (as opposed to mono) vision [Freeman, 1997], [IJsselsteijn, 1998], and by inclusion of dynamic shadows of objects in the virtual environments [Slater, 1995a]. Also, it has been found that people whose primary experiential system is visual (they experience the world primarily via images and not audio or kinesthetic stimuli - established by counting the number of visual, audio, and kinesthetic expressions used by subjects in an essay) tend to experience a higher sense of presence in virtual environments [Slater, 1993b 1994a].
    In presence studies, two types of presence measures have been used: questionnaires [Slater, 1993b; 1993a; 1994a; 1994b; 1995a; 1995b; 1996; 1998] [Witmer, 1998] [Regenbrecht, 1997; 1998] [Dihn, 1999] [Uno, 1997; Usoh, 1999] [Welch, 1996] [Steed, 1999] and behavioral measures [Slater, 1995b] [Freeman, 1998] [Usoh, 1999]. The reliability of these measures is being established via reuse. However, since presence is difficult to define and no objective measure of the phenomenon exists, the validity of these measures is occasionally called into question. To examine the validity of the behavioral and subjective measures, we propose a third, objective measure: physiological response.
    For clarity in examination of the previously used presence measures, we break the concept of presence into three components:   subjective, behavioral, and physiological presence. Subjective presence is the subject’s  reported sense of ‘being there’  in the virtual environments and feeling more like the world portrayed was a place visited and not just a series of pictures seen . Behavioral presenceis behaving and acting in an environment in a manner consistent with human response to similar real situations.  Physiological presence is responding physiologically to the environment in a manner consistent with human response to similar real situations.

Figure 1. Side view of the virtual pit.

    To measure subjective presence we use the University College London (UCL) presence questionnaire [Usoh, 1999]. To assess behavioral presence we use two measures: 3 behavioral-presence questions from the UCL presence questionnaire and a behavioral scoring system (post-experiment from videotape) that is designed to asses behavioral reaction to heights. We believe that if participants behave as if they are near a precipice, they are more present. To measure physiological presence, we monitor heart rate, electrodermal activity, and skin temperature (these measures have been seen to vary due to fear: heart rate increases, finger skin temperature drops, and skin conductivity increases [Weiderhold, 1998]). We will use the ProComp+ tethered telemetry system (Thought Technology Ltd.) to monitor participants’ physiological responses.
   Other questionnaires that were used are a simulator sickness questionnaire [Kennedy 1993], a fear of heights assessment questionnaire [Cohen 1977], and an oral debriefing sheet.
      Using these presence measures, we are investigating both the correlation among   physiological, behavioral, and subjectivepresence in virtual environments and the degradation of presence over multiple exposures to a virtual environment.
    Our hypothesis is that the three types of presence will correlate. We expect that behavioral and physiological presence will have the highest correlation since the subjects will behave and physiologically respond in the same time frame, whereas their subjective scores are obtained post-experiment.
    We also hypothesize that presence evocation, as measured by the three presence measures, will decrease over multiple exposures to the virtual environment due to desensitization and that there will be a sensitization partial-correction between days (i.e. on the second day, the system will evoke more presence then at the end of the last session).
The Experiment
    The University of North Carolina (UNC) Hybrid Reality virtual environment (HyRe) is frequently reported by users to be a compelling virtual environment. HyRe, which portrays two rooms, one of which depicts a 20-foot drop-off, was used in Usoh’s Walking Experiment [Usoh, 1999]. Randy Pausch said, "There is this VR demo, and then there are all others". Jaron Lanier said, "This is what I always thought it would be like." We believe that the difference in presence-induction between the this and other systems is due to technological improvements: more visual realism (radiostiy as textures, sphere mapping, reflections, dynamic objects), faster update rate (generally 30FPS), lower movement lag (estimated at 100 msecs), and inclusion of static haptics (a rough physical model corresponding to the objects in the virtual environment). We do not know, however, which of the technological advances are important for presence-induction and which are not. We believe that, as a VE system betters – better image quality, increased update rate, reduced lag, inclusion of static haptics – up to our technological limits, believability of and presence induced by the environment will increase. As an initial step in our research, we are investigating an objective measure of presence and are examining the degradation of presence over multiple exposures to a virtual environment: the user is exposed to the same environment each time.
   The virtual environment system consists of three parts: the tracking, rendering, and display systems. The tracking was handled by the Hiball Tracker, a large-area optical tracking system designed and built at UNC Chapel Hill. It allows users to roam over a 4m x 10m area with 2000 updates/second; 0.5mm position error; and 0.02 degrees of orientation error. The rendering software was written at UNC Chapel Hill and runs on a 32-processor Reality Monster from Silicon Graphics. The head-mounted display is a V8 display from Virtual Research.
    The virtual environment consisted of two rooms: the training room and the virtual pit room. The training room, in which the subjects are trained to navigate in the environment and pick up and move a virtual book, contains a number of virtual objects including a television, a mirror, a reproduction of the Mona Lisa, a plant, two chairs, and a French door with a view of the outside. The other room consisted of a virtual ledge above a 20-foot (virtual) drop to a room below containing living room furniture (the virtual pit room). There was also a chair on the wooden ledge on the far side of the room. A side view of the environment is pictured above.
    The real environment has a 1.5-inch wooden ledge that corresponds to the virtual ledge in the virtual pit room, Styrofoam walls corresponding to the walls in the virtual pit room, and a chair corresponding to the chair on the virtual ledge.
    During each task of the experiment (3 per day), participants spend approximately two minutes in a training room, practicing navigating and moving a virtual book from a table to a chair. The subject is then instructed to take the virtual book to the chair on the far side of the virtual pit room. The subjects never see the real room with the 1.5" wooden ledge (hidden pre and post by a curtain), and we made an effort to give the same instructions each time a subject performed a task. 

Table 1. Variables of interest.
Variable  Description
Percentage change in skin conductance level (%D SCL) Percentage difference in skin conductance on the participant’s left hand between virtual pit room and pre-experiment training/ baseline session. Increased fear causes %D SCL increase
Percentage change in finger skin temperature (%D ST) Percentage difference in finger skin temperature between virtual pit room and pre-experiment training/ baseline session. Increased fear causes %D ST to decrease.
UCL presence (P) Number of high responses (6’sand 7’s) to 7 presence questions from post-experiment questionnaire [Usoh, 1999].
UCL-based subjective behavioral presence (BP) Number of high responses (6’sand 7’s) to 3 behavioral presence questions from post-experiment questionnaire [Usoh, 1999]. The UCL questionnaire originally scored this on a [0,5] basis. Due to a clerical error, we omitted one question. We also omit the addition of "did the participant cross the virtual precipice" from this measure so that we will be able to compare this purely subjective measure with our behavioral scoring system, which takes into account whether the participant crosses the virtual precipice.
UCL ease of locomotion (LO) Number of high responses (6’sand 7’s) to 3 ease-of-locomotion questions from UCL questionnaire [Usoh, 1999].
Level of computer game play Response to the question: "I play computer games" [0=not at all, 7=very much].
    The experiment (n=10, average age 24.4, s 7.8; 7F, 3M; 2 graduate, 5 undergraduate, 3 professional; $6/hour) investigates the degradation in presence over multiple exposures to a virtual environment. Our hope is to verify that HyRe induces physiological, behavioral, and subjective responses and to ensure that the decrease in response over a small number of trials (3 trials per day on four different days) does not hinder interpretability.

Results and discussion
Correlation among presence measures
Table 2. Spearman correlation among the presence measures.
r2
P(Ha: |r|>0)		 P BP %D SCL %D ST
P 1.000 0.5452 0.1836 0.07068
  <.0001 0.0571 0.4673
BP 0.5452 1.000 0.3435 -0.1350
<.0001   0.0003 0.1653
%D SCL 0.1836 0.3435 1.000 0.0488
0.0571 0.0003   0.6446
%D ST 0.07068 -0.1350 0.0448 1.0000
0.4673 0.1635 0.6446  

    Not all of the data has been analyzed, but our findings so far include a high and significant correlation among the various presence measures: UCL Presence (P), UCL Subjective behavioral presence (BP) and percentage change in skin conductance level (%D SCL) (Table 2). Within the next few weeks (before the Presence 2000 workshop) we will have analyzed the remaining two measures: heart rate and behavioral presence scored from videotape.
    Table 2 shows the Spearman correlation among the various presence measures. We chose the Spearman correlation because of its suitability to non-linear relationships. This matches the relationships we have observed and fits the model of asymptotic physiological reaction to stimuli [McMurray, 1999].
    Percentage change in skin conductance level (%D SCL) is calculated as average skin conductance level between time in virtual training room (~1-2 minutes) and time in virtual pit room (~1 minute). %D SCL is significantly correlated with both UCL presence (P) and UCL behavioral presence (BP), with the more convincing correlation between BP and %D SCL. This follows our original hypothesis: the three types of presence measures (behavioral BP; physiological %D SCL; and subjective P) will correlate with the higher correlation between behavioral and physiological presence.
    Similar to the findings in [Usoh, 1999], BP and P show a high degree of correlation.
    Percentage change in skin temperature (%D ST) shows no correlation with any of the other presence measures. While we believe that skin temperature is a valid measure for fear response in virtual environments, we believe that (1) we did not allow for a long enough time for the effect and (2) other factors may have contributed to the change in finger skin temperature. First, temperature change in finger skin is gradual. It can 2-5 minutes for the effect of finger skin temperature to be completely realized. This is partially due to latency in maximal change in skin temperature and partially due to latency in heat transfer to the temperature probe [McMurray, 1999]. Second, other factors, such as locomotion, may have affected ST. We have not given up on finger skin temperature as an objective correlate of presence. We plan to modify our experimental design to better accommodate for the slower response of skin temperature.
Table 3. Partial Spearman correlation corrected for gender.   Table 4. Partial Spearman correlation corrected for level of computer game play.
Gender-partial r2
P(Ha: |r| > 0)		 P BP %D SCL Comp game-partial r2
P(Ha: |r| > 0)		 P BP %D SCL
P 1.000 0.6899 0.2471 P 1.000 0.7254 0.3175
  <.0001 0.0103   <.0001 0.0009
BP 0.6899 1.000 0.3421 BP 0.7254 1.000 0.3257
0.0103   0.0003 <.0001   0.0006
%D SCL 0.2471 0.3421 1.000 %D SCL 0.3175 0.3257 1.000
0.0103 0.0003   0.0009 0.0006  
    Tables 3 and 4 show the Spearman correlation among P, BP, and %D SCL. There was no significant correlation with %D SCL. Correcting for gender, the correlation and probability among P, BP, and %D SCL, all improve. Correcting for level of computer game playing, all of the measures, except the correlation between BP and %D SCL, improve more than for gender-correction. As in [Usoh, 1999] we found a high correlation among computer game usage and gender: -0.70 (p < 0.05; n=12).
    After correcting for level of computer game playing, all Spearman correlations among BP, P and %D SCL are significant at the 0.1% level and are above 0.30. This provides significant support for our hypothesis of correlation among the presence measures and supports the idea of using skin conductance level as an objective correlate of subjective and behavioral presence in virtual environments.

Table 6. The change of %D ST over exposures corrected for computer game usage.
Model
Pr>F: 0.0002		 Estimate Type I
Pr > F		 T-test
Intercept -2.6905   <.0001
Session 0.5587 0.0014 0.0001
Session*task -0.1205 0.0180 0.0204
Computer game usage 0.1689 0.0329 0.0329
Degradation of presence measures over subsequent exposures
    Table 5 shows the change in %D ST over session (day) and session*task interaction (task = nth exposure on a day). Correcting for computer game usage significantly improves the model. The model without comptuer game usage is also significant. The addition of task to this model approached significance.
    %D ST decreases in amplitude (% decrease in skin temperature diminishes) over subsequent exposures with an upward correction between days. This follows our original hypothesis that %D ST (and the other presence measures) will decrease with each exposure with an upward correction from one day to the next.     Table 6 shows that, after correcting for gender and ease of locomotion, BP decreases with subsequent exposures with a correction between days. This is similar to the pattern for %D ST and supports the same hypothesis: the presence measure decreases over exposures with an upward correction between days. Males reported lower BP and higher LO was associated with higher BP.
    Table 7 shows that, after correcting for ease of locomotion, P decreases over exposures. In this case, though, there is no significant difference in P among the exposures on a given day. As with BP, ease of locomotion is positively associated with P.
    %D SCL did not show significant degradation over subsequent exposures in any model.
Table 6. The change of BP over exposures after correcting for gender and ease of locomotion (LO).   Table 7. The change of P over exposures after correcting for gender and ease of locomotion (LO).
Model
Pr>F: <.0001		 Estimate Type I
Pr > F		 T-test   Model
Pr>F: <.0001		 Estimate Type I
Pr > F		 T-test
Intercept 1.7759   <.0001 Intercept 2.4740   <.0001
Session  -0.2103 0.4451 0.0077 Session  -0.2728 0.4183 0.0473
Task -0.2128 0.3254 0.0404 Ease of locomotion 1.1737 <.0001 <.0001
Gender M=-0.6194 0.9126 0.0030  
Ease of locomotion 0.4752 <.0001 <.0001

Conclusions
    We have significant experimental support for the hypothesis that physiological measures can be used as an objective correlate of presence. Percentage change skin conductance level provided the best support for this hypothesis with high correlation with both UCL presence and UCL-based behavioral presence measures. The correlation improves if we control for gender or computer game usage.
    Percentage change skin temperature did not correlate well with any other presence measure. We believe this is because the experiment did not allow enough time for maximal skin temperature change: 2-5 minutes. Participants were only in the arousal condition for approximately one minute. In our future studies, we will ensure experimentally that sufficient time is allowed for monitoring skin temperature.
    We also found significant support for our hypothesis that there would be a decrease in presence over subsequent exposures to the same virtual environment. Presence, reported behavioral presence, and percentage change in skin temperature all exhibited this behavior. %D ST and BP both showed a decline in presence evoked over subsequent exposures on a single day and had an inter-day upward partial-correction. P showed a decrease in presence elicited over subsequent days, but did not have significant intra-day changes.
    For now, we are using a fear-inducing virtual environment to induce presence and physiological reaction in our subjects, but we hope that with additional investigation and refinement of our approach, we will be able to utilize these measures in a broader range of virtual environments.

Acknowledgements
    I would like to thank the Hybrid Reality team at UNC-CH, especially Dr. Frederick P. Brooks, Jr. and Mary Whitton, for their help, support, and ideas. I would also like to thank Drs. Mel Slater and Martin Usoh, who have been integral in making this work possible at UNC.
    This work was partially funded NIH National Center for Research Resources number RR02170. Intel generously donated equipment used in this research.

Bibliography
Brooks, F. P. (2000). Unpublished grant proposal, University of North Carolina.
Cohen, D. C. (1977). "Comparison of self-report and overt-behavior procedure for assessing acrophobia." Behavior Therapy 8: 17-23.
Dihn, H. Q., N. Walker, et al. (1999). Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. Virtual Reality, Houston, Texas, USA.
Freeman, J., S. E. Avons, et al. (1997). Effects of stereo and motion manipulations on measured presence in stereoscopic displays. ECVP, Helsinki.
Freeman, J., S. E. Avons, et al. (1998). Behavioral realism as a metric of presence. BT Presence Workshop.
Hendrix, C. and W. Barfield (1996). "Presence within virtual environments as a function of visual display parameters." Presence: Teleoperators and Virtual Environments 5(3): 274-289.
Ho, C., C. Basdogan, et al. (1998). An experiment on the influence of haptic communication on the sense of being together. BT Presence.
IJsselsteijn, W. A. and H. d. Ridder (1998). Measuring temporal variations in presence. BT Presence Workshop.
Kennedy, R., N. Lane, et al. (1993). "A simulator sickness questionnaire(SSQ): A new method for quantifying simulator sickness." International Journal of Aviation Psychology 3(3): 203-220.
McMurray, D. R. (1999), Director of Applied Physiology lab, University of North Carolina. Personal Communication
Regenbrecht, H., T. Schubert, et al. (1998). "Measuring the sense of presence and its relations to fear of heights in virtual environments." International Journal of Human-Computer Interaction 10(3): 23-250.
Regenbrecht, H. T. and T. W. Schubert (1997). Measuring presence in virtual environments. Human Computer Interface International, San Fransisco, USA.
Slater, M., V. Linakis, et al. (1996). Immersion, presence and performance in virtual environments: An experiment with tri-dimensional chess. ACM Virtual Reality Software and Technology (VRST).
Slater, M., A. Steed, et al. (1998). "The influence of body movement on subjective presence in virtual environment." Human Factors and Ergonomics Society 40(3): 469-477.
Slater, M. and M. Usoh (1993a). Presence in immersive virtual environments. Virtual Reality Annual International Symposium (VRAIS).
Slater, M. and M. Usoh (1993b). "Representations systems, perceptual position, and presence in immersive virtual environments." Presence: Teleoperators and Virtual Environments 2(3): 221-233.
Slater, M., M. Usoh, et al. (1995a). The influence of dynamic shadows on presence in immersive virtual environments. Virtual Environments '95.
Slater, M., M. Usoh, et al. (1994a). "Depth of presence in virtual environments." Presence: Teleoperators and Virtual Environments 3(2): 130-144.
Slater, M., M. Usoh, et al. (1994b). Steps and ladders in virtual reality. ACM Virtual Reality Science and Technology (VRST). G. S. a. D. Thalmann. New York, ACM: 45-54.
Slater, M., M. Usoh, et al. (1995b). "Taking steps: The influence of a walking metaphor on presence in virtual reality." ACM Transactions on Computer Human Interaction (TOCHI) 2(3): 201-219.
Steed, A., M. Slater, et al. (1999). Leadership and collaboration in shared virtual environments. Virtual Reality, Houston, Texas.
Uno, S. and M. Slater (1997). The sensitivity of presence to collision response. Virtual Reality Annual International Symposium (VRAIS).
Usoh, M., K. Arthur, et al. (1999). Walking > walking-in-place > flying in virtual environments. SIGGRAPH.
Weiderhold, B. K., R. Gervirtz, et al. (1998). "Fear of flying: A case report using virtual reality therapy with physiological monitoring." CyberPsychology and Behavior 1(2): 97-104.
Witmer, B. G. and M. J. Singer (1998). "Measuring presence in virtual environments: A presence questionnaire." Presence: Teleoperators and Virtual Environments 7(3): 225-240