Your brain on virtual reality

The psychology of virtual reality.

The buzz around Virtual Reality, VR, is everywhere. The technological advances are starting to make scenarios reserved for sci-fi a reality. Spanning the likes of The Matrix’s fully immersive worlds, Star Trek’s holodeck and Minority Report’s personalised and targeted advertising.
Described by Forbes as ‘technology's next big wave’1, Google, Facebook and now Apple are investing heavily in the sector. With Google’s cardboard VR simulators and Facebook’s purchase of Oculus Rift last year, VR devices are starting to make their way into our homes and classrooms2. Undoubtedly, it’s an exciting field.

What is VR?

In it's simplest definition, VR is an immersive and interactive experience based on real-time 3D graphics. They are experienced through a headset, with Google cardboards on one end of the scale and a fully tracked and integrated headsets on the other.

Designing VR with psychology in mind

How to create immersion and presence in VR?

As you dip into VR literature, ‘immersion’ and ‘presence’ always pop up. But, what exactly are they and how can we design to optimise for them?

Studies into VR and VR professionals alike agree that the most convincing VR worlds are those which appear seamless. Seamless worlds are a mixture of what is termed immersion (the ‘technological quality of the media’) and presence (the ‘technological quality of the “experience of being there”’). They both play a role in maintaining the fabric and believability of a virtual world.

A lot has been written about how to effectively use immersion and optimise for presence. The model by the psychologist Wirth3 is the clearest and structures presence as a two step process:

  1. “The user must draw upon spatial cues” which create the appearance of a “plausible space”, creating a “sense of self-location”.
  2. The user must feel like they are located in that space. Which in turn presents the possibility that they can act and interact within that space.

What are the cues of presence?

When creating VR worlds, Wirth’s model gives the framework for determining spatial cues. They include:

  • Static monocular cues.
    These are elements that help give a perception of depth within the VR world, including the use of perspective, relative size of objects, and use of textures.4 Here's a great article that explains it in full.
  • Motion cues.
    The movement of objects, environments and the avatar can enhance presence, by utilising the layering effects of motion parallax and cast shadow motion when objects are moving.5
  • Binocular cues.
    I'd highly recommend Boyd's comprehensive paper on Binocular cues.

How to create immersion?

Immersion is based primarily around the application of the technology. Slater and Wilbur’s system outlines the three elements needed to ensure immersion:

  1. A world that offers high fidelity and multisensory simulations.
  2. Which finely maps the actions of a person’s VR body to their physical body.
  3. And uses self-contained plots and narratives in the VR world so that they become detached from what is happening in the external world.

What are the technological features of immersion?

Immersion is determined by a using a balance of 7 features.

  • Tracking level
    Tracking level is the degrees of freedom the user is tracked whilst in the VR world. The better the tracking, the better the sense of self-location and feedback.
  • Stereoscopic vision
    Use of either monoscopic or stereoscopic visuals. Stereoscopic visuals present two slightly different images to each eye, aiding depth perception in the array. Monoscopic on the other hand presents the same image to both eyes, so doesn’t create as immersive experiences.
  • Image quality
    Both the design of the image aesthetics and technological feasibilities affect the image quality. From the realism and detailed features of the display to the technical aspects of flicker rate, lighting types and resolution.
  • Field of view
    How much the user can see in the environment’s visuals.
  • Sound quality
    A world which is devoid of sound is less believable, and a little eerie. Using sound within a VR world increases user feedback and makes the space more believable. There are a number of sound channels and effects that can increase immersion. These include ambient sound of generalised sound effects and sounds that happen within view – for example character voices, footsteps, and specialised sounds.
  • Update rate
    The rate the virtual environment is rendered.
  • User perspective
    Shifting the perspective from 1st person, where a person embodies their avatar, to 3rd person, where a person can view their avatar.

With the dependence on technology as being the main determinant in creating a believable VR world, it leads to the question.

How immersive is enough?

Glancing at the list above, it is only natural to assume that the more technology and immersive quality results a higher level of presence.
A study by psychologists James Cummings and Jeremy Bailenson6 suggests that this is only part of the story. In their study, they tested which elements resulted in higher presence. They found that three main elements produced higher presence. These were high levels of user-tracking, the use of stereoscopic visuals instead of monoscopic and wider fields of view of the visual displays.

Reeves and Nass also found that a ‘high fidelity of visuals have no impact of user attention, recognition or subjective experience.’7 What appears to be important are functional graphics. By all accounts, people are still able to extract the spatial cues from the simplest of graphics as they are from the top of the range effects.

How do you determine ‘place’ in a VR space?

Since the formation of the industry, Architects have been driven by, and crafted the exchange from ‘space’ to the connotation of ‘place’. In a VR context it is the difference between thinking you are in a VR world, and being in the VR world. When you remove physical walls and buildings, how can the same attribution of ‘place’ be created in VR worlds?

Balakrishnan and Sundar8 provide the answer. In their study, they show that ‘media factors’ otherwise known as the affordances of the technology affects a person’s ability to define space and place. Both the design of the VR display and the interactive capabilities cause the shift over from simple space creation to the idea of one being in a place.

Designing for Virtual reality

VR for Gaming. By Maurizio Pesce

Virtual touch in a Virtual world

Touching is intrinsic to human happiness. It allows us to physically connect with the world around us and everything within it.
The ability to touch a virtual world not only makes it more immersive, but it is a huge element of creating a believable and enjoyable world. So, understanding how people interact with each other is crucial in designing a virtual world.

The subtleties of how humans interact will have to be mirrored in a virtual world9.
We wouldn’t want to be left with the VR equivalent of a limp handshake or indeed a misalignment of the wide array of social touching that people integrate on a daily basis. It could easily become awkward.
Psychologists Jeremy Bailenson and Nick Yee examined touch, both in the physical and virtual reality10.
It appears people use a lighter touch when touching different areas of the body. People touch faces with a more gentle touch than the torso or limbs. The same light touch was used when people touched representations of others, with a heavier touch used for inanimate objects.
They also recorded a difference in how male and female avatars were touched. Male avatars were touched with more force than female representations, by both sexes.

Design of a VR avatar

Turning the focus from the VR environment to the design of the avatar bring to light some interesting psychological research.
The avatar is a powerful representation of self. Where it is the equivalent of our VR twin or representation of our choosing, any manipulation in our avatar has psychological impact on our real world selves. Some may argue that we have used a number of avatars in gaming – anyone that created a Mii character can contest to how attached we become to them. The difference being we embody a VR avatar. Being able to both see and feel the cause and effects of our behaviour. Interacting in a VR world and the effect of getting the sensory feedback means we are intrinsically linked to these representations.

This embodiment can have both positive and negative effects on our real world selves. Bailenson’s study into Doppelgangers shows how this psychological bridge can be used for good.11

Want to know more about the psychology of VR?

Download our studio guide


References

  1. Gerber, R. Four Big Technology Trends For 2016 2016

  2. Statt, N. Google is offering its virtual reality classroom system to schools for free 2015

  3. Wirth, W. Hatmann, T. Böcking, S. Vorderer, P. Klimmt, C. Schramm, H. A process model of the formation of spatial presence experiences. 2007

  4. Hu, B. Knill, D. Binocular and Monocular Depth Cues in Online Feedback Control of 3-D Pointing Movement 2011

  5. Ibid

  6. Cummings, J. Bailenson, J. How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence 2016

  7. Ibid

  8. Balakrishnan, B. Sundar, S. Where am I? How can I get there? Impact of navigability and narrative transportation on spatial presence. 2011

  9. Jakobsson, M. Virtual Worlds & Social Interaction Design

  10. Bailenson, J. Nick Yee, N. Virtual interpersonal touch: Haptic interaction and copresence in collaborative virtual environments

  11. Bailenson, J. Doppelgängers – a new form of self? 2012

Header image is re publica 2015 by Jan Zappner. All images used under CC 2.0

Form for thought - A blog about by Dura design psychology and design thinking for graphic designers, web designers, ui designers, ux and illustrators. Looking into the psychology of colour, user behaviour and advertising psychology.