4 Chapter 5 – Augmented, Virtual, and Mixed Reality

Overview

If you look back through history and document people’s reactions to newly emergent technologies, you will see a predictable pattern: there were some who saw the new thing for all its promising capabilities; others insisted that it posed extraordinary dangers.

Naysayers said that the first electrified subway system was an underground death trap. “People were not made to travel underground!” it was said. But then, when people finally rode on the subways, they found that it was actually quite a nice experience.

Motion picture films, in their first decades as an entertainment medium, were believed to cause juvenile delinquency and depraved behavior based the imitative behavior theory. For example, in 1913, “After [censor Major Funkhauser] banned some films of people dancing (the turkey trot and the tango), he said that ‘the objection is not based so much upon these pictures in themselves, but upon the effect they would have on thousands of young people.'” (Denby, D., 2016). Communities all over the country mandated censorship despite arguments about Free Speech. This practice dissolved through a combination of legal decisions and changes in social standards that were less paternalistic.

In the 1950s, comic books were targeted as a corrupting influence on young people. Piles of comic books were burned, fostered by the belief that “…Hitler was a beginner compared to the [influence of the] comic-book industry.” (Werther, F., 1954).

In this chapter, we examine the promise (and outrage) held in one of the newest media technologies to become accessible to the general public: Augmented, Virtual, and Mixed Realities (AR/VR/MR), collectively referred to as reality technologies.

The readings and media in this chapter touch upon the basic capabilities of reality technologies and then go beyond the “wow factors” to explore their potentially negative collateral effects.

Key Terms

As a relatively new and evolving field of study, the terminologies used to describe AR/VR/MR are often changing.  The following definitions provide basic descriptions, though there may be additional nuances to them under certain conditions.

Augmented Reality (AR) – “…taking digital or computer generated information, whether it be images, audio video and touch or haptic sensations and overlaying them [onto] … a real-time environment” (Kipper, G., 2012, p.18).

Virtual Reality (VR) – “A completely artificial digital environment that uses computer hardware and software to create the appearance of a real environment to the user” (Kipper, G., 2012, p. 22).

Mixed Reality (MR) – “…where real world objects or people are dynamically integrated into virtual worlds to produce new environments and visualizations where physical and digital objects co-exist and interact in real time” ( Retrieved 12-17-2017 from: http://www.realitytechnologies.com/mixed-reality ). Microsoft offers a more detailed illustration of MR with graphical and video representations of MR across the full spectrum of reality technologies.

Reality Technologies – A blanket statement used to describe all of the varieties of AR/VR/MR.

Copyright Microsoft, inc. Retrieved 12-17-2017 from: https://developer.microsoft.com/en-us/windows/mixed-reality/mixed_reality. The graphic is based on Milgram’s Virtuality Continuum (Milgram, P, 1994)

Haptic Feedback – The experience of vibrations and other sensory experience associated with a VR experience. Examples of this are found in surgical training.

What should you be focusing on?

Your objectives in this module are:

• Identify the elements in AR/VR/MR sensory experiences that differentiate them from other forms of media.
• Identify the ways in which immersive technologies can produce positive impact on the human experience.
• Evaluate the collateral risks and challenges that emerge from the proliferation of immersive technologies such as:
  • How does immersion in artificial environments adversely affect behavior in the real world, especially if the virtual world is more emotionally and physically fulfilling?
  • Are behaviors or thoughts in the virtual world subject to the same moral, ethical, and legal codes as those in the real world? Can “thought crimes” be considered real crimes?

Readings & Media

Examples of various reality technologies:

You are in an unfamiliar airport with thousands of people swarming around you. You can’t read or speak the local language. You have no idea how to get to the check-in counter. Then you hold up your AR-enabled smartphone and load an app that maps information onto the airport interior. The information floats in front of you so that you can navigate directly to your destination.

You are repairing a complex piece of machinery and need assistance. You put on an AR device and it maps animated information onto the device to guide you step-by-step through the process.

Video: Augmented Reality Training Demonstration – by Scope AR using the Epson Moverio BT-100

https://youtu.be/eVV5tUmky6c

You are scheduled to be deployed to a complex military theatre of action where it would not be possible to recreate the engagement environment in the physical realm. You and your team put on a VR apparatus that produces a simulated representation of the engagement environment. The VR system enables your team to practice and refine your strategy through active participation and playback analysis.

Video: Virtual Reality: The Future Of Military Training

As your 2-year-old child grows, you sense something isn’t quite right. She is withdrawn, doesn’t make eye contact, and is not communicative. She is diagnosed with a form of autism. You are frightened about how she will be prepared to get by in the world. As a parent, you wish there was a way you could see the world through her eyes to help her cope as she grows up.

So you locate a VR resource that helps you to understand how autistic people experience stimuli in the real world so that you can respond to your child’s reactions and needs.

Video: Autism TMI Virtual Reality Experience (Note: This is recorded so that it can be viewed in 360 VR using a Google Cardboard device. You will not be able to get the full experience from watching this video without wearing the device, but consider the immense value in having this kind of resource freely available on YouTube.)

You feel alienated from your peers and feel desperately lonely. You don’t like how you look, you are sometimes bullied, and you feel detached and withdrawn from the world. You have no place to go in your community that makes you feel like you belong to something meaningful.

So you put on your VR device and it transports you to a virtual world where you can appear to be anything you want – a different person, an animal, an alien. It gives you a newfound sense of freedom and comfort to interact with others in realtime, to make friends, do things together, or even change genders. Other participants seek you out to dance, chat, or have sex with you – an experience you believe could never happen in real life. The virtual world is beautifully fulfilling in ways the real world cannot be.

As an educator for medical students, you want to enable them to experience empathy with patients. Embodied Labs is a company inspired by Carrie Shaw, a medical student, to create VR embodied experiences. Read more in NPR’s feature article, “Virtual Reality Helps Hospice Workers See Life And Death Through A Patient’s Eyes“.

    Required     Ethical issues and codes of conduct

Article: Frontiers in Robotics and AI – “Real Virtuality: A Code of Ethical Conduct. Recommendations for Good Scientific Practice and the Consumers of VR-Technology” by Michael Madary and Thomas K. Metzinger. This is a formal research article that contains more than what you will need for this module. Please skim the initial sections if you like, but focus in detail on the following sections:

• Beginning with “Context-Sensitivity All the Way Down” and ending before “The Research Ethics of VR“.
• Beginning with “Risks for Individuals and Society” and ending before “Summary“.

Here is an excerpt relevant to our studies which bears a connection between the psychological effect of social media participation and the implications of “need” within reality technologies:

…immersive VR introduces new and dramatic ways of disrupting our relationship to the natural world. Likewise, the newly created “need” to interact using social media will become even more psychologically ingrained as the interactions begin to take place while we are embodied in virtual spaces….(Madary, M., & Metzinger, T., 2016, p. 2)

    Required     VR and the rules of virtual sex

Article:  The New York Times:  “Virtual Reality Gets Naughty”.  Please note that this article refers to aspects of sexuality in language which some may find unexpectedly frank for an academic activity. There is no nudity or sexually explicit content shown here, but the subject matter touches upon aspects of sex-oriented technologies which some may find uncomfortable. It is, however, a critical exploration into the areas where reality technologies are implemented.

Krueger explores several dimensions to the “virtual companionship” technologies: online pornography, robotic companionship, sex therapy, sex education, and PTSD therapy.

Optional: Supplemental resources that are relevant to AR/VR

COMM601 alumnus Peter Riendeau produced his undergraduate Capstone project as an online e-book tracing the impact of AR/VR in the field of Marketing. Mr. Riendeau had worked in the field of sensory technology for about 30 years and combined his work experience with his Marketing studies to produce this open document.

Educause Review: “VR and AR: The Ethical Challenges Ahead” describes the issues of using immersive technologies in the field of education.

Firefox has developed a mixed reality Web browser called Firefox Reality.

Mashable:  “Virtual reality porn is here. Get ready for the first coming” by Chelsea Stark. This article uses explicit language in its reporting of the VR technologies now being tested in the pornograhy industry. It includes descriptions of other remotely controlled devices such as teledildonics. The purpose in offering this article is to surface the implications of self-centered sexuality in a social context where the rules of consent differ from those in the real world.

DigitalBodies.net: VR, AR, Wearables and the Future of Learning. Explore how AR/VR is being used in the field of education.

References

Kipper, G., & Rampolla, J. (2012). Augmented Reality : An Emerging Technologies Guide to AR. Rockland, MA: Syngress.

Milgram, Paul & Kishino, Fumio. (1994). A Taxonomy of Mixed Reality Visual Displays. IEICE Trans. Information Systems. vol. E77-D, no. 12. 1321-1329. Retrieved 12-17-2017 from: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.102.4646&rep=rep1&type=pdf

Wertham, F. (1954). Seduction of the innocent. New York: Rinehart.