How We Gain Immersive Literacy from Computer Games

Explains the relationship of gamification to immersive literacy

2018 Mixed reality with the Microsoft HoloLens version 1 (image: Erik Champion).

In the previous webpage I distinguished immersive literacy from digital literacy even though it is a subset of digital literacy, creating immersive environments requires a specific understanding of virtual, artificially constrained environments AND understanding how other people would use, understand and explore those virtual environments, because immersive environments are not pale copies of the real-world.

To be immersive, they need to be interactive, overarching, and engaging. But their design requires an understanding of user-driven embodied interaction, usually without the normal clues and cues we receive from the real world around us. So immersive literacy requires not only an understanding of clues (like a detective), but an understanding of how the spatial and conceptual world fits together in appearance and mechanics (more like a stage director or a novelist who designs builldings) and how technology enables that to happen (interfaces, screens, hardware). For example, in Figure 1, the HoloLens allows you to move digital 3D objects or movies imposed over your sight by clicking on them with your fingers. But if your fingers are not at right angles to the HoloLens camera, it does not recognize them (and most people click away not side one with their fingers).

In the real world, architects can visualise buildings from simple 2D plans. In digital games, experienced gamers can work out how to navigate, where to find and manipulate objects, and how to perform tasks faster and more efficiently than non-gamers. They are experienced with the special affordances or signifiers (Norman, 2018), cues, and clues of games and virtual worlds. In that sense, they have higher levels of immersive literacy. However, they may not necessarily gain a deeper understanding of the content (Champion, 2005).

Navigating and orienting yourself in earlier virtual environments could often lead to nausea. More recent head-mounted display environments can still cause motion sickness and confusion (Heffernannov, 2014, Lewis, 2016, Mason, 2017). There are far less sensory cues to the body and virtual environments can lack the navigational cues of the real world (there is usually no smell or touch, kinaesthetic or proprioceptive cues).

Also, learning how to engage people in virtual environments is not easy, because designers often under-estimate the importance of mechanics (Figure 1), of creating challenging but also rewarding interaction that leads to a goal, that is not too easy or too difficult. So there is also immersive literacy as a designer, creating engaging and rewarding (not sickening) immersive environments, based on the knowledge of how people move in immersive (virtual) space.

Figure 1: Schematic by Dr Juan Hiriart and Erik Champion for designing games for history and heritage.

References

CHAMPION, E. 2005. Place Space and Monkey Brains: Cognitive Mapping in Games and Other Media. DiGRA '05 - Proceedings of the 2005 DiGRA International Conference: Changing Views: Worlds in Play [Online], 3. Available: http://www.digra.org/digital-library/publications/place-space-monkey-brains-cognitive-mapping-in-games-other-media/ [Accessed 28 May 2021].

HEFFERNANNOV, V. 2014. Virtual Reality Fails Its Way to Success [Online]. online: NY Times Magazine. Available: https://www.nytimes.com/2014/11/16/magazine/virtual-reality-fails-its-way-to-success.html [Accessed 10 July 2017].

LEWIS, T. 2016. Samsung Gear VR: Virtual Reality Tech May Have Nasty Side Effects [Online]. Online: Livescience. Available: https://www.livescience.com/49669-virtual-reality-health-effects.html [Accessed 22 July 2017 2017].

MASON, B. 2017. Virtual reality has a motion sickness problem-People prone to nausea may opt out of immersive experiences. ScienceNews [Online], 191. Available: https://www.sciencenews.org/article/virtual-reality-has-motion-sickness-problem [Accessed 27 July 2017].

NORMAN, D. 2018. Signifiers, not affordances. jnd.org [Online]. Available: http://jnd.org/dn.mss/signifiers_not_affordances.html [Accessed 9 April 2008].