There is an urgent need for filmmakers, immersive theatre practitioners, and game designers to work together. 360 video and virtual reality offer promise, but there is still uncertainty about how to fulfill those promises. Many of the filmmakers stepping into the VR space are already anticipating the other side of the hype cycle: “it only has such an impact now because it is novel” said multiple creators at Sheffield Doc/Fest earlier this year. So what happens when this is no longer fresh and new?
Myron Krueger (1972) Videoplace
The imaginary of VR is heavily figured on the idea of immersion. In fact, while immersive art as an aesthetic quality goes all the way back to the 18th-century panorama, the technological notion of immersion that has proliferated in videogames originated in early VR research. Although VR pioneers such as Myron Krueger did not use the word “immersive” as they began building installations in the 1970s, by the time VR developers were meeting for annual conferences in the early 1990s, immersion was a common by-word accompanying discussions about how to make users feel a sense of “presence”.
Immersion for these VR researchers was a product of sensory information: higher-resolution displays, faster framerates, more elaborate surround sound, etc. Games developers added user agency to this list; the more you feel able to impact the world and/or your place within it, the more immersive it is. More immersion meant more presence. More presence meant that perhaps VR could go beyond spectacle, and grow into the dream of cyberspace.
This dream is a distraction. In VR just as in games, “immersion” turned out to be a cloudy idea with little immediate use for refining the work of developers. The promise of “immersion” in VR does not provide much in the way of creative benchmarks or even a meaningful goal. Any VR project can claim to be “immersive”, without stating anything about the intent or creative strategies at work — it’s something we attribute to the technology.
In the past few years, empathy has become another shallow selling point. A number of VR and 360 projects chronicle the experiences of other people, in the hope that the illusion of immersion or presence in a different place will create a stronger sense of being in somebody else’s situation. Like immersion, because empathy means a few different things at once, VR developers can claim to have provoked it simply by virtue of the technical facilities of telepresence. When empathy is not triggered by VR, it’s the tech that’s blamed for not yet being immersive enough, with the empathic tech horizon always one more feature upgrade away.
In fact, the very interactivity that is often thought to increase immersion can get in the way of empathy, as one person found when user-testing a (non-VR) game about poverty: player agency can give the impression that negative situations are a result of personal decisions. Even when empathy is triggered, the results can be counterproductive, leading either to an antisocial desire for retribution on somebody else’s behalf, or to that overwhelming sense of empathic distress, and in turn to a desire to remind oneself that this is all just Somebody Else’s Problem.
Documentary filmmakers working in VR and 360 seemed a little lost at sea at Doc/Fest in Sheffield earlier this summer. Many of the cinematic techniques that they have grown accustomed to using are not workable when the user is experiencing telepresence. Move the camera, and you risk inducing motion sickness. Frame a shot around one object, and the user might still choose to look the other way. And that’s before you even begin to get into room-scale VR such as the HTC Vive, where the user might choose to walk away from the scene entirely. Finally, the viewer may use any interactive abilities they have to disrupt the action deliberately.
Interventions by architects into film could offer a model for filmmakers moving into spatial media such as VR. Instead of concerning oneself with the frame and the scene, architects such as Steven Jacobs in his study of Alfred Hitchcock films point to an approach that centres on charged objects, familiar spaces made unfamiliar by the uncanniness of a medium, and the juxtaposition of structures arranged near one another.
In My Shoes installed at Union Street coworking space in Sheffield. Photo by Ellie Robinson
Those from the immersive theatre world know a lot more about how to do this: In My Shoes is currently one of the most effective 360 video projects out there, and that’s because it is presented as part of a bigger installation that recreates the tactile and even olfactory sensations experienced by the protagonist of this first-person film. Created by Jane Gauntlett, an artist with experience in immersive theatre, the video is organised not around the framing of shots, but the entrance and exit of characters. In My Shoes makes deliberate use of the minimally-interactive quality of 360 video and the disorienting feeling of wearing a VR headset to foreground a sense of helplessness. You feel that you are in a space that is both familiar and unfamiliar, your body is heavy, and you are running on autopilot.
Documentaries rooted in film traditions, such as Clouds Over Sidra, take a different approach, placing a 360 camera at the site of the documentary and then giving viewers the illusion of being there as a passive observer. The feeling alternates between a dreamlike sense of being present but unseen, and suddenly being aware that a passer-by is staring at you — no, not you, the ball of cameras that had been suddenly brought into their environment.
360 video arranged for Google Cardboard
These projects reflect different understandings of what empathy is and how it is achieved. In My Shoes, as the title implies, is about experiencing a slice of somebody else’s life. Clouds Over Sidra is about temporarily feeling that you are there with someone, as they narrate to you what their life is like. In both, I was left with a sense of powerlessness; In My Shoes implied that the powerlessness belonged to the narrator, but in Clouds Over Sidra this feeling is more troubling: I know that an unprecedented number of people are braving life in refugee camps, and I have increased admiration for the resilience of Syrian refugees, but I don’t know how this knowledge will change my behaviour or political activities.
Attempts to insert some kind of agency through interactive elements are limited, and are complicated by the social politics surrounding documentary and even theatre. Nobody wants to be accused of making a videogame: some filmmakers speaking on panels at Doc/Fest even headed the accusation off before it reached them. “People asked me if I was going to make a videogame and I said of course not, haha”. Any form of interactivity in documentary media has to balance meaningful agency against the need to display the formal and stylistic trappings of a informative media. I get the impression that if one was to step too far into playfulness one risks losing status as a true documentarian.
A concrete example of this is currently touring the UK. Notes on Blindness: Into Darkness is an interactive 360 experience that accompanies the excellent movie Notes on Blindness. Both use audio from the diaries and memoirs of the late theologian John Hull to explore questions about how we experience the world around us and how we cope with the distance between us and the people who we love. The directors have said that their intention with the interactive element was to give people the opportunity to experience more of the diaries than they were able to fit into the film, and I imagined being able to spatially explore a representation of a world without sight, being surprised by the audio recordings that I find along the way. The experience is actually much more static; a predetermined set of recordings are played as digital animations are displayed in various locations, and sometimes I am instructed to stare intently at one of the animations in order for the scene to continue. Imagine a point-and-click adventure, but with only one clickable object on screen at any given time, and no ability to move from room to room. I had no sense that I was unearthing knowledge independently: it was being presented to me, and I was being scolded if I didn’t look at it hard enough.
To make the documentary interactive might require leaning into the feeling of exploring a space as an outsider, rather than always aiming for the up-close personal encounter with a single person’s experience. Owen Vince has offered a model for this in an essay for this publication, forcing game spaces to “disintegrate” in order to better understand them. I can imagine documentaries that give users the ability to see their impact on the environment, and feel their own responsibility for it — rather than simply being an inert camera-head as in Clouds Over Sidra. If giving users the ability to impact an environment through their interactions seems contrary to the need for documentary to present things as they are, consider that any system of interaction is a simulation that necessarily makes an argument about how a space comes to be. In the same way that a highly-interactive simulation of town planning reifies the power of the planner over the agency of local people, a 360 video that makes you feel present in Syria but erases your impact on the local area can actually downplay our global interconnectedness, by reifying the position of the neutral observer.
It would be reductive to say that the answer to all of VR documentary’s problems is game design. Goodness knows, videogames have their own problems to contend with, within and without the VR space. But when filmmakers ask questions such as “how do we guide the viewer to look in one place in particular without having a fixed frame of view?” or “how do we allow people to explore a wealth of information at their own pace?” then the architectural techniques developed for games seem to offer some solutions. The Doc/Fest alternate realities exhibition featured a fantastic example in Tracy Fullerton’s Walden, a 3D videogame in which the player explores an environment to unearth fragments of poetry. Walden is not VR, but it is an immersive experience that recreates how nature is known to those who have dedicated their life to giving it their full attention. It combines the active participation of game design with a sense of peacefully witnessing a part of literary history.
More people in games could take on the challenge undertaken in Walden of helping players to navigate documentary sources. Likewise, more people working in VR documentary could learn spatial storytelling from videogames.
Steven Jacobs (2013) The Wrong House: The Architecture of Alfred Hitchcock
We have explored this idea of an “imaginary” of VR before in a piece by Christopher Goetz:
This article demonstrates the common slippage between the novel experience of three-dimensional graphics and the fantasy of immersive virtual reality. There is an important difference between “immersion” in the general sense of any engaging or engrossing activity—certainly many or most games fall into this category—and “immersion” as a narrative “mode of address,” which occludes the space of spectatorships (hidden behind a “fourth wall”) and offers a window onto diegesis, the world of the story.
Christopher Goetz (2015) “Stepping Out of the Virtual World Paradigm” at Memory Insufficient
Surprisingly, participants in the virtual reality scenarios (which used graphics rather than filming) had less empathy for the victim than in a normal video.
The researchers weren’t 100 per cent sure which elements of virtual reality led to the lower empathy levels, but they hypothesise that we relate strongly to things that look “real”, as opposed to virtual scenarios which simply have a 360-degree purview. Therefore, they write, “findings here suggest that photorealistic graphics should be used in VR simulations to evoke empathy”.
Barbara Speed (2016) “How Virtual Reality Could Combat Compassion Fatigue” in The New Statesman
[…] the empathy effects were cancelled out by the effects of feeling personal agency. Even more troubling is the fact that the game actually lead to more negative attitudes when the participants were low in meritocracy beliefs (and likely to feel positively toward the poor). Playing the game convinced them that poverty was personally controllable, turning their positivity into negativity.
Gina Roussos (2015) “When Good Intentions Go Awry” in Psychology Today
Empathic distress is the feeling of intolerable pain triggered by seeing somebody else’s suffering. While the ability to experience empathic distress is highly valued, it can prevent someone from acting to aid the other person and hastens burnout in care workers.
Somebody else’s problem (also known as someone else’s problem or SEP) is a psychological effect where people choose to dissociate themselves from an issue that may be in critical need of recognition. Such issues may be of large concern to the population as a whole but can easily be a choice of ignorance by an individual. Author Douglas Adams’ comedic description of the condition, which he ascribes to a physical “SEP field,” has helped make it a generally recognized phenomenon.
Images copyright Owen Vince 2016, from Memory Insufficient
For more on this, see Owen Vince:
A city needs real people, but also needs institutions and infrastructure. Placemaking is about how to achieve these things without hurting the poor, the marginalised, the oppressed. It’s about enabling space for transgressive uses, the unexpected, the daring. These are cities many, many miles away from the cities of Sim City. They are more like the city of City Life v.3 (a game), or of Occupy (a movement), or of the E15 Mothers group in London’s Stratford (a community protest). It is about rejecting the idealism of the zoned Radiant City, and of the neo-liberalism and vapid gentrification of the New Urbanism. It is something that cannot currently be found in city simulations, whether as games or software for town planners. Consistently, the city’s other is displaced by a normative profile of its ideal citizen, a rubric so broad – so general – as to embody both everybody and nobody (the citizen as sim).
Owen Vince (2015) “The Death and Life of Simulated Cities” in Memory Insufficient