Augmenting our hopes and fears: Reflections on the use of immersive technologies at the Stockholm VR Sci-Fest 2018
The way immersive technologies (virtual, augmented and mixed reality) are used are fundamentally shaped by the way in which we already see the world. In that sense, they could potentially just expand our capacity to isolate ourselves, to exploit nature, to transform painful realities into sensational entertainment. But they could also help us to render tangible our cognitive worlds, to dive deep into our inner worlds of imagination and the outer worlds of space exploration, to bridge science and art in collaborative experiences, to augment the voices of the unheard through new spatial narratives.
On mid-may I attended the VR Sci Fest 2018 held at the Royal Institute of Technology in Stockholm. This was an exciting space to learn about the current and potential uses of immersive technologies or extended reality technologies (XR): virtual reality, augmented reality and mixed reality. It was also a door to explore my hopes and fears of these emerging technologies.
Let me be upfront — just because this is what I have been asked the most- I can perfectly imagine quite disturbing ways in which immersive technologies could be used. If you wonder how that looks like, just take a look to Hyper reality , a short film that depicts my fears of walking into a city saturated by advertisement, where our minds have been hijacked by trashy content.
And it is quite easy to escalate these fears. One speaker at the VR Sci Fest pointed out that first we were portraying World War I in photographs, then the Vietnam war in film, and then the Syrian conflict in Facebook livestream, what comes next? will we be portraying the next war in a VR experience? I think he had a point. If we only use these technologies to reproduce the failures of our system, we will distract ourselves from looking at the root causes underlying our planetary crisis. Our shallow entertainment culture is already succeeding at this. However, it is also possible to see hopeful pathways immersive technologies can bring about, which are part of bigger cultural shifts.
In the following lines, I will share my thoughts about some cases showcased in the VR Sci Fest 2018.
Immersive storytelling: Fast content for empathy or embodied empathy?
A lot of people is jumping into the trend of creating content for Virtual Reality platforms. The argument is that by making people have a first-person experience of, otherwise, unknown worlds to them, we can make them emphatize with relevant issues like social injustice, environmental degradation. Sounds promising right?
At the VR Sci Fest I got to know a couple of examples of immersive journalism. Shehani Fernando, director of VR experiences from the The Guardian, presented ‘The Party’, a 360 film seen through the eyes of an autistic teenager, and ‘6x9’, a simulation of solitary confinement. It was clear to me that you can certainly bring people closer to understand how the world feels like for autistic people (for example), but in the case of 6x9, I couldn’t help to wonder if somebody would merely use this technology to have a voyeuristic experience. In US, 80 000–100 000 people are in solitary confinement. How do we make sure that the experiences of marginalized people don’t become part of a VR high-tech museum?
I believe that if VR help audiences become protagonists at the heart of these crisis, it is only expected that people can become protagonists in acting. As Sam Gregory says, we should go from “‘being there to being there together”. For content creators the challenge is to be aware of the danger of “reinforcing an emphasis on dramatic moments and in not structural causes”. We should remember that fast content seeks clicks, and good content seeks conversation, and I should add: action. So what else could The Guardian do to help people understand the criminal justice system and address the need for criminal justice reform? should they even embrace this responsibility? and in which other ways can journalism be truly “immersive”? (beyond the technological use of the world). Extended reality technologies pose the question on whether we should also talk about extended roles of actors using these technologies.
I should also add here my skepticism to use immersive technologies to replace direct experiences in nature. This is coming from a comment I overheard in regards to using VR experiences to allow people in Singapore to actually see a starry night. Light pollution is so bad in Singapore that people’s eyes never fully adapt to the dark when looking at the sky after sunset. Singapore is the most light-polluted country in the world, but it’s pretty bad elsewhere too. According to a study published in Science Advances, one-third of humanity lives under skies where you can’t see the Milky Way. Pretty sad huh? Well, let’s acknowledge that the real issue is that our urban lives are disconnecting us from nature in the first place. Then maybe showing people what they are missing with VR experiences is a good strategy to engage citizens. But maybe not, maybe the leverage point is somewhere else. So in this case, I don’t think that replacing direct nature experiences is the way to go.
However, did you ever think it would be possible to experience being another life form? Yedan Qian, from MIT Media Lab showed how this is already happening, potentially hacking the limits of our empathy.
A VR experience combined with a multisensory system can create illusions of body ownership, and connects us visually (but also including other senses) to a 1st person perspective of nature. This is something called “embodied storytelling”. Yedan Qian told us about TreeSense, a sensory VR system that transforms a person into a tree, from a seedling to its full-size form, to its final destiny. A person can experience what it feels like to be a tree by seeing and feeling branches growing, a worm crawling, or a bird landing on his/her arm. How? Of course, you have the visual experience in VR and on top of that you have electrodes at several key locations on the forearms to stimulate muscles and the skin accordingly to what you see (this is called Electrical Muscle Stimulation- EMS). So could the illusion of being a different life form promote environmental protection?
Immersive Science: visualizing the hidden dynamics, stepping into our cognitive spaces
Stepping into our inner cognitive worlds sounds a little bit like magic. But is actually possible now. Boo Aguilar creator of Blank Canvas designed a VR experience that makes it possible to step into microscopic worlds where brilliant innovations in science are happening: DNA editing, hacked viruses and CRISPR. Making these otherwise-isolated scientific processes into experiences to which everyone can attend is just mindboggling.
Of course that educational implications of such possibilities are countless but still, how can these types of experience change the way science is being produced at the micro and macro scales - where a direct experience is impossible or nearly inaccessible (not everyone can go to space for example!)-? how can these experiences change the systems of consumption of science- from almost disconnected from the general public to mainstream content consumption?
Another thing I was excited about was the use of the tool Tiltbrush in the Hackathon — that was part of the VR-Sci fest. I knew about Tiltbrush before, but I had only seen art-related applications, and I had already thought of how such immersive experiences can change cognitive processes in the artist? what happens in your brain when you can actually be in your art, walk in your inner worlds — worlds of imagination? But now, I also wondered if this immersive experience could have impacts in collaboration? what would it feel to walk into somebody else’s inner world? I think I have more questions than answers in this matter.
Final reflections
Altogether, the VR-Sci Fest 2018 was a good opportunity to face some fears and hold on to some hopes in regards to the use of immersive technologies. What it is clear to me is that we should not forget to put on our critical glasses before playing with any headset. We should be aware that everything comes with an agenda, so does the use of immersive technologies. So before you give in to the hype of another VR/AR/XR application, ask yourself if these tools are being used to expand our empathy, push the boundaries of our imagination, democratize science, increase collaboration in design? or just to fuel consumerism, to feed shallow entertainment, to replace direct contact with nature or to isolate ourselves from others?
