ALL THE LATEST NEWS ABOUT THE VIRTUAL + AUGMENTED + MIXED REALITY BUSINESS

Interviews & Opinion

The Future Of VR Narrative

The Future Of VR Narrative

Daniel Burwen is Creative Director at Robot Sea Monster Games and one of 50 world-class speakers appearing at VR Connects San Francisco 2017 on the 27th and 28th June. Here, the award-winning comic book creator and interactive storyteller explores how VR will change storytelling and looks at some of the successes in the medium so far.

 

As VR takes off, primarily in the entertainment space, there’s a lot of hubbub around how to tell stories with this new medium. How has VR storytelling shaped up over recent years with this groundbreaking new platform? To better understand the landscape, I’d like to start by framing the evolution of VR/AR industry at large and then dive deeper on the storytelling component.

VR has been accelerating along a trajectory primarily comprised of three sectors:

  1. Technology
  2. Arts + Media
  3. Consumer Adoption

 

The evolution of VR devices has been incredibly fast compared to previous technology products.

Tech Foot Forwards
Technology has largely been dominating the headlines, as the evolution of VR devices has been incredibly fast compared to previous technology products. The technology side has been evolving quickly because it’s obvious where tech needs to go in the future - ending up with something like a pair of contact lenses that you wear all the time, every day, in all settings - both work and recreation - and not unlike our smartphones now.

We can reverse back from that eventual destination to a pair of attractive glasses, a pair of ugly glasses, and to the bulky expensive tethered headsets of today. The goals with technology are obvious: higher resolution, better visual fidelity, lose the wires, make it smaller, lighter, cheaper and so on... It’s not easy to accomplish these things, just that it’s obvious where to go next in the evolution and to be able to judge if progress is being made.

 

Arts And Craft

However, in the arts and media sector - where storytelling lives - this isn’t the case. If you look at the evolution of storytelling over the past 100 years or so, it’s predominantly been owned by film and television. Eadweard Muybridge created The Horse in Motion in 1878, giving rise to the medium and eventually to the first modern film in 1941; Citizen Kane about 60 years later).

Eadweard Muybridge's The Horse In Motion from 1878.

Plus Ca Change

The experience of watching Horse in Motion and Citizen Kane is actually pretty similar to VR from a UX standpoint. You watch, you listen, you infer meaning. Granted, Citizen Kane employs many sophisticated creative techniques that simply didn’t exist in 1878, but that’s how storytelling and art in general evolves.

Editing, camera language, sound and other technological or creative approaches would come from decades of experimentation and innovation, but the fundamental medium and the way we consumed it stayed relatively constant. Through consumer adoption, it became the dominant media format of the 20th century.

As a result, we got a mature film and television industry built on a well proven process and formula for creating the medium. Screenplays, the three-act structure, talent agencies, production companies, film financing and distribution models - and now digital subscription platforms - have all been able to emerge and be sustained due to the format itself remaining relatively unchanged since Citizen Kane.

The film format has remained relatively unchanged since 1941's Citizen Kane.

In addition, virtually every media device you own can now play video. All video. From footage captured a century ago to the latest blockbuster feature film. And, the means to create and consume video have become all but completely democratised with the advent of smartphones and services like Youtube.

 

Rapid and constantly changing VR, AR and MR technologies make it almost impossible for storytellers and artists to master the form.

Strong And Stable

VR, by contrast, does not currently enjoy a stable and constant technological base to build from. Rapid and constantly changing VR, AR and MR technologies and the resulting dramatic changes in User Experience make it almost impossible for storytellers and artists to master the form, as the capabilities keep changing in a very short period of time.

This also creates a challenge for the established and monolithic storytelling machinery of Hollywood, which works on much longer timeframes for bringing projects to market. 18 months or longer is not uncommon for developing a feature film. However, 18 months in VR can see multiple steps forward with the technology and unfolding consumer strategy. As a result, you run a high risk of the project you originally outlined no longer making market or creative sense in light of these changes in technological and consumer behaviour.

 

A 180 On 360?

For example, if 360 video was the end of the road for the format, and all our resources creatively and technologically were going towards figuring out how to best tell stories in that medium, we would probably get there fairly soon. However, the moment you add positional tracking (6DoF) to a headset and allow the user to see the real world through it (like with the Occipital Bridge), the overall experience is fundamentally different from where you were merely months before as an industry.

Suddenly, all the issues of 3DoF 360 video are either rendered moot or no longer make sense within this new User Experience. This affects our third issue and the one that matters the most: consumer adoption.

Has Occipital Bridge already made 3DoF 360 video redundant?

 

If consumers can’t create a consumption ritual around a product, it will never catch on.

Ritual De Lo Habitual

At the end of the day, if consumers can’t create a consumption ritual around a product, it will never catch on. Film and television evolved from a series of prior media consumption rituals. After consumers were accustomed to going to the movie theatre, in and of itself a ritual derived from attending stage plays and other live performances, the TV set started appearing in living rooms of the home in the late 1940s. It was taking the place of the fireplace where the family gathers for social time and in the past, the time-honoured tradition of telling stories to each other.

The space never changed. Rather, the technology came to fit a behaviour that was already there, first made familiar by attending theatres. TV was also originally broadcast live as radio had been before it, before eventually becoming film-based, thanks to I Love Lucy giving birth to the rerun.

What's a re-run? You'll find out...
 

Games in general have struggled with story, primarily to do with the abstract interfaces.

Location, Location, Location

Conversely, where do consumers go to use a VR headset? What pre-existing behaviour can we build on? When you’re isolated from the people immediately around you as is the situation with the headsets of today, you need to look to other places in the home where being isolated makes sense.

The workbench and reading nook spaces are often where people go for a deeper individual experience, sometimes connected to others socially online, but the workbench is not a mainstream media consumption space.

PC gaming is a large demographic, but not one especially known for deep forms of narrative. Games in general have struggled with story for a variety of reasons, primarily to do with the abstract interfaces used to control them. We eventually saw standardisation of the gamepad, born out of the use of the TV remote control, but it remains an interface comprised of pushing buttons and levers.

 

It is because the interfaces of VR / AR are so natural that we can finally develop interactive narrative experiences with deep emotional content.

Natural Remedies

It is because the interfaces of VR / AR are so natural that we can finally develop interactive narrative experiences with deep emotional content. But the skillset needed to create these types of experiences doesn’t really exist yet. You see bits and pieces of it out there, floating around in indie gaming, web documentary and other fringe subcultures that have largely remained below the radar of mainstream culture, but only now are we starting to see a demand for this rare combination of user agency with narrative. There is little-to-no documentation, knowledge, process, workflow, or even a library of consistent experiences to draw from as we figure it out.

That’s not to say that the medium isn’t emerging. Every week there’s something new and noteworthy to examine and learn from. Much like the early experimental filmmaking of the late 1800s and early 1900s when many modern filmmaking techniques like editing and camera language were being discovered and evolved, we’re now in a similar phase of VR narrative.

 

Lets take a look at some noteworthy examples and why they’re relevant, starting with a couple of 360 videos before moving on to some realtime 3D...

Click here to view the list »
  • Paul McCartney, Live And Let Die At Candlestick Park, Jaunt (2014)

    Paul McCartney, Live And Let Die At Candlestick Park, Jaunt (2014) logo

    360 video made a big impact when it first emerged at the start of the VR industry. Being camera based, it’s a natural departure point for mainstream Hollywood culture to dip its toes into this new medium as many of the technology, workflow and general processes can be recycled.

    However, grammar for telling stories is ripe for discovery once again. Much has been written on the ‘death of the rectangle’ in this regard. Paul McCartney, Live And Let Die At Candlestick Park is noteworthy because it was one of the first applications of this medium to show the potential with subject matter that resonated with mainstream audiences.

    Link to content 
    Link to Jaunt One video 
    Link to Jaunt 

     


  • Real, Perception 2 (2015)

    Real, Perception 2 (2015) logo

    One of the first applications of the spherical video format is to incorporate previous formats within it. Real employs both traditional media and stereoscopic 180 imagery to create a context aware story that acknowledges the limitations of the medium. This is a key factor for creating effective narrative in VR, and this is an excellent example. Whether techniques like that of Real will endure remains to be seen.

    Link to content 
    Link to Perception 2 
    Link to Connor and Alex creator profile 
    Link to Connor and Alex presentation at Jaunt 


  • I Expect you to Die, Oculus / PS VR, Schell Games

    I Expect you to Die, Oculus / PS VR, Schell Games logo

    I Expect you to Die is a seated experience using the mouse, designed for the DK2. The game does a great job of acknowledging and embracing the limitations of the interface and of the medium, creating and experience that feels good because your limitations of not being able to stand or move freely, not being able to pick up things with your hands are explained and framed in the experience.

    Jesse does a great job explaining the experimentation behind using the mouse as a way to express user intent in the world, including visual queues for leaving objects hovering in space.

    Link to content 
    Link to Voices of VR podcast 
    Link to GDC Narrative Summit 2013 


  • Accounting, Vive, Crows Crows Crows / Squanchtendo

    Accounting, Vive, Crows Crows Crows / Squanchtendo logo

    Accounting is a room-scale experience built in a crash world, intended for mature audiences. It’s a collaboration between the creators of Rick and Morty and Stanley Parable, a highly successful example of a walking simulator narrative game, (worthy of an article in and of itself).

    Like I Expect you to Die, what Accounting does well is work within the boundaries of a Vive / room-scale experience. The spaces you navigate through are room-scale themselves, so you never feel like you want to travel outside the trackable volume.

    Many Vive games employ a teleportation mechanic (more on that later) to overcome the limitations of room scale. But I would argue that unless that mechanic is built from the ground up into the narrative, it will break immersion.

    What Accounting also does incredibly well is employ reactive audio queues from a pair of obnoxious but hilarious narrators. This creates a sense of connection with the characters in the game, but because you can’t talk back, frames the limitation as a strength.

    Future advents that will fundamentally change the medium will resolve around AI characters, where you can talk to your VR experiences. As Jesse said in his talk we linked previously, “Film was taken seriously when it could talk. Games will be taken seriously when they can listen.”

    Link to content 
    Link to developer 


  • Budget Cuts, Vive, Neat Corporation

    Budget Cuts, Vive, Neat Corporation logo

    Locomotion mechanics are a tricky part of VR right now and a challenge that isn’t going to go away anytime soon. For home use, the boundaries of the user’s home is pretty much set in stone – perhaps quite literally - so room-scale will remain room-scale for the foreseeable future.

    The question is how to make locomotion an integral part of the experience, so that the user gets more drawn in the more they use it. Budget Cuts has perhaps one of the best designed and most engaging locomotion systems to date and the resulting immersion and sense of freedom is one to study. Combine that with satisfying interaction models and the world is one you can stay in for a very long time.

    Link to content 
    Link to developer 

     


  • Wilson's Heart, Oculus, Twisted Pixel

    Wilson's Heart, Oculus, Twisted Pixel logo

    Wilson's Heart is a great example of putting resources in the right places. While not without its flaws, it draws on similar conceits to Accounting, in that you essentially teleport to pre-existing locations in the world and then employ a variety of very satisfying interactions in that world, often novel and memorable.

    Link to content 
    Link to developer 


  • Adventure Time: Magic Man’s Head Games, Gear VR, Turbo Button

    Adventure Time: Magic Man’s Head Games, Gear VR, Turbo Button logo

    Adventure Time: Magic Man’s Head Games is a great example of how to take an existing franchise into VR. You’re in third-person, watching Finn and Jake run and jump around the world in this platformer. What sets this experience apart is that the characters talk back to you, so you have a relationship with them as the player and they acknowledge your existence; there’s even a moment when Jake takes a selfie with you.

    This game does a lot of things right and despite its limited scope, I can see this setting the groundwork for the future of the ‘show’ where the audience is a live participant in the story from the get-go.

    Link to content 
    Link to Developer 


  • Gary the Gull, Vive/Rift, Amotional

    Gary the Gull, Vive/Rift, Amotional logo

    Gary the Gull is an early attempt at creating a fully interactive character in VR that actually speaks to you and responds to your input. It’s a simple experience, where a talking animated seagull in Pixar like fashion has a conversation with you and responds based on where you are looking.

    While not without its flaws, this is a bold experiment in creating a character that responds to you, when current VR input is limited to pushing buttons and looking around.

    Link to content 
    Link to developer 


  • Follow The Pixel Brick Road

    Follow The Pixel Brick Road logo

    Unlike the technology side of the industry, the path to VR storytelling is neither direct nor obvious. To succeed, we will need a combination of patience, persistence, an open and honest sharing of our learnings and failures, along with the right lens, language, approach, and expectation.

    Nothing should be seen as a mainstream product offering until the technology side stabilises and we can all build on the same foundation of capability. If consumers aren’t buying it, it doesn’t matter how great your content is. After all, if a tree falls and nobody hears it…

     

    About The Author

    Daniel Burwen has over 15 years’ experience in creative production across various industries including games, motion, interaction design, apps, and visual storytelling. He is a specialist in building, training and directing creative teams with a focus on new forms of interface including: large scale installations, touch screen kiosks, mobile apps, console games, VR applications, and comics. Burwen has presented on his work and design philosophy at conferences and film festivals around the world.

    Burwen and Robot Sea Monsters are curently working on their new interactive comic, Refraction for Occipital Bridge.

     

    You can find out more about Daniel Burwen in our speaker profile. Don’t miss your chance to hear him share demographics of VR businesses, content creators and consumers at VR Connects San Francisco. His session is at 10:35 on the 28th June 2017. Book your ticket now!


PocketGamer.biz regularly posts content from a variety of guest writers across the games industry. These encompass a wide range of topics and people from different backgrounds and diversities, sharing their opinion on the hottest trending topics, undiscovered gems and what the future of the business holds.