Interview: VR's Cool Now, But AR's Sexy

Last week's VR Connects conference had over 800 registrations and saw over 30 speakers and panelists share their wisdom at the Bespoke in San Francisco.

Among those presenters was Aruna Inversin, movie VFX expert turned Digital Domain's VR/AR Creative Director. He delivered a superstar session on Wednesday 28th June to a packed room, talking through the creation of a unique immersive NBA experience.

We grabbed him to talk about his work on Hollywood juggernauts The Matrix and Fast and the Furious to sporting royalty like Nike, Neymar Jr and the NBA. Inversin explains his love of VFX, how VR technology is catching up with creatives and why AR is really taking off.

 

TVR.biz: How did you get started in the entertainment business?

Aruna Inversin: I started in visual effects in Vancouver. I went to Vancouver Film School originally for 3D model and animation and I fell in love with the actual visual effects side of animation. There was a lot of things coming up at the time; Reboot and straight-to-TV animated things.

But I wanted to focus on digital effects in TV and film. I started 20 years ago doing compositing, and then from there I did a lot of freelance work in Vancouver and Australia and the US. I settled down in the US in the Bay Area, doing some Matrix, some Constantine, a bunch of really cool stuff.

But I wanted to do more and learn more. After a certain point at certain companies, you feel like you need to move up the chain. But at the same time, you become the big fish and you’re not learning as much! I felt like I was kind of stagnating.

At that moment, Digital Domain picked me up. I had a call saying, “Hey, do you want to come down and work on a Clint Eastwood film?” I’m like, “Uh… yeah! (I don’t know how many more movies he’s going to do!)" That was Flags Of Our Fathers. I moved to LA, started working on Flags, and started my new career in LA. Worked on a bunch of other movies – everything from Meet The Robinsons to Furious 7 and Blackhat.

One of the other VFX supervisors at the time, Janelle [Croshaw], was supervising a 3D 360 film called Evolution Of Verse. It’s very popular. We did all the CG creation for that. That was the first project, I think, that DD did, doing photo-real CG in 360-degrees in stereo.

How did you transition into Virtual Reality from there?

Janelle was on her next project and she brought me on because she was super-busy, and I’d expressed interest in VR. And that was the Björk Stonemilker project. That was also stereo, with Björk in Iceland. We got the footage, stitched it, put multiple Björks in the location. So that was our second piece. That was all live-action, but cut out so she’s running around. Really cool, really cool. So I worked with the director on that. And then after that, it’s been a plethora of other types of VR experiences. A lot of the time, it’s mostly been 360 VR experiences in monoscopic.

Then we got nominated for was the Neymar Jr experience. That was a really huge one. Janelle was the VFX supervisor, and I ran the floor. I was there, working with her on how to shoot it. Because nobody really knew! We had this elaborate setup and a huge amount of infrastructure.

We’re shooting in Barcelona for eight days. It’s a big budget thing. So Janelle didn’t want to go in there blind, and I kind of knew what I was doing at the time, and was figuring out solutions to problems that didn’t exist yet. We did an internal test at DD to prove that my methodology worked. Right now, a lot of people are starting to do that, which is basically shooting in ‘pies’ – in quadrants – and then stitching it in afterwards. So you get higher fidelity stuff. The Neymar Experience was about two-and-a-half years ago. It was one of the first times I think almost anybody started shooting in high fidelity pie slices.

 

 

And then we did a bunch of smaller pieces – fully CG stuff for Syfy in 2015, for example. My core in what I do at Digital Domain is heavy visual effects VR pieces.

  

Tell us a little about NBA. How did that come about, and what were the challenges of making that experience?

Early in January 2016, I did a small internal piece called Into The Storm - it’s on our Digital Domain app so you can look at it. It was a test-case scenario to show that we could put live-action people into VR environment. And it was good. And the work that we did on Storm allowed us to get the Syfy piece.

With Into The Storm I shot with a stereo rig, through a mirror – so you get the proper IPD [interpupillary distance] of a person. We shot on green screen, and we put him into a digital environment. It’s just really small, a minute long. (It's actually the model of Vin Diesel we had from Fast & the Furious! Nobody knows this. Could be anybody’s legs, but it’s his legs.) And our live-action person is outside the door, and you see this tornado come out and it whips up the car.

We showed that to a lot of internal people that came by. It’s a show piece: “Hey, we can do VR with visual effects. Come in and let’s talk.” So that was kind of how that conversation with NBA started. About the middle of the year, we went out and shot a basketball court in Oklahoma with the idea that we would use that in some sort of NBA project. But they didn’t go with the core idea. They came back and said, “We want to do something a little more intimate, a little more up close with NBA legends.” So given that directive, the content studio group came up with the idea of: “Let’s do a face-to-face interview in a really cool lounge, not on a basketball court.”

What’s sexy? What’s cool? Right now, it’s VR. VR is there. We see a lot of that happening right now for AR.

With Nike and Neymar as well as the NBA, you’re dealing with some very big brands. What’s the experience like of creating something for another person’s IP who’s going to be very protective?

For Syfy as well as for NBA and Nike, they didn’t know too much about VR. It's a marketing reason to get into VR. A lot of these brands want to reach a younger audience. In order to do that, what’s sexy? What’s cool? Right now, it’s VR. VR is there. One of the other things that coming out is AR. So that’s even bigger. We see a lot of that happening right now for AR.

But in order to sell a client on VR, or to get them interested, we have to show them certain high-calibre projects in VR, as well as educate them on what VR is. And that’s the key. So a lot of the time, if a client comes to us and they’re serious about doing work with us, we have to educate them on what’s a headset and what’s the distance between the eyes and what’s stereo and what’s monoscopic. So it’s a lot of repetitive explanation of what VR is.

And also, there’s different experiences on the phone or on a Rift or on a Vibe. Even at DD, there’s been very few people who understand al the terminologies and all the technologies involved in creating it. So it was a perfect opportunity for me to continue what I’ve been doing for so long on the technological side, and bring all the VFX knowledge that I have into this one little pocket of information.

 

NEXT PAGE...

Click here to view the list »

• Aruna Inversin Interview

  You made a good point there about terminology. People use the term VR, and sometimes they mean mono 360 video, and sometimes they mean completely computer-generated gaming. And sometimes AR. Is that a particular challenge? Do we need to clarify our terminology?

  As the consumer gets more savvy with what they want, they’ll drive the marketability - and they’ll have to start learning. As opposed to you explaining to them what it is.

  I think it’ll happen organically with time when consumers know. Because they’re going to ask you that. Right now, we’re seeing more knowledge about what 360 video is. And now, Google is doing 180 video. But you know who’s been doing 180 video for years? The adult industry! So there’s a market that they kickstarted already.

  And now Google’s like, “We need fidelity in our video. We can’t do 360, because there’s just too much information, and you’re not looking at it all at the same time.” So they found a happy medium, which is VR 180. You get better fidelity.

  So the consumers are going to be like, “OK, I’ve got 180, I’ve got 360. That’s just video. OK, now I want something I can walk around in.”

  Then we see Google pushing that with their standalone headset, and then you have the Vive and the Rift that are room-scale. So I think as the consumer gets more savvy with what they want, I think they’ll drive some of the marketability, and they’ll have to start learning. As opposed to you as a vendor, as a creator, explaining to them what it is.

  

   

  It’s a new frontier. Are you finding every day you have to figure out how to do a process? You’ve got to invent a way to do things? That must thrill you as a creative.

  Somebody else would have figured it out. We just happened to have that opportunity first.

  A lot of the time, when we’re given a creative or we develop a creative, we have to push that technology as well. Because there’s nothing that’s been developed for that. For example, with the Neymar job, they came to us and they said, “We want to do a POV experience of Neymar running through a bunch of defenders and scoring a goal.” That was it. One line. I can do it, simple.

  And then, “Oh, and it’s in 360.” But to do it in 360? Oh my goodness, there’s so many other things to do.

  So I brainstormed it and came up with a creative solution at that time. I’m sure with that line – “do it in 360 with these caveats” – somebody else would have figured it out as well. Another problem solver would have been able to think outside the box and solve that problem. We just happened to have that opportunity first.

  

  The technology is catching up with the creative, and that’s powerful.

  For example with the NBA VR experience, it was going to be on the Daydream and it was going to be on the Samsung. Of course, if it’s in a headset, it’s got to be stereo. But it’s locked stereo. So in developing the camera system for that, I said, “Okay, let’s not do like a Jump Odyssey. Let’s not do a GoPro 360. They’re not stereo.”

  We wanted to put CG in it at high fidelity, so I needed to find a camera that could capture that, that would sync together, as well as having high fidelity and was stereo. And none of that existed. The closest at the time that existed were the Blackmagic micro cameras. So I talked to the live-action producer, and I said, “You know what? We need to have some of these cameras, just to see if this idea is going to work.”

  We fabricated a little bracket for it. We put the two cameras in there, shot a test, and you look at it in stereo and VR. It’s like, “He’s right there. He looks like he’s the right size, and he’s right there.” A lot of the time when you see stereo 360 that’s captured side by side, everybody looks small, because the cameras are so far away. But for NBA, we shot with the proper eye distance, so everything looks proper-sized. But unfortunately, no technology exists yet to put those sizes together, so that’s why I went and developed a mounting rig for that.

  Now you can buy a rig online. A stereo rig, right away, from Z-Cam. I saw it at NAB. I took a picture and was like, “You have got to be kidding me. I did this a year ago and had to get it manufactured, and now it’s off the shelf.” You can see the technology is catching up with the creative, and that’s powerful. They can include it all in there.

   

  What’s your advice to anyone who goes, “I hear VR is going to be huge. I want to get into that”?

  You can create a story with a really crappy system. And when people see that it’s a good story and they can follow it in 360, then you get more money.

  It depends on what part of VR you want to get into. There are different avenues. There’s the hard coding aspect – getting into Unity and Unreal and making that game. There’s also the creative, figuring out the storytelling creative side of it. It’s like, “How do I make a story in 360?” Or maybe it’s not 360. Maybe it’s just in certain quadrants, and you can have three stories going at once.

  Right now, every single conference I’ve gone to, from the OC3 Oculus Connect conference to NAB to VR LA, everyone’s talking about story. “How do we create story?” The best way is to go out and try shooting it. I think one of the people that was talking about it, Gabo (Arora), who did Clouds Over Sidra, he had mentioned: “Just go and shoot something in 360.”

  Right now, Gear VR 360 – everyone’s got a 360 camera. So just shoot it and see what’s good about it! You can create a story with a really crappy system. And when people see that it’s a good story and they can follow it in 360, then you get more money. You get stereo, you get all that cool stuff coming after you. Visual effects and all that.

  

  And then we have the education. People need to learn about what’s involved in VR. Like you said, there’s a lot of different types of VR, from movies to games.

  Then there's executive level, and legal, business: trying to get money to do it. It’s one thing to have all the other three, but to actually sell that package to somebody is a big powerful thing. Sometimes a creative can’t do that. Sometimes you need a CEO that understands it as well, that can make those plays and tie those things together. At Digital Domain we have folk like, “OK, we can sell that to NBA or NFL or whatever it is.” And then they allow me to do the creative, and run the floor, and create the tech, and do all that stuff. That’s the package.

   

  I’m trying to figure out how to do AR. That's the sexy thing coming up this year. I don’t want VR. I want AR.

  What are you working on now?

  I wish I could say. I really do. We are doing a bunch of internal VR projects. I’m in charge of doing a lot of AR development. I’m trying to figure out how we’re going to do some AR stuff. Like I said, that seems to be the sexy thing that’s coming up this year. “I don’t want VR. I want AR.” So I’m doing a bunch of pitches for AR. We have a couple of really big projects coming up this year that I can’t talk about because they’re not signed. But again, if NBA decided to do a season two, they’ll probably come to us as well, because we have the app and they know we have the methodology down.

   

  The potential of AR is huge.

  Absolutely. It’s going to replace phones. And when they get smaller, it’s going to replace how you see the world – literally how you see the world. When Google Glass came out, that was my first inkling of like, “Oh my God, they’ve got it already?” And sure enough… they didn’t. But now with everything getting smaller, the HoloLens, it’s all-encompassing right there – it’s a small field of view, but you can imagine… oh my goodness, right?

  

  Don’t try to be the supermarket, but be in all the supermarkets. You can compete on the shelf with everybody, having the best spaghetti or the best pickles.

  That’s a blessing and a curse, but on the marketing side, in terms of clients coming to us – I mean, now you can put billboards where you want. You can make CG characters pop out of the sky. That drives the market. That drives the creative. Now you can put a dinosaur roaming down the Long Island expressway or wherever or going through London or wherever it is. There’s no limit to your imagination.

  That’s one of the reasons why I got out of visual effects. Visual effects – we can do everything now. Even being on Digital Domain, there’s so much cool stuff, but at the same time, what do you want in a movie? You want a blue monster that becomes a gaseous creature that falls into the water? We can do that. It’s so extravagant, and I was reaching a limit. 'I’m not innovating. I want to do something that pushes the envelope.'

  That’s why getting into VR was really good, too. Also, VR and AR are at the stage that the app store was 10 years ago. It’s the primetime for a start-up to make something amazing and put it in the store, and people are going to buy it. They’re hungry for all sorts of stuff.

  I think Google is not playing it safe, but they’re opening up all their assets and their platforms. Same thing with Facebook – like Facebook 360 surround, their sound kits… they’re pushing it. They want people to make their own content. I find that if you own the platform, you can monetise that platform. That’s what everybody’s got to do.

  I have this metaphor for VR, and for people starting to do VR apps and things like that. Don’t try to be the supermarket, but be in all the supermarkets. Be an app that is in Facebook, that is in Google Play, that is in iTunes. Because you can’t compete with those guys. But you can compete on the shelf with everybody. Like having the best spaghetti or the best pickles. If you have the best in that category, then you’re going to make money. But you have to be where people want to go.

   

  Tickets to the next XR Connects event in Helsinki (19-20 September) are available now – speakers will include Pete Speller of Greenpeace and Chris Donahue of AMD. There's also another chance to enter the VR Indie Pitch.