Nate Mitchell is Vice President of Product and co-founder of Californian Virtual Reality pioneers, Oculus. Formerly a Senior Project Developer at Gaikai, Nate previously worked for Autodesk as a Senior Software Engineer, and started his industry career as a Software Support Engineer with Scaleform in June 2009. Nate graduated from Dickinson College, Pennsylvania USA with a BS in Computer Science. We spoke to him recently about Oculus Touch, the new controller which tracks your movements and gives your hands presence in the virtual world. We started by asking about the controller's development and what it aims to achieve.
NATE MITCHELL: We’ve been developing Touch for a while now. For a long time. I mean, since long before the Rift actually came out! We really got excited about this concept of hand presence – truly feeling like your virtual hands are your own. We wanted to build an input device that enabled hand presence. And then more than that, made it really natural and intuitive, to actually interact with a virtual world – just like we do in real life.
And then building on that, we also discovered – or are starting to think – a lot about social interactions, and enabling all sorts of social behaviour. So with Touch, you can now give a thumbs up or point at something – which also doubles as gameplay features. Because you’ll walk to a light switch, put your finger out and flip the switch. In VR, that’s something you’ve never done before on any platform or medium ever.
So we have been iterating on Touch, just tweaking it, taking the time to really get it right. We’re really excited about where it is today. The version you’re using isn’t the final, final version that’ll be in boxes later this year. But we are launching Touch later this year. We’re going to be announcing more news at OC3, and it is going to have an incredible line-up of games, just like you’d expect – day one.
The Virtual Report: You talk about physical hand presence in the VR world. It does something quite different from the Vive. What was the biggest challenge with creating the hand presence?
NM: There are a lot of different challenges – sort of fundamental challenges that mostly were solved early on. But things like latency, for example: making sure it really does track and feel, almost one for one with your actual hand movement, is something that’s fundamental to really feeling like it’s yours.
The avatars… you’d be surprised! Getting the avatars just right; making the interactions and look and feel of the avatars is super-important. We have a lot of developer tools for Touch and a sort of Touch SDK that helps developers do all that stuff. But getting that just right requires a lot of tweaking, both on the design side but also on the engineering side.
Another big sort of tricky one is this concept of hand registration, which is basically how accurately we’re mapping your position basically of your physical hands to the virtual hands in the world. Basically, this has to do with tracking precision. It has to do with rendering. It has to do with the blend and synthesis of all this stuff – because you can imagine if your hands are like this but in the virtual world your hands are like this, the sense of presence is totally broken.
All of these details have to come together across hardware, software, content, everything. Again, we’ve been working on this stuff for a long, long time. Now we’re getting it just right, and we really do feel it’s fundamental for the experience that you’re going to see developers create.
TVR: So physical presence is a very visceral thing. It's a difficult question, but how do you deal with issues like race and gender? A person’s hands are a very important part of their identity.
NM: Great question. It’s actually very difficult. And I don’t think we’ve nailed it or cracked the code totally. I’ll give you an example. When we announced Touch at E3 last year, we had a new demo that we’d been developing in tandem with Touch. We actually had a software team building an experience to help inform and guide the hardware. And while we were trying to do things in Toybox, we would change Touch to enable those behaviours in Toybox. So the two were developed hand in hand.
One of the challenges with Toybox right out of the gate is we didn’t actually have all the time ahead of E3 to add lots of different genders or races or things like that. We basically had one set of man hands. So what happened was, you get a female journalist in and she tells you, “I love it, except I have man hands. I feel it’s weird.” We’re like, “Sorry, it’s just a demo.”
But this is a big challenge. Ultimately, we’re doing some things on our side as part of the software stack to help developers get some basics in place as far as gender specifically. But the reality is, developers are going to have to enable a little more customisation on their side to actually let people set what they want their hands to feel like.
The exception to that is: I think there’s going to be a whole lot of games out there where you won’t be able to pick your hands. You can imagine if we’re all dragons sitting around this table, playing poker. I am making something up! But we’re not going to have man hands or female hands. We’re just going to have dragon claws. And that’s going to be up to the developer.
In other cases though, you are going to want to have your persona. “My identity through the Oculus universe” – that’s something we talk about a lot. In that case, you’re going to want to customise that, and again, we’re going to provide some stuff, developers are going to provide some stuff, and we’re going to be talking more about all of this at OC3.
But it’s a big challenge. I think you hit the nail on the head. It’s one of those things, like many things in VR, where developers are inventing this stuff for the first time. It’s part of the reason why VR is so exciting.
TVR: Can you tell me a little about some of the games that you’re most excited about, what your favourites are perhaps?
NM: No favourites! But just running through the list really quickly.
Wilson’s Heart is a game by Twisted Pixel. Wilson’s Heart feels extremely, extremely polished. You drop in there. It’s atmospheric. It’s this film noir. Great sound, great voice acting. And the interactions you have with Touch, like opening up medicine cabinets, picking up objects, looking for things – it’s very, very refined and it’s a real glimpse into true AAA experiences that as a VR enthusiast you’ve been dreaming of. So Wilson’s Heart is so cool.
Ripcoil is a multiplayer competitive game by Sanzaru, the same team that’s doing VR Sports. Really fun, really lightweight, very arcade-y. You drop in. you’re kind of doing this disc-throwing thing. It’s just a lightweight, very fun experience.
Dead And Buried is actually developed by a small team within Oculus in our Oculus studios. A couple of guys with the help of Gunfire, who we’ve worked with on a number of projects. It’s another sort of arcade-y game, but it’s just a super-fun shooter. The demo we have is a multiplayer two vs two, so you drop into this tavern, this pub, and you sort of like high-five your team-mate. Then it’s like: you go! And you’re spawning in the room. You’re taking shots in this bar fight. There’s grenades and there’s shotguns and all this stuff. It’s very, very fun. And as a gamer, you drop in there; you just know exactly what you’re doing immediately.
Unspoken is a multiplayer competitive game like wizards duelling. Imagine Harry Potter, except much, much, much more like a badass, rough kind of crossover with a rock band style. And then yeah, they’ve done this really nice job making it very engaging. It’s all wizards duelling. It’s very cool. Oh, and Unspoken is by Insomniac – world class developer.
The Climb is a title by Crytek. The Climb is totally unique. I don’t think anyone would make a game like The Climb before VR came out. The Climb already launched earlier this year on Rift. It’s actually one of my favourite titles on Rift, with a gamepad. They have a new Touch version launching alongside Touch. So we’re really excited about that. But definitely, stepping into The Climb, it’s a pretty visceral experience for most people – especially those who’ve never been rock climbing. And you look down and you’re like, “Oh my god…” And obviously the CryEngine is beautiful, so that the scenery, the locales… it’s a pretty special, memorable experience.
TVR: We have this thing where everyone’s talking about the fatigue effect in VR gaming. Do you find that having the Touch controls is helping that in some way? It’s helping people stay in the game longer, because now they’re even more immersed? They’re not holding a gamepad now, they’re more active.
NM: Anecdotally, I think a little bit, realistically, it does a bit more of the opposite. I don’t know exactly what fatigue effect you’re talking about specifically. There’s people today who are playing Minecraft or Project CARS for hours and hours and hours. Maybe they’re sitting there with a flight stick. Maybe it’s the game pad. Whatever it is, they’re just having an incredible time. I’m guilty of playing a lot of hours myself.
When you get up and start moving around with Touch, you’re definitely more active. I think by default, you tend to want a pure experience, right? I can sit and play Elite: Dangerous for hours. But playing Dead And Buried for a half-hour, I’m sweating.
I think that’s just the reality. Two things there. One is, we’ve tried to develop Touch so that it’s really comfortable in your lap too. So if you’re playing an experience or maybe it has a mode where you can just throw it in your lap and use it more like a gamepad – Touch is a hybrid between a gamepad and motion controllers. We think it’s a perfect hybrid for games, and it really fits with the other games.
I think there will be experiences where you sort of shift between modes, depending on what you want to do.
TVR: I played I Expect You To Die with a Touch control, and that’s a sitting experience. I didn’t realise I'd run out of time!
NM: I think that’s the other thing, and that I was going to say. It’s that, there are all of these Touch experiences that are pretty comfortable standing and pretty comfortable sitting down. Or they’re so active. When I play Wilson’s Heart, I played the whole 15-minute demo. Aside from sweating because I was nervous, I was like totally relaxed.
Dead And Buried or something like the sports challenges – those are a lot more active. So you’ll probably want to consume those in a more bite-sized format.
TVR: Do you think it’s experiences like that that will endear VR to the mainstream? There’s folks like us who’ll happily sit down and play games. But what we want is to get VR into people’s homes, and that might mean games need to be more casual. Is it things like VR Sports that will do that?
NM: I think it’ll have a big impact on that, yeah. If you look historically at the Wii and the experience there – I think the Wii did something truly magical, just bringing couch gaming, a really casual form of couch gaming – to the whole family, in a way everyone could appreciate. I think it’s going to be harder with VR to do that, just because a single unit requires a computer.
But what you’re seeing is that on VR and on Rift today, one of the most common use cases is people playing with their friends watching the experience that they’re having. Even in gear where you can’t truly watch the experience – I’m sitting there and my colleague's playing Affected. I’m like, “What do you see? Tell me, tell me.” We’re like playing together. Our research and talking to customers, that’s like a super-common use case. It’s just very common.
So anyway, I think that that’s going to be a big thing. If you look at when you play VR Sports and you’re watching your friend play – it is very, very fun. It is engaging in a whole new way because your friend is truly in that experience. so I do think that some of these experiences that are a little bit more natural for folks, where you don’t necessarily need to know how to use a gamepad, where my dad or nephew can jump in and shoot basketball or play ice hockey or play Dead And Buried or Ripcoil – I think that will appeal to a broader audience, which will bring more people in.
TVR: Let’s talk a little bit more about VR just generally and Oculus’ place in it. We’re very excited about VR. Can you just tell us a little bit about where you think the VR market is right now, and what the next steps are going to be? What are we going to see over the next 12 months?
NM: Let’s start with mobile, which is maybe a little bit easier. I think on mobile you’re going to see this explosion around the mobile drop-in market. We’ve already seen tremendous engagement on the Samsung Gear VR side, both in terms of sales and users – like actual active people using the device on a daily basis. We had over a million unique people use Gear VR last month alone.
So there’s a huge amount of excitement around the whole drop-in. I think that space is going to continue to heat up and grow. I think if you watch any of the Olympics, at least in the United States – maybe it’s just regionalised markets, but there’s lots and lots and lots of Gear VR ads. I think the interest is there and the excitement is there, especially as more and more people are like, “Hey, have you tried that mobile VR thing?” “Oh yeah, you’ve got to get the Samsung device.”
You’re going to see just excitement build and build and build there. And there’s obviously some other folks getting into this space too, so I think between all of that, mobile VR’s going to continue to grow tremendously. And especially around 360 video, that’s going to be one of the primary use cases on the mobile drop-in site. It’s so easy to put on the headset and check out an Olympics experience on Gear VR. I think that’s going to be one of the key user cases.
On the PC side, I think that’s a bit more interesting. I want to be careful and not pre-announce anything with this answer. But I think that there is— we’ve got Rift and Touch launching later this year. I think motion controls, the content… you know, we’ve been pushing updates to the Rift’s software stack on a monthly basis. We’re shipping software endlessly.
We do the same thing on Gear, but on Rift, we’re just constantly updating and adding new features. The next year, you can imagine, just like in any console cycle, that first generation of content is great. We’re sort of through our first generation of content, and in a way, Touch is our second generation of content – or maybe 1.5.
But when you look at 2017, it’s going to be the evolution of the content – people really starting to get good at: “OK, I’ve got the right spec. I’ve got Touch or a gamepad. We have the new innovations we’re going to bring to the table.”
You’re going to see this new round, this new evolution of VR content that’s going to be super-exciting. The console players have been really good at this. Letting the console sit, in a way, is good because developers start to really develop an expertise. And when you look at PS3’s final titles versus PS3’s starting titles, it’s such a wide gap, because you just give developers time to cook and understand everything.
So the content is going to be a tremendous focus for us. I think there will be an interesting… we’ll see what happens next year, as far as new hardware. I think there are a few opportunities for folks to make updates to VR hardware that’s out there, whether it’s input, whether it’s headsets, whether it’s potentially new things.
I think 2017 and 2018 will both be exciting years for VR.