As part of Sony’s build-up to this year’s E3, Dominic Mallinson, Senior VP, Research & Development West, revealed the work involved in bringing the PS VR to market, as well as future innovations such as focus, gaze tracking, talking to games and brainwave interfaces. Here’s what he had to say:

 

On The Process Of R&D

Thinking back to the early days of exploring the concept of VR – even before PS VR was known as Project Morpheus – Mallinson explained, “You brainstorm, you tinker, you try ideas out - all of which is great fun. And then you have to crystalise that and bring it together and finally make something.”

In this case particularly, the focus was clearly on one question, “How can we make the gaming experience as immersive as possible?” When working with prototypes, the team invested a lot of time and energy into elements such as, “How you fit it on your head; the design, the aesthetics, the comfort factors.”

The work obviously paid off, as PS VR is commonly acknowledged as the most comfortable and user-friendly design of the three leading headsets. Mallinson said, “It’s hard work, but it’s rewarding because eventually you see people using it and you realise, ‘Wow – I helped make that!’”

 

On Incorporating Existing Technology

Even a company with pockets as deep as Sony can’t just create hardware from scratch as Mallinson confirmed, “We have to look at what’s available on the market because cost is important.” In this case, Sony’s mobile phone department must have been on speed dial as Mallinson said, “Some of that tech is fundamentally what we needed to build VR, like the high resolution display; the accelerometers and the gyros were exactly what we needed.”

Sony had another advantage over the competition, in that it already had a suitable control interface to experiment with. Mallinson recalled that, “Pretty soon in the development of Move, one of our researchers strapped a PlayStation Move to his head and interacted with the TV - and when he moved his head, the image on the screen moved in the same way. This was 1.0 VR, if you like.”

 

On Future Innovations

In line with other forward-thinking projects from Google and Facebook – read more about that here and here - PlayStation’s R&D lab is now exploring more esoteric areas such as AI and Natural Language Understanding. Mallinson imagines a future where gamers can talk to their games, much as they do to Apple’s mobile assistant, Siri today. He forsees that gamers will, “Use it to talk to game characters and actually have a dialogue instead of selecting dialogue options using a joystick; you’re just speaking naturally.”

 

On Improving Display

One area that Mallinson singles out for improvement on PS VR is the display. The current PS VR hardware uses two panels, one for each eye with, “Stereopsis that gives you a depth cue – it explains to the brain how far away something is. But the brain also thinks about focus... The eye does this all the time, but it doesn’t happen in VR.”

Mallinson would like it to happen in future hardware iterations though, “We want to bring that in. In the future, as we develop these technologies, we’ll get to a place where we can do both stereopsis and focus. That will give us the ultimate visual experience

On Gaze Tracking

Mallinson is clear that his team is excited about the potential for gaze tracking and what that means for gameplay experiences on PS VR. He said, “To build that into a game and have both the players and non-player characters make those same assumptions about eye contact... It can get kind of uncomfortable because you’re staring at them.”

 

On Brainwave Interfaces

And in a nice segue from gaze tracking, Mallinson continued, “If the eyes give you a window into the mind, imagine what you could do if you could read the mind. This is a little but further out, this is almost in the realms of Science Fiction, but it’s grounded in the laboratory as well.” He’s talking about brainwave interfaces, “Where you measure the brainwaves and it gives you an insight into what people are thinking at a very basic level.”

 

On The Future Of Gaming

That said, Mallinson cautioned that, “This is all ideas; this is the future, looking just over the horizon. Maybe we’ll see it, maybe we won’t.” But he remains dedicated to his work saying, “What’s important to us is pushing the future of gaming. It’s driving the next big thing”. And he clearly loves his work, “When we somebody enjoying that technology that we’ve helped to build, it makes us really proud.”