Why AI Is Crucial To Facebook’s AR Plans

Why AI Is Crucial To Facebook’s AR Plans

Facebook CEO, Mark Zuckerberg, may have opened their F8 event yesterday in fine form proselytising about AR – and if you haven’t caught up with his opening address, find out the key points here - but he wasn’t the only member of senior management to take to the stage to disclose the social media company’s progress in the sector.


CTO Mike Schroepfer’s presentation focused on AI, its development over the last two decades and the implications for computer vision and object recognition. With Facebook’s recent public shift from VR to AR comes an increased emphasis on AI in creating immersive experiences.

In order to blend the real world with the virtual – or in this case, augmented – one, the technology needs to accurately map the user’s surroundings and recognise local objects in the scene, consistently and quickly. That’s where AI comes in.


Subtraction In AR

Schroepfer illustrated just one potential use case with a pair of video clips demonstrating Mask R-CNN – released by the Facebook AI team last month - in action. The second clip showed how straightforward it appeared to be to remove a windsurfer in motion from video of the sea. Most AR demos depict elements being added to the scene, so it was interesting to see the opposite scenario. We can’t know if there was any pre-render process, but based on this evidence, it looks seamless.

Removing the windsurfer that's 'ruining' a holiday video.

The Mask R-CNN framework for instance segmentation.

The First Step On A Long Road

Just like boss, Zuckerberg, Schroepfer was also keen to emphasise that this is just the first step on a long road, saying that, “Step two is taking these algorithms and getting them squished down into the cell phone in your pocket.” Today is the second and final day of F8, when Schroepfer will share more details on AI and how Facebook plans to, “Put this power in your pocket”.


F8 2017 continues today, with a focus on long-term investments in connectivity, AI, VR, AR and hardware. Check back for updates.


Steve has been a games journalist on and off since the first PlayStation made games ‘cool’, but he’s never forgotten the thrill of finding a Space Invaders machine in the wild. Neuromancer is one of his favourite books and The Matrix is one of his favourite movies, so he’s been ready for VR all his life.


No comments
View options
  • Order by latest to oldest
  • Order by oldest to latest
  • Show all replies
Important information

This site uses cookies to store information on your computer. By continuing to use our site, you consent to Steel Media's privacy policy.

Steel Media websites use two types of cookie: (1) those that enable the site to function and perform as required; and (2) analytical cookies which anonymously track visitors only while using the site. If you are not happy with this use of these cookies please review our Privacy Policy to learn how they can be disabled. By disabling cookies some features of the site will not work.