5G could be a transformational technology in terms of bringing VR, AR and MR to the masses, as we've mentioned before. By removing the biggest barriers to entry of cost and convenience in one fell swoop, cloud computing could be the future of XR. Once seemingly an impossible feat, Nvidia recently demoed VR development via the cloud at SIGGRAPH.
At the moment, that would rely on broadband access for sufficient internet speeds, which is of no use to everyday applications for AR smartglasses when out of home or office. However, the next mobile tech, 5G has begun to rollout worldwide, with Qualcomm forecasting 2.8 billion connected devices by 2025. Once blanket coverage of 5G has been reliably achieved, users would be able to access remote processing for XR covering everything from games to navigation and networking.
While VR holds great promise, AR and MR use cases have even greater transformational potential.Erik Ekudden
Transforming industry and society
Ericsson CTO Erik Ekudden says in a recent blog that XR technologies offer, "Enormous potential to transform both industry and society". He also notes that, "Widespread adoption has been hindered by heat generation issues as well as the limited processing power, storage and battery life of small-form-factor head-mounted displays (HMDs). 5G makes it possible to overcome all of these challenges by offloading most XR processing to the mobile network edge".
Ericsson describes XR functionality as real-time media that is, 'Time-critical... with demanding requirements on reliable low latency'. For his part, Ekudden believes that, "While VR holds great promise, AR and MR use cases have even greater transformational potential".
He acknowledges that, "Many smartphone users have already experienced basic forms of AR, through games like Pokémon Go and apps that enable shoppers to visualise new furniture in their homes before making a purchase". We wholeheartedly agree when he continues to say, "AR technology becomes much more powerful, though, when it is used with HMDs".
The mobile network edge
Standalone headsets are now the best-selling VR hardware - with Oculus Quest 2 the most popular of any headset on Steam - but Ekudden says, "These are relatively large and heavy and cannot provide the same experience as when off-device processing is used". We'd be the first to say that while standalones are great, the experience just can't match high end PC VR. Yet.
And maybe they never will - not in their current hardware format, at least. Ekudden is of the opinion that, "Building fashionable, small form factor HMDs to meet the demands for XR is challenging due to limited processing power, storage, battery life and heat dissipation". The solution? "We believe that the best way to address these challenges is by offloading parts of XR processing to the mobile network edge".
Mobile XR connectivity requirements depend on the level of split architecture and the targeted Quality of Experience.Erik Ekudden
As you can see from the diagram above, complete solutions range from the cloud offering minimal support for local hardware to completely servicing a dumb headset. This may be reductive, but we can imagine local commands such as gesture input and gaze being processed locally, with content such as a virtual world or environmental data - graphics, audio - being piped in via 5G.
Regardless of its extent, 5G XR processing will doubtless require extensive compression, with proposed techniques including delta encoding, linear extrapolation, polynomial regression, spline extrapolation and more - think MP3 or MP4 files; will we see MPX in the future?
According to the post, mobile XR connectivity requirements depend on the level of split architecture and the targeted Quality of Experience (QoE), leading to a wide range of bit rates and bounded latency requirements as you can see in the diagram below.
By capitalising on their ongoing 5G rollouts, mobile network operators are in an excellent position to enable the realisation of XR on a large scale.Erik Ekudden
Any XR experience will require heavier and faster processing than conventional cloud gaming. VR typically needs higher bit rates for video than cloud gaming to support retinal resolution, but AR could actually be less demanding by rendering video to only a portion of the display. In that scenario, the uplink is also less critical for say, object detection or tracking, than the download of video data with a target latency of less than 20ms - essential for avoiding nausea.
Ekudden suggests that, "By performing object detection in the network and tracking on the device, the bounded latency requirement for a 5G network can be relaxed up to 50ms". To help reduce latency, Ekudden proposes a framework that separates XR traffic from Mobile Broadband (MBB) services saying, "This will require smarter and faster network algorithms as well as stricter processing requirements at the device".
Ericsson has carried out 5G XR simulation studies for both wide-area and indoor enterprise deployments. As you might expect, the results suggest that the cell capacity in the traffic served decreases by increasing the minimum user bitrate requirements or by reducing the maximum latency target. According to Ekudden, "These results show that communication service providers can start to address cloud gaming and low-end AR use cases… with the ongoing 5G network rollout... and gradually evolve capabilities and densify deployments to address greater coverage and more demanding requirements".
Ekudden concludes that, "By capitalising on their ongoing 5G rollouts, mobile network operators are in an excellent position to enable the realisation of XR on a large scale".