Ending the Hype Cycle: A Discussion of Today’s Challenges and Opportunities in 360 Degree Video

By Dr. Changyin (CY) Zhou, CEO and co-founder of Visbit

VR has been in a lasting hype cycle since 2015. Until now the focus on VR, and the tremendous potential this technology holds, has been on gimmicks and wizardry – flashy, conceptual videos and the announcement of products that weren’t yet market-ready. However, this year marks a shift in focus toward VR’s practical uses and how to deliver on the promises made to consumers, especially with 360 VR videos. We’ve already seen evidence of this trend at recent industry events including CES 2017, Mobile World Congress 2017 and GDC/VRDC 2017.

Among the many VR formats, 360 VR video has stood out as a lower-hanging fruit opportunity than others. But what the industry can currently deliver in 360 VR videos is still far behind what our eyes are looking for. A common complaint to many existing 360 videos is that they appear blurry and choppy. For many consumers, it is frustrating and confusing.

The World’s Most Viral 360 Degree Video Archive

An In-Depth Look at VR Video’s Reality

Why do VR videos appear blurrier than our regular videos today?

Human eyes of 20/20 vision see about 60 pixels per degree, making 16K the ideal resolution for 360 degree videos. Most modern smartphones, if viewed from an arm’s distance, or TVs, if viewed from a few feet away, can deliver more than 60 pixels per degree thanks to many years of advances in display technology. As a result, they’re both capable of delivering a pleasant experience to 20/20 eyes.

However, VR requires much higher resolution due to its extremely large field of view – a complete sphere. To achieve the same pixel density in VR, you need about six to ten times more total pixels. Therefore, the industry considers 4K as the minimal resolution requirement for VR videos.

Of course 4K is just a starting point. Just as regular video evolved from 240p in the 1980s to 1080p, and is now moving toward 4K, users will also expect a better experience in VR.

A better VR experience will require improvement in three key areas:

• Higher resolution from 4K to 8K, 12K and eventually 16K to achieve 20/20;
• A shift from monoscopic to stereoscopic to deliver 3D effects;
• Going from 360 videos to volumetric videos or even light field to achieve full freedom in VR!

To achieve progress in any of these three aspects, ten to hundreds times more pixels will need to be created and delivered.

If 4K is today’s ‘must have’ standard, with even 8K capabilities in existence, why can’t general consumers already watch VR videos in 4K?

This is because delivering a VR experience has a long pipeline. To deliver a positive experience, it requires that every step in the pipeline be done right and up to the current industry standard – from creation to streaming to display and playback. Although nowadays all professional level VR cameras can already shoot at 4K+ resolution, and 360 videos and VR headsets can also display 4-6K resolution, a critical step – VR streaming – is not able to support the 4K standard yet. In the US, according to Akamai’s 2016 Q3 report, only 40% of home wifi can support streaming 4K monoscopic video and less than 10% of home wifi can support streaming 4K stereoscopic or 6K monoscopic.

Outlook on VR videos’ future

Although VR videos are not up to the industry’s expectations or consumers’ minimal standard yet, its short-term and long-term future is quite positive and even promising. Today, technology advancements across all three major limitation areas – creation, streaming, and display – are happening.

On the content creation side, people are moving forward to bring higher resolution, 3D effects, and interaction into VR videos. Starting in 2017, 4K became a standard for both VOD and live streaming, and at the SXSW 2017 conference taking place now, you hear the VR production community talk more about making 8K VR videos. We all know the GoPro Omni has been able to shoot and stitch 8K VR videos since mid-2016. There are also other cameras such as zCam and BlackMagic rig that can shoot at 6K now. With the very first limited-scope prototypes of 8K VR live streaming and volumetric VR video production making an appearance at MWC 2017 and CES 2017 earlier this year, coupled with the continued improvement of video stitching and editing workflows, the evolution on the content creation side won’t take too long.

On the display side, new headsets with significantly improved resolution are hitting the market. For example, at VRDC 2017, LG announced their new desktop VR headset that has a resolution of 1280 x 1440 for each eye, which is higher than HTC Vive and Oculus Rift, and capable of delivering up to 6K clarity. On the mobile VR side, Sony was the first one to deliver a 4K mobile phone, and it’s expected that an 8K resolution capable mobile VR device will be available in the near future.

Moreover, standalone mobile VR headsets are also emerging. Qualcomm recently demonstrated a reference design of their stand-alone VR headset at VRDC, and said they are aiming to launch it commercially in the second half of 2017. Meanwhile Facebook, Intel, AMD and Samsung are all exploring this new type of mobile VR headset. The best part of this kind of VR headset is that it allows positional tracking, and from the display perspective, people will be able to move freely in mobile VR.

On the content delivery part, we see two kinds of efforts moving this step forward.

First, we see the telecom industry speeding up their pace to bring 5G networks live and fundamentally improve our data transfer capacity. At MWC 2017, not only did Intel partner with Ericsson to show a 5G network prototype, almost all the major players had something to say about 5G. 5G could potentially achieve commercial use in 2020, and by 2025 around 1/8 of all mobile phones will be able to get a 5G connection. Compared to the pace of user experience upgrades, this might not be fast enough.

Secondly, since 2015, a new technology in the VR industry called foveated rendering and streaming has emerged as a more efficient streaming solution, solving the bandwidth issue and filling the VR streaming needs gap. This technology works by only streaming the field of view area that VR users see at the time, and whenever a user’s head or eyes move, the view follows and the streamed data adapts accordingly. Several players in this space are taking two different technical approaches to deliver foveated streaming – some are viewpoint based, while others take a tile-based approach. Although this new technology is still in the experimental stage, VR industry leaders from Oculus, Google, and Vive in attendance at VRDC 2017 in San Francisco all mentioned that foveated rendering and streaming is both hard to be done correctly and probably the inevitable way to go.

Conclusion

Consumer needs for a better watching experience will drive advances in streaming for the VR industry. The majority of users, sadly, are still watching 360 degree videos in 1080p. But technology breakthroughs are being made as you’re reading this. We foresee that 4K 360 degree videos will become the standard for the industry in 2017, and a better quality, more immersive, interactive VR video experience will be a reality in just a few short years.