Google’s new tech turns VR headsets invisible

If you’ve ever watched someone wear and use a VR headset, you know how odd they look with a high-tech blindfold on. That’s not good for demonstrating the emerging technology, especially while it still has a lot to prove before it enters the mainstream.

That’s why Google’s Research arm and Daydream Labs have partnered to figure out a way to let spectators see VR headset users’ full faces while they’re immersed in an experience – as if the headsets were practically invisible. The end goal is to allow for less awkward demonstrations of VR technology and experiences on video.

To achieve this, the team first constructs a 3D model of the user’s face, using a color and depth camera in a process that takes about a minute to complete.

Next, they’re handed a modified HTC Vive headset that’s capable of tracking eye-gaze. For a fully immersive video demo, the user is shot in front of a green screen while they’re engaged in a VR experience, with their VR environment composited into the background.

Additionally, the specialized camera setup for filming this is calibrated and time-synced with the headset, which makes it possible to replace the headset with the 3D face model and line it up with the visible portion of the user’s face.

Slapping on a digital top half of a person’s head onto a real image can look unnatural (the phenomenon is known as the ‘uncanny valley’); as such, the team decided to depict a ‘scuba mask effect’ that shows the 3D face model composited with a translucent headset. This reduces the chance of viewers getting distracted by imperfections and artifacts in the 3D model.

In a blog post, Google noted that this solution will go beyond improving VR and mixed reality experiences:

Headset removal is poised to enhance communication and social interaction in VR itself with diverse applications like VR video conference meetings, multiplayer VR gaming, and exploration with friends and family. Going from an utterly blank headset to being able to see, with photographic realism, the faces of fellow VR users promises to be a significant transition in the VR world, and we are excited to be a part of it.

The GIFs above show that Google’s implementation isn’t yet entirely perfect, as one can still tell that there’s some digital trickery involved in making the top half of the user’s face visible. Still, it’s certainly an improvement over watching someone in a VR experience with their headset on.

Google says that it’ll make this tech available in select YouTube Spaces, its studios that are open to popular creators on its video platform.