Stereolabs is currently working on some new hardware for virtual reality called ZED. This is a positional tracker that uses stereo cameras, allowing you to go outside and walk around and have full tracking of where you are. The company has been developing it and testing it with a Gear VR, though it should work on just about any VR headset with some minor tweaks. Right now, it is only a developer product, unfortunately.
The company hopes to be able to package it for full integration in mass-produced headsets but that is still a challenge due to its size. Interested developers can get started with the SDK now and purchase the development versions of the ZED for $449. Additional details about the project can be found below.
Stereolabs' ZED camera gives positional tracking to any VR headset
Good positional tracking is the VR industry's biggest obsession, managed by only the priciest headsets. But Stereolabs now makes this tracking possible on any VR headset using an add-on camera.
San Francisco – June 13, 2016 – Positional tracking is what makes the HTC Vive and Oculus Rift so immersive. But it's not without its challenges. Tracking a person's movements in a room requires complex equipment and external sensors such as HTC Vive Lighthouse system, not available on mobile headsets such as Gear VR.
Getting positional tracking to work on mobile headsets has been the obsession of the VR industry, with even Oculus CTO John Carmack spending most of this time on it.
Thanks to computer vision company Stereolabs, developers can now add positional tracking to any VR headset with a single add-on camera. By attaching the ZED depth sensor to a headset, the camera tracks the user's position and orientation in space and sends this data to the headset.
"Positional tracking is the difference between looking at a virtual world, and living in it," says Cecile Schmollgruber, CEO of Stereolabs. "Now every developer has access to it. The next wave of VR headsets is going to be way better than the current generation." Stereolabs demonstrated the feature by demoing a smartphone-based Samsung Gear VR with a ZED attached. A standard Gear VR can only track rotation, or how a user turns their head. But the demo saw users walk, jump, crouch and even dodge projectiles, with every movement captured in the VR experience itself.
The camera's software uses a technique called stereo SLAM (Simultaneous Localization and Mapping) to map the three-dimensional world in front of it in real time and understand how the user moves in space.
This "inside-out" approach of observing the environment rather than the user has been called the holy grail of VR. As it's independent of external equipment, it opens the doors to taking a VR headset anywhere, including outdoors. This is also the first tracking solution for Oculus and Vive users who would like to go mobile and use their headsets with battery-powered VR backpack computers.
"This is just the first step. We're already working on a package that makes it better and easier to integrate," continues Schmollgruber. "After that, we're not at liberty to say. But we've got big news coming up."
The ZED itself works by emulating human depth perception. Two RGB cameras feed their signal to an external GPU, where Stereolabs' software calculates a real-time depth map from the disparity between the images. This technique, called stereovision, has the advantage of working at long ranges and outdoors, both of which pose a challenge for IR-based sensors like Intel RealSense or Microsoft Kinect. The ZED is already used by developers to provide 3D vision for robots, drones, self-driving cars and other projects. Besides VR, this feature can also be used to help such vehicles keep track of their own location as they move around the world without the use of GPS or beacon technology.