Microsoft’s new ‘canetroller’ brings VR to the visually impaired

Microsoft recently developed a haptic controller for VR with an unusual target audience in mind: the visually impaired.

Developers from Microsoft’s research lab created the “canetroller,” a peripheral device designed to work with the HTC Vive headset and tracker. The team’s goal was to create a device that would successfully enable the visually impaired to form an accurate mental map of a virtual space.

The canetroller translates the traditional experience of using a white cane, a long stick used by the visually impaired to tactilely scan their environment, into a virtual reality environment.

With VR you can have an out-of-body experience to see what death is like, or become a stormtrooper from Star Wars in a game where you can feel and smell an alien planet. But it isn’t a technology often associated with helping the blind.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

The canetroller overcomes the need for visual cues by using vibrations and sound to simulate the effect of tapping different virtual objects and a wearable braking system to make walls and barriers seem solid.

There are still some limitations to overcome — for example one participant found that the audio modeling, particularly the way sounds echo off of solid objects, wasn’t quite up to real world standards:

I didn’t have a good sense of direction where I was at [in the real world]. I can hear roughly where the wall is at, by the way it blocks off sound in the real world. I didn’t have that in the VR world.

But, overall the experiment shows promise. In the indoor simulation eight out of nine study participants were able to successfully navigate a room, and six were able to complete the outdoor sim without issue.

An orientation and mobility instructor working with Microsoft on the study felt it was a particularly useful tool:

It can be used with students who might be nervous about crossing a particular busy street. Let’s try it this way first… without having the risk of hitting by [a] car.

One of the most important features of the canetroller is its ability to work like a full-length white cane, which is typically much longer than the actual device itself. Since the braking system the researchers developed only inhibits side-to-side motion to make virtual objects like walls and curbs seem solid, the team had to come up with a way to limit vertical travel so users would be able to determine when their virtual cane was penetrating the floor or poking through walls.

The team accomplished this using the Vive’s headset tracking, the additional tracker, and beeping sounds to notify the user they’d “pushed” through an object. The entire package creates a novel system of navigation that could better prepare the visually impaired for difficult situations like crossing streets in heavy traffic.

It’s amazing to see a technology usually meant for visual immersion adapted for the visually impaired. We can’t wait to see (or hear and feel) what the team does next with the canetroller project.