Google has been working on improving its AI and computer vision for quite some time now, and it looks like they’re getting pretty darn good at identifying objects. The company is now building a clever Android app that takes advantage of these developments to help the visually impaired understand what’s around them.
Lookout is designed to work with Google’s Pixel phones, worn in a lanyard around your neck, or kept in your shirt pocket with the rear camera facing outward. It can recognize things like printed text, signage, people, and objects like chairs and tables, and notify you of their locations with audio cues like ‘couch at 3 o’clock.’
That should help people with vision problems move around more independently. Google says the app will work offline, and support four different modes to deliver relevant information, depending on what you’re doing:
If you’re getting ready to do your daily chores you’d select “Home” and you’ll hear notifications that tell you where the couch, table or dishwasher is. It gives you an idea of where those objects are in relation to you, for example “couch 3 o’clock” means the couch is on your right. If you select “Work & Play” when heading into the office, it may tell you when you’re next to an elevator, or stairwell.
The fourth mode, Experimental, is meant for testing out Lookout features that are still in development.
The 💜 of EU tech
The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!
The app sounds similar to Microsoft’s Seeing AI, which was launched on iOS last year and works as a ‘talking camera,’ audibly describing things around the user as they point their camera at people and objects. However, Lookout seems like it could be more useful as its various modes can help highlight only the important objects in one’s surroundings based on what they’re doing – and therefore cut out a whole lot of noise from the app.
Lookout might work well in conjunction with something like Microsoft’s other app, SoundScape. It helps people get around and become aware of exactly where they are, by pairing with a stereo headset and calling out the names of roads and landmarks that they pass by.
The app will become available to Android users in the users later this year.
The Next Web’s 2018 conference is just a few weeks away, and it’ll be ??. Find out all about our tracks here.
Get the TNW newsletter
Get the most important tech news in your inbox each week.