Facebook is teaching AI to talk you through directions

Facebook is developing a “Talk the Walk” AI capable of giving walking directions without knowing a user’s location.

What it is: A team comprised of a researcher from the University of Montreal in Canada and Facebook Artificial Intelligence Research (FAIR) scientists recently published a white paper describing a neural network capable of giving a person plain language directions without the use of GPS or other location tracking aids.

According to the researchers:

We introduce the Talk the Walk dataset, where the aim is for two agents, a “guide” and a “tourist”, to interact with each other via natural language in order to achieve a common goal: having the tourist navigate towards the correct location. The guide has access to a map and knows the target location, but does not know where the tourist is; the tourist has a 360-degree view of the world, but knows neither the target location on the map nor the way to it. The agents need to work together through communication in order to successfully solve the task.

How it works: Hypothetically, if this neural network were to be fully fleshed-out, it could provide end-to-end directions to a person, even if location services and internet connectivity was unavailable. In such a case, it would function by allowing users to have a conversation with an AI in much the same way they would with a human.

The “tourist” describes the landmarks they see, such as “I’m standing next to a theater,” and the AI tries to determine where they are. It can ask questions in return, for example it may ask if the user sees a shop on the corner to help narrow down which theater they’re looking at. Once it determines where the user is, it gives a plain language response guiding them to the next waypoint.

When it’s coming: Maybe never, as this isn’t a new feature the company is rolling out. It’s early research which appears to lay the ground work for future development. The “Talk the Walk” white paper establishes a data-set and some basic algorithms to prove that the concept works, but it’s far from being ready for prime time.

The major significance of this work is in its focus on creating AI capable of working together with humans to achieve a goal.

Go Deeper: You can read the white paper here. To learn more about neural networks read our primer here. And don’t forget to visit our artificial intelligence section for all the latest news and updates in machine learning.