Facebook’s new search AI lets you find images by content

Facebook now lets you search for images by typing in the actual content within them, much like Google Photos.

The technology – called Lumos – was first used to improve Facebook’s accessibility for visually-impaired users, allowing images to be described by their content. But the technology has other uses, such as making it easier to find photos of places or objects. E.g. looking up all the photos of pugs you’ve taken over the years.

https://www.facebook.com/Engineering/videos/10154898351752200/

Facebook analyzed billions of photos using deep-learning techniques, using a variety of signals in order to rank results.

Object recognition allows it to search for things like scenes, animals, places, attractions and clothes. Searching for a “black shirt photo” will pull up images where people are wearing a black shirt, even if there’s no text or tag in a photo indicating the presence of a black shirt. Searching for “photos of chihuahuas” brought up, well, a bunch of chihuahua photos (although ironically, not of my own two).

Furthermore, Facebook is working on understanding what people are actually doing in the images. It gives the example of searching for “people walking,” “people dancing,” “people riding horses,” “people playing instruments,” and more.

Of course, Facebook isn’t the only one to implement computer vision. We’ve seen it in the aforementioned Google Photos, while resources like like Cloudsight are to make computer vision tools accessible to more people.

The company says it’s looking to bring its AI recognition to video in the future, but that’s likely a ways off. For now, Facebook tells me image recognition should be available to all users in the US (no word on other regions for now), so go ahead and look for all the cute animal photos you want.