Yelp created the hotdog app from Silicon Valley (only better)

Yelp’s Photos Team is developing AI and machine learning algorithms to understand content in user photos. Using neural networks, the systems are able to identify specific attributes that make a photo “beautiful.”

The neural network can also identify and categorize based on the type of dish in the photo ー meaning that Silicon Valley’s hotdog-identifying app actually exists.

The motivation behind the AI is the rise of photo sharing at restaurants. People love snapping and gramming the trendiest spots, dishes, and menus, but they also love posting on Yelp. Over 100,000 photos are uploaded to Yelp per day, so the Photo Team leverages that content to help people make decisions like what dessert to order and where to eat.

The mouth-watering photos featured are based on certain attributes like the depth of field, contrast, and symmetry. This model determines how appealing a photo is and surfaces content around groups of food, drinks, menus, and environments people are looking for. Relevant, high-quality pictures get pushed to the top, and old, low-quality photos find their way to the bottom.

The AI can identify 20 types of dishes, including, but not limited to: pizza, sushi, salads, tacos, donuts, burgers, cupcakes, burritos, hot dogs, and pancakes.

People are obsessed with food pics, but according to Product Manager Lindsay Goyne, they’re also obsessed with looking at menus. The fact that people can see who took the photo, the date they uploaded it and their review of a menu item, signals a sense of relevancy and authenticity. If a restaurant uploaded their own menu ー it wouldn’t elicit the same “word of mouth” human response people are looking for.

Yelp’s photo team is also working on smart cropping. Instead of featuring the entire picture, machine learning algorithms identify the primary focus while cropping everything surrounding it. So if a user uploads a picture of a pizza with a blurry person stuffing their face in the background, their oily mess won’t be included in the picture.

Yelp is doing a major service to restaurants that don’t post their own pictures because a whopping 86-percent of customers regularly check out menus online before they dine out. And a stunning 60-percent always or frequently check out images of restaurants and food before they go. Businesses and restaurant goers owe Yelp a thank you.