Robot chef learns to cook by watching humans make the recipes

As a lover of good food, but someone who dislikes cooking, I’ve always fantasised about having a robot chef at home. Now, thanks to the work of researchers at the University of Cambridge, my dream may soon come true.

The research team has succeeded in training a robot to watch cooking videos, learn from them, and then recreate dishes. “We wanted to see whether we could train a robot chef to learn in the same incremental way that humans can — by identifying the ingredients and how they go together in the dish,” said Grzegorz Sochacki from Cambridge’s Department of Engineering, the study’s leading author.

To test that, the scientists created eight simple salad recipes and recorded videos of themselves preparing them. Then, they trained the robot using a publicly available neural network, which had been pre-programmed to identify a variety of objects, including the fruits and vegetables used in the salads.

The robot chef analysed every frame of the video, using computer vision methods. It was able to identify not only distinct objects, such as knives and ingredients, but also the human demonstrator’s actions. The recipes and the videos were converted into vectors and, doing mathematical correlations, the robot recognised similarities between a demonstration and a vector. As a result, it was able to determine which of the recipes were being prepared.

The mechanical chef watched 16 videos in total and managed to identify the correct recipe 93% of the time. In addition, it was able to recognise variations in a recipe and the demonstration of a new, ninth salad, which it added to its cookbook and recreated.

For now, the robot’s success requires recipes that aren’t complex, while fast-paced food videos on social media would simply be too hard to follow. “But as these robot chefs get better and faster at identifying ingredients in food videos, they might be able to use sites like YouTube to learn a whole range of recipes,” Sochacki said.

The research team notes that the robot chef’s capacity is still limited with many bottlenecks to be overcome. But they have succeeded in showing that the robot can incrementally learn how to cook from human demonstration on video — enabling the easier and cheaper deployment of robotic chefs.

I know I’ll be waiting impatiently for that day to arrive.