Microsoft developed an AI that creates amazing caricatures

Stanford graduate student Kaidi Cao will join fellow AI researchers Jing Liao, of City University of Hong Kong, and Lu Yuan of Microsoft at SIGGRAPH Asia in Tokyo this December to present their incredible caricature-drawing neural network.

That’s not bad, considering Cao was only an intern at the Visual Computing Group at the Microsoft Research Lab in Beijing when he worked on the project.

The AI, actually a pair of generative adversarial networks (GAN), is called CariGANs. The first of its neural networks, CariGeoGAN, determines the geometry of a face in a photograph and maps it to a caricature model. CariStyGAN, the other half of CariGANs, does the “style transfer,” or applies the artistic look to the geometry map.

In order to imbue CariGANs with the ability to turn a relatively boring photograph into a delightful feast for your eyes (tourists on the boardwalk, I’m talking to you) the system was trained on thousands of hand-drawn images.

To determine the efficacy of the machine, the researchers conducted two studies. The first was to ensure the AI’s caricatures retained the identity of the portrait subject. The assertion here is that a good caricature has to capture a person’s essence in exaggerated form. According to the researchers, respondents indicated the CariGANs caricatures compared favorably to hand-drawn artists’.

The researchers conducted the second study to determine if the overall effectiveness of the “drawing” compared with human-drawn pieces. This too appears to indicate success:

Note that ours is ranked better than the hand-drawn one 22.95% of the times, which means our results sometime can fool users into thinking it is the real hand-drawn caricature. Although it is still far from an ideal fooling rate (i.e., 50%), our work has made a big step approaching caricatures drawn by artists, compared to other methods.

The CariGANs AI can also parse frames from video and create caricatures from it. Basically, it can generate a drawing from a single frame that is consistent with ones generated from other frames. The source images in the following picture are taken from the individual frames of an public-domain video of the President speaking.

This could be incredibly useful for animators. It’s also a hilariously spot-on way to look at the president, and proof that “art” created in tandem with an AI can stir something in the human spirit.

The CariGANs AI can also reverse-engineer a caricature and determine what the person in the cartoon really looks like. The researchers say “We believe it might be useful for face recognition in caricatures.”

That sounds a bit terrifying. But if it means I can use an ink drawing of me with a giant head and a sombrero as ID in the future, count me in.