Google AI recreates Gustav Klimt paintings destroyed during WWII

Gustav Klimt created some of the world’s most expensive masterpieces, but around 20% of his artworks have been lost.

Among them are the so-called Faculty Paintings: Philosophy, Medicine, and Jurisprudence.

The three pieces are believed to have been destroyed in a fire during World War Two. Only black and white photos of the artworks remain.

The original paintings may never be seen again, but machine learning has come close to bringing them back to life.

Using AI and the artistic expertise of Vienna’s Belvedere Museum, the Google Arts & Culture Lab developed an algorithm that applied Klimt’s colorization style to the photos.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

The painter’s palette

AI colorization of black and white photos has been criticized for inaccurately depicting the past.

Paintings add extra complexity to the process, as artists often don’t intend to replicate reality.

Klimt, for instance, may paint the sky with a shocking shade of green rather than a conventional blue.

To understand these choices, the Google team turned to Dr Franz Smola, a curator at Belvedere.

Smola collected comments about the Faculty Paintings artworks from news articles, exhibition catalogs, and letters. This information was then compared to Klimt’s surviving artworks.

Smola discovered, for example, a description of “golden snakes” in the hair of three women in Jurisprudence. Golden snakes also appear in Klimt’s Beethoven Frieze.

As the two artworks were made during the same time, the team assumed Klimt used a similar tone of gold.

This research was used to develop an algorithm that replicated Klimt’s style.

The artist’s model

The algorithm was first trained on a collection of 91,749 artworks, which allowed the model to understand general aspects of painting. works.

Next, the algorithm was trained on Klimt’s own paintings in order to replicate his colorization style.

The researchers also developed an interface to interact with the algorithm. This was used to apply reference colors from Smola’s research to the artworks.

“If we know that a certain object has a specific color, we add that color directly to the black and white photos,” said Emil Wallner, a creative coder at the Google Arts & Culture Lab, in a blog post.

We may never know how well the reproductions replicate the originals. But they’re the closest people have come to experiencing the Faculty Paintings for more than 70 years.

Greetings Humanoids! Did you know we have a newsletter all about AI? You can subscribe to it right here.