Join us at TNW Conference 2022 for insights into the future of tech →

Human-centric AI news and analysis

This article was published on October 7, 2021

Google AI recreates Gustav Klimt paintings destroyed during WWII

A machine learning masterpiece

Google AI recreates Gustav Klimt paintings destroyed during WWII
Thomas Macaulay
Story by

Thomas Macaulay

Writer at Neural by TNW — Thomas covers AI in all its iterations. Likes Werner Herzog films and Arsenal FC. Writer at Neural by TNW — Thomas covers AI in all its iterations. Likes Werner Herzog films and Arsenal FC.

Gustav Klimt created some of the world’s most expensive masterpieces, but around 20% of his artworks have been lost.

Among them are the so-called Faculty Paintings: PhilosophyMedicine, and Jurisprudence.

The three pieces are believed to have been destroyed in a fire during World War Two. Only black and white photos of the artworks remain.

The original paintings may never be seen again, but machine learning has come close to bringing them back to life.

Using AI and the artistic expertise of Vienna’s Belvedere Museum, the Google Arts & Culture Lab developed an algorithm that applied Klimt’s colorization style to the photos.

Google used AI to restore disappeared Klimt masterpieces.
The restored paintings. Credit: Google.

The painter’s palette

AI colorization of black and white photos has been criticized for inaccurately depicting the past.

Paintings add extra complexity to the process, as artists often don’t intend to replicate reality.

Klimt, for instance, may paint the sky with a shocking shade of green rather than a conventional blue.

To understand these choices, the Google team turned to Dr Franz Smola, a curator at Belvedere.

Smola collected comments about the Faculty Paintings artworks from news articles, exhibition catalogs, and letters. This information was then compared to Klimt’s surviving artworks.

Colors from surviving artworks were matched to sections of the Faculty Paintings. Credit: Google.
Colors from surviving artworks were matched to sections of the Faculty Paintings. Credit: Google.

Smola discovered, for example, a description of “golden snakes” in the hair of three women in Jurisprudence. Golden snakes also appear in Klimt’s Beethoven Frieze.

As the two artworks were made during the same time, the team assumed Klimt used a similar tone of gold.

This research was used to develop an algorithm that replicated Klimt’s style.

The artist’s model

The algorithm was first trained on a collection of 91,749 artworks, which allowed the model to understand general aspects of painting. works.

Next, the algorithm was trained on Klimt’s own paintings in order to replicate his colorization style.

The researchers also developed an interface to interact with the algorithm. This was used to apply reference colors from Smola’s research to the artworks.

“If we know that a certain object has a specific color, we add that color directly to the black and white photos,” said Emil Wallner, a creative coder at the Google Arts & Culture Lab, in a blog post.

The interface was used to guide the algorithm.
The interface allowed humans to overrule the algorithm’s inferences. Credit: Google.

We may never know how well the reproductions replicate the originals. But they’re the closest people have come to experiencing the Faculty Paintings for more than 70 years.

Greetings Humanoids! Did you know we have a newsletter all about AI? You can subscribe to it right here.

Get the Neural newsletter

Greetings Humanoids! Did you know we have a newsletter all about AI? You can subscribe to it right here.

Also tagged with