Poems are one of the favorite topics to explore for AI researchers. Google built one to help you compose poetry that doesn’t look like you shat words on paper, Microsoft built one to convert images into Chinese poetry, and OpenAI’s GPT-3 helped you correct the grammar of your sad prose.
However, this new video created by computer artist and animator Glenn Marshall uses AI to narrate and create visualizations for poems. It’s based on the poem ‘In the Bleak Midwinter’, written by an English 19th-century writer Christina Rossetti.
[Read: This AI-powered gadget could completely disrupt the ridiculous hearing aid market]
The visualizations are striking in a slightly disturbing way. They are not the most sophisticated animations, but rather low-res GIFs. It almost reminds me of band Storm Corrosion’s video for their dark song ‘Drag Ropes.’
Marshall has used the Story2Hallucination library to convert words into the video. It’s interesting to see some parts of the video interpreting words of the poem in a literal sense. For instance, you’ll see a stone emerge from the water to signify “water like a stone.”
For narration, the video uses the vo.codes tool to borrow Christopher Lee’s voice.
Creating visualizations from text is growing as a use case for AI models. Last month, OpenAI revealed a new library called DALL-E that converts words and phrases into images. It would be fascinating to see when poets and writers get access to tools that let them choose from a range of AI models to transform their writings into videos.
Get the TNW newsletter
Get the most important tech news in your inbox each week.