Earlier this year, Microsoft opened up its Custom Vision framework to the public, empowering researchers and enthusiasts to effortlessly build and experiment with AI models; and it seems crafty users are already finding interesting ways to put the technology to use.
Recreational AI-ficionado Geoff Boeing, who is currently completing a doctorate degree in urban planning at the University of California, Berkeley, has leveraged Microsoft’s computer vision algorithm to build a bot that captions gorgeous city landscapes.
CityDescriber, as Boeing calls his invention, currently resides on Twitter, where you can follow its awkwardly phrased visual descriptions. All images are sourced from Reddit’s CityPorn community.
As you can imagine, the quirky bot gets a lot of things right, but it does make an occasional mistake here and there (like spotting invisible people and trees).
Here are some of the less accurate ones:
The 💜 of EU tech
The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!
a table full of food pic.twitter.com/it9320OKfS
— City Describer (@CityDescriber) September 14, 2017
a view of a cactus pic.twitter.com/BkV0DSinPx
— City Describer (@CityDescriber) September 11, 2017
a stack of flyers on a table pic.twitter.com/S8M3wNWf2g
— City Describer (@CityDescriber) September 14, 2017
a traffic light hanging from a tree pic.twitter.com/Am9F0ldx1T
— City Describer (@CityDescriber) September 10, 2017
a street filled with water and a city at night pic.twitter.com/bZ6QTDVoCx
— City Describer (@CityDescriber) September 9, 2017
a close up of a brick wall pic.twitter.com/fZKy9dNPwc
— City Describer (@CityDescriber) September 9, 2017
And here are some of the ones it actually describes rather well:
a view of the earth from space pic.twitter.com/ZQHZRW8vOC
— City Describer (@CityDescriber) September 9, 2017
a group of people walking across a snow covered street pic.twitter.com/TujM8ugH9s
— City Describer (@CityDescriber) September 19, 2017
an aerial view of a city pic.twitter.com/YaRm1oUNGg
— City Describer (@CityDescriber) September 19, 2017
a close up of a busy city street pic.twitter.com/S4OM6UyI7f
— City Describer (@CityDescriber) September 13, 2017
Of course, this isn’t the first time someone has used Microsoft’s AI to develop a captioning bot for Twitter.
Previously, self-proclaimed ‘creative technologist’ Brian Moore relied on the same technology to build a bot specifically designed to watch and interpret Pornhub videos for hours. You know what they say: Different, but still the same.
Meanwhile, you can follow CityDescriber’s wonky captions here.
For those curious to find out more about how computer vision actually works, you can learn the basics in under 10 minutes in this awesome explainer video on YouTube.
Get the TNW newsletter
Get the most important tech news in your inbox each week.