AI paints dreamy cityscape that blurs the line between code and art

Researchers have created AI that can render fictional street scenes based on human requests. The scientists load in a “paint by numbers” kind of template, and the computer outputs a dreamy display which rivals the graphics of the current generation of video games.

Qifeng Chen, co-author of the research paper, believes his team has taken the next step in providing increased realism for games like Grand Theft Auto. The scientists created the algorithm and trained it using images from more than 3,000 city streets.

The program references its database and creates something wholly unique based on ‘memory’ and training. Researchers direct the AI with a simple image map that tells the computer what to output. The AI processes the request and outputs a realistic rendering of cars, streets, and buildings that don’t actually exist.

On the left we see the results of the algorithm, on the right we see a screenshot from Grand Theft Auto V. Instead of spending hours painstakingly creating a virtual world, developers may become directors in the future, using simple voice commands like “Computer, create a city block.”

The dreamy way in which this algorithm presents its fictional worlds looks almost hand-painted, making it feel more ‘real’ than traditional CGI. The softness of the image keeps everything on the correct side of the uncanny valley.

Presumably this application for deep-learning can be expanded upon, or layered, with further development of the neural-network it works on. While there are still challenges and limitations, the results speak for themselves.

The ultimate goal, according to the report, is to create photo-realistic images. Chen says “Using deep learning to render video games could be the future.”