How Photoshop’s new Neural Filters harness AI to generate new pixels

Adobe today launched a new AI-powered workspace for Photoshop that can apply a range of different effects and filters to photos within seconds.

The first of these so-called Neural Filters are Skin Smoothing, which automatically removes blemishes, skin spots, and acne with one click, and Style Transfer, which applies a range of artistic styles to images.

Adobe’s also released six beta filters that the company aims to improve through feedback from users. A particularly striking beta filter is Smart Portrait, which uses a neural network to transform features including a subject’s age, expression, pose, and lighting.

Adobe is far from the only company using filters in photo-editing. But the new features harness a novel combination of face segmentation algorithms and generative adversarial networks (GANs) to accelerate the workflow.

“With GANs, what we’re introducing for the first time, is the capability to generate new pixels,” Alexandru Costin, VP of Engineering for Abobe Creative Cloud, told TNW.

[Read: All the new AI-powered features Adobe announced for its creative suite]

Skin smoothing, for example, was previously done in Photoshop by copying and pasting pixels, a time-consuming process that didn’t always produce realistic-looking results. The new filter is instead trained on tens of thousands of before and after images that enable it to produce new pixels for different situations.

The Neural Filters can also add new features that didn’t exist in the original images, such as teeth that are generated from a GAN to give the subject a smile.

Adobe collaborated with Nvidia to develop the Neural Filters, and then reduced the model’s biases, such as ethnicity and age, by adding another 130,000 stock images to the training dataset. Users who spot any remaining biases can provide feedback and an anonymized image to Adobe that can be added to the training dataset.

After retraining the model, Adobe incorporated high-resolution techniques from Photoshop to improve the quality of the output.

The end result is a platform that should make a lot of image edits a whole lot easier. Photoshop 22.0 customers can try it out by opening the app and clicking Filters > Neural Filters.