Human-centric AI news and analysis

Russia’s most notorious troll farm reportedly used deepfakes to push a fake news outlet on Facebook

Facebook and Twitter both said they've taken down a network linked to Russian state actors

Facebook and Twitter said on Tuesday that they’ve removed a network of accounts that were promoting a fake news outlet, which was staffed by fictitious journalists with AI-generated headshots.

According to Facebook, the network was linked to Russia‘s most notorious troll farm: the Internet Research Agency (IRA). The company says the network primarily sought to amplify a purportedly independent left-wing news site called PeaceData that was run by fake editors.

These personas then recruited unwitting freelance journalists to write English and Arabic-language articles about topics of legitimate concern, such as racial justice, environmental issues, and President Trump’s policies.

The publication may have been fake but I wouldn’t call its articles ‘fake news.’ Credit: Facebook.

The data was analyzed by Graphika, a US government-linked social media analytics firm whose partners include DARPA and the US Senate Select Committee on Intelligence. The company said the operation was focused on “left-leaning voters in the United States and United Kingdom who may be dissuaded from supporting the more centrist leadership of the Democratic and Labour parties.”

[Read: We asked 3 CEOs what tech trends will dominate post-COVID]

Graphika analyzed six of the PeaceData editors’ profile pictures and concluded all of them were created with Generative Adversarial Networks (GAN). The firm based this conclusion on their asymmetrical features, distorted backgrounds, and by superimposing the images on one another to show they align more than real photos.

Left: screenshots of six of the personas. Right, all six superimposed and make more opaque to
show how their eyeballs align. Credit: Graphika.
The glasses in the photos contained asymmetries where the arms join the frames. Credit: Graphika.

The network was tiny compared to the thousands of users reportedly involved in previous IRA operations. In total, Facebook identified just 13 accounts and two pages on the platform, while Twitter suspended only five accounts.

It also received little engagement: only 14,000 accounts followed the two pages, with just over 200 of them following the English-language page. As for the content on PeaceData, almost all the articles were either copied from other websites or written by freelancers.

While the network apparently had little impact, its use of deepfake images was a noteworthy precedent: Graphika said it’s never seen IRA-linked accounts using AI-generated avatars before. But their creators clearly need to sharpen their artistic skills before their next campaign.

So you’re interested in AI? Then join our online event, TNW2020, where you’ll hear how artificial intelligence is transforming industries and businesses.

Published September 2, 2020 — 15:07 UTC