Google’s AI to detect toxic comments can be easily fooled with ‘love’

A group of researchers has found that simple changes in sentences and its structure can fool Google’s perspective AI, made for detecting toxic comments and hate speech. These methods involve inserting typos, spaces between words or add innocuous words to the original sentence.

The AI project, which was started in 2016 by a Google offshoot called Jigsaw, assigns a toxicity score to a piece of text. Google defines a toxic comment as a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion. The researchers suggest that even a slight change in the sentence can change the toxicity score dramatically. They saw that changing “You are great” to “You are fucking great”, made the score jump from a totally safe 0.03 to a fairly toxic 0.82.

This clearly denotes that the toxicity score is probably not the best measure to identify hate speech. Last year, another study found that inserting spaces and making typos reduced the toxicity score drastically. Google has improved its AI since then to detect these changes. But it’s not perfect, the researchers presenting the latest study said if someone introduced a word like ‘love’ in these sentences the score took a plunge.

So anyone can probably introduce a few positive words in a hateful sentence, to reduce the score, or insert a few cuss words, to increase the score.

The 💜 of EU tech

The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!

Historically, tech companies and their algorithms have struggled with hate speech. In 2016, Microsoft had released a Twitter bot called Tay, whose tweets quickly turned abusive, as it relied on the user responses. Twitter, meanwhile, had a curious case of banning userswho had the phrase ‘Kill me’ in their tweets without knowing the context. And between October 2017 and March 2018, Facebook’s systems were able to filter out only 38 percent of the hate speech posts, made its way to the platform.

Google’s team should be working on understanding the context of a particular word used in a sentence and detecting intentional and unintentional typos which can game the system. We have written to Google to learn more about the project.