Google is developing a free moderation tool to help smaller websites identify and remove terrorist material, as new laws in the UK and EU force internet companies to take more action against illegal content.
The software is being developed in collaboration with the research and development arm of search giant Jigsaw and Tech Against Terrorism, a United Nations-backed initiative helping tech companies fight online terrorism.
“There are many websites that just don’t have anyone to enforce the law. It is a very tedious thing to build the algorithms in the first place [and] then you need all these human reviewers,” said Yasmin Green, CEO of Jigsaw.
“[Smaller websites] I don’t want any Isis content there, but there’s a lot of it everywhere [them],” she added.
The move comes as internet companies will be forced to remove extremist content from their platforms or face fines and other penalties under laws such as the EU’s Digital Services Act, which comes into force in November, and the expected UK Online Safety Act expected to become law this year.
The legislation has been pushed by politicians and regulators across Europe who argue big tech groups haven’t gone far enough to control online content.
However, the new regulatory regime has raised concerns that small startups are not equipped to comply and that a lack of resources will limit their ability to compete with big tech companies.
“I’m noticing a big change in the [leading] Platforms are becoming much more efficient at moderation and this is driving terrorist content and COVID hoax allegations higher [other sites]’ added Green.
A 2021 report by the Global Internet Forum to Counter Terrorism estimated that out of every 10,000 posts on Facebook, six would contain terrorist or extremist content. On smaller platforms, this number can reach 5,000 or 50% of the content.
GIFCT, a non-governmental organization founded in 2017 by Facebook, Microsoft, Twitter and YouTube to promote partnerships between many technology platforms, supports the Jigsaw project. The NGO has a database of terrorist content shared by members of its tech companies, which moderation systems can use to identify existing content.
On December 13, Facebook and Instagram owner Meta launched open-source software that other platforms can use to match terrorist content with existing images or videos in the database and highlight it for urgent human investigation.
Jigsaw’s tool aims to take the next step in this process, helping human moderators make decisions about content flagged as dangerous and illegal. It will begin testing at two unnamed locations earlier this year.
“In our experience, terrorists seek to exploit smaller platforms where moderation of content is difficult due to limited resources,” said Adam Hadley, director of Tech Against Terrorism.
Jigsaw has around 70 employees, most of whom work in Google’s New York offices. Green, who became chief executive in July, said the loss-making division is unlikely to become profitable.
“It goes without saying that the deal pays off in the long run… Google needs a healthier internet,” Green added. “We’re helping Google and the internet in ways that add value, even if it’s not about money.”
© 2023 Financial Times Ltd. All rights reserved. May not be redistributed, copied or modified in any way.