Controlling hate speech online is one of the biggest challenges of our information age. Everyone says it’s important, but how effective is it? Some platforms have chosen to remove individual accounts that spread toxic content. An internal Facebook study that analyzed interactions between 26,000 users shows that excluding extremist community leaders is an effective way to eliminate hate speech on social media, especially over the long term. The removal of just 100 accounts had a tangible effect as it denied the mic to advocates of hate speech and ultimately improved the broader social media ecosystem.
Previous studies had shown that deleting malicious accounts on platforms like Twitter, Reddit, and Telegram helped reduce unwanted activity, including a broader range of hate speech. However, a cause-and-effect relationship was only recently demonstrated by Meta researchers (Facebook’s parent company) in a study published in PNAS, a peer-reviewed journal of the National Academy of Sciences (NAS).
Daniel Robert Thomas and Laila A. Wahedia examined how the removal of the most active representatives from six Facebook communities affected their audiences. The goal of the meta-researchers was to measure how often the audience continued to view, post, and share malicious content after the instigators were removed. The study found that “on average, network disruptions reduced the consumption and production of hateful content and viewer engagement within the network.”
After the accounts were deleted, users watched almost half as much hateful content every day. In other words, people who once viewed five posts with toxic content viewed fewer than three. Additionally, those who stopped interacting with toxic community members were exposed to other content, groups, or communities that were not explicitly associated with violent behavior. However, Facebook’s privacy policies prevented data tracking of certain user accounts throughout the study.
Organizations that spread hate may retain a loyal audience for a while, but the expulsion of their leaders may drive away some viewers. Meanwhile, those less connected to these executives are less likely to engage with this content at all. This is a positive finding as this group is the most vulnerable to the influence of malicious communities. “The results suggest that targeted removal strategies, such as For example, executive removal and network disruption efforts can reduce the ability of hate organizations to thrive online,” the study concludes.
But there is no silver bullet that can kill this werewolf. People who are banned from a platform can easily create new accounts and build new networks. You can also migrate to other platforms. In addition, the authors suspect that other toxic organizations may be taking over the deleted accounts and gaining sympathizers for them. To increase the effectiveness of the deletion strategy, the authors suggest removing multiple accounts at once, as this affects an organization’s ability to find and regroup its members.
Hate speech or toxic speech?
But if the decision to delete accounts is left up to the platforms, will they really want to do that? Sílvia Majó-Vázquez, research fellow at the Reuters Institute for the Study of Journalism at the University of Oxford (UK) and professor at the Vrije University in Amsterdam, said that moderating content on social networks is “by finding a balance between freedom of expression and respect for other rights”, so it is important to distinguish between hate speech, toxic speech and impoliteness.
Majó-Vázquez says that rudeness, like disrespectful and sarcastic comments, is the mildest form of negative language. But when it gets more extreme and “dissuades people from engaging in conversation,” toxic language emerges that can become violent. “From a democratic point of view, this is very damaging because it discourages the democratic ideal of public debate,” she said.
In order to ensure freedom of expression on social media platforms, careful thought should be given to account suspension or deletion. According to Majó-Vázquez, the suspension process must include conceptual dimensions and use manual mechanisms that sufficiently reconcile the right to freedom of expression with respect for other fundamental rights. She advises applying a similar approach to political figures. Automated mechanisms for deleting messages and banning accounts must be continually reviewed, with an emphasis on expert message evaluation, similar to the external advisory boards that some platforms have already implemented.
According to a recent Reuters Institute study in seven countries, the link between toxicity and engagement is not always direct and varies by content topic and severity. The study analyzed Twitter data during the pandemic and found that the most toxic tweets were often unpopular with audiences. “In fact, we’re seeing the most toxic tweets lose popularity and low-toxicity messages gain popularity,” Majó-Vázquez said. The study was inconclusive on whether this was due to audience dislike of toxic content or the moderation techniques employed by the platform. “We can’t answer that question with the data from our study, but this result challenges the assumption that toxicity is always the most popular online currency,” she said.
Sign up for our weekly newsletter for more English language news from the EL PAÍS USA Edition
