My Account
Extremists are becoming increasingly sophisticated at exploiting social media algorithms to indoctrinate unsuspecting internet users, warns a local IT expert.
Ti Point’s Nat Torkington, who advised Jacinda Ardern at the Christchurch Call summit in May, says the more critically people consume what they see online, the better.
“There are intelligent adversaries using social media platform systems against the public. They will use all the features they can to make a platform’s algorithms recommend their extremist content.
“One of the things white supremacists or violent extremists do is coin new terms that they embed in material, so that when people search for its meaning they will see extremist content.
For example, the Christchurch gunman had a phrase written on both his gun and van, which was reported on by media and subsequently led to people searching for it on Google.
Some platforms may continue to recommend similar content, therefore exposing individuals to further radical ideology.
“This is dangerous because research shows that if someone is presented with a false idea online that is convincing, it is hard to overthrow that bad idea, even when the truth is presented. It is not just a matter of the truth winning out in the marketplace of ideas,” Nat says.
Extremists are also exploiting the comment sections of social media platforms such as YouTube to further their content.
“For example, if you search for vaccination information you might get a video from the centre for disease control, which will provide facts, but then in the comments someone will link to a video that raises disturbing questions about vaccination.
“That video might only be one step away from scientific content, just raising questions about it, so you might click it. But then, in the comments on that video, extremists will link to a video that is a degree more extremist in nature and before you know it you are being exposed to a conspiracy theory suggesting vaccines cause autism.
“Other techniques used by extremists allow them to deliver their content to you whether you make a conscious decision to click on it or not.
“If enough people view two videos, one after the other, then a social media algorithm will decide that those two videos must both appeal to similar people. As a result, a link is created and the algorithm will recommend the second video to those who watch the first.
“Extremists exploit this by getting a large group of their members to view what might be an ordinary video and then immediately watch a video containing their own ideology.
“Again, the links that they create are escalating in radicalisation so that you start out on something relatively benign, but the content becomes increasingly extreme. That’s why people remark that all paths online lead to crazy,” Nat says.
“The longer you stay online and click, the more you increase your chance of being exposed to radical material.”
