RAN C&N ‘What’s going on online? Dealing with potential use of deepfakes by extremists’, Helsinki 10-11 November 2022

Being online, and doing online work, has become an integral part of everyday life. More and more work in the field of preventing and countering violent extremism (P/CVE) is also taking place online. But the online landscape changes and develops at a fast pace. In an earlier RAN C&N Working Group meeting, practitioners doing online work expressed the need to stay up to date regarding online developments. To cater to this need, the C&N Working Group held a meeting on a topic novel to the field of P/CVE: What’s going on online? Dealing with (potential) use of deepfake technology by extremists.

Artificial intelligence (AI) technology is able to let a computer create video, audio or other media content: synthetic media. AI-created audio-visual content impersonating a person is called ‘deepfake’. But what are the potential usages of this technology by extremists, and what are the implications for the field of P/CVE? And, could P/CVE practitioners potentially use this emerging technology ‘for good’? On 10 and 11 November 2022, practitioners with experience in working online gathered with (tech) experts and researchers experienced in the topic of deepfakes and synthetic media to discuss these questions.

Several experts presented their views on the topic, and all participants took part in a ‘red team/blue team’ exercise to step into the shoes of either extremist groups or P/CVE practitioners to brainstorm about potential usage of synthetic media technology. This meeting was highly exploratory and covered hypothetical scenarios of potential (mis)usage. This allows practitioners to be ahead of the curve and anticipate problems that might emerge soon.

The key recommendations of this meeting are:

• Be aware that attempting to use deepfake or other synthetic media technologies ‘for good’ can be counterproductive. They could easily be used by extremist groups to undermine your goals.
• Take into account ethical and moral considerations. In this paper, you can find a set of questions that can guide you through these considerations.
• Keep in mind that the emotional aspect is key in communicating, both when working to counter or prevent extremist content and when you want to use synthetic media in a positive way.
• Consider working together with emerging tech companies, for example, in deepfake recognition software.
• Incorporate awareness and knowledge about deepfakes and synthetic media in media literacy training.

The remainder of this paper describes the highlights of the discussions held during the meeting, covering the keynote presentations by experts, the red team/blue team exercise, and several current initiatives dealing with the topic at hand. Subsequently, the recommendations following these discussions are formulated as well as potential follow-ups