Evaluation of secondary and tertiary level P/CVE programmes, October 2023

Nowadays, the importance of external evaluations in the field of P/CVE practice is widely acknowledged. With a variety of programmes and approaches in this field it is crucial to comprehend what works and what does not work and to fully understand how these interventions promote change and impact on the problems they aim to tackle. Quality assurance measures as well as evaluations are confronted with the fast-moving and complex nature of extremist scenes, deradicalisation dynamics, socio-political developments and changing funding structures. Approaches must do justice to the varying characteristics of target groups, as well as to the variety of different stakeholders involved in planning, designing and implementing P/CVE measures. Evaluations are, however, indispensable to make grounded statements on good practices and in order to strengthen professional practices across the whole programmes and projects lifecycles.

In the vast field of secondary and tertiary prevention (including funding institutions, government authorities, civil society organisations), perspectives on how successful P/CVE programmes should look like, may also vary greatly. It is therefore necessary to develop a more nuanced understanding of the complex nature of secondary and tertiary prevention programmes (also regarding difficult access to target groups and sensitive data) to determine what can and should be reliably assessed by evaluations and under which circumstances. Apart from possibly diverging perspectives and unclear terminology, the process of conducting external evaluations may interfere with the daily work of first line practitioners, e.g. due to short funding periods and limited time during their daily work, but also due the sensitive information and complicated information sharing protocols, which is inherent to P/CVE work. Furthermore, there are important ethical issues that should be taken into account when designing external evaluations.

While there is a noticeable increase in the number and quality of P/CVE evaluations, some interventions still lack any type of evaluation or have only internal self-evaluation as a form of assessment, rarely focusing on change or impact. In our experience, M&E efforts in P/CVE initiatives usually tend to focus heavily on activities and outputs, not on assessing outcomes and broader impact on trends toward radicalisation or violent extremist activity. Some of these difficulties in focusing on change (outcomes or impacts) derive from weak programming, planning shortcomings or in general of intervention designs with poorly defined results and objectives. It is often noticeable that programmes and projects are not based on an underlying Theory of Change. Central question around ‘What is the change we want to promote?’ or ‘How do the project's activities lead to meaningful change?’ often remain insufficiently answered. (Project) planning and evaluation are two sides of the same coin, a poorly designed programme and/or project make the job of an external evaluator much harder.

When designing evaluations for P/CVE initiatives we face both, analytical and practical challenges. One of the most central analytical challenges is the difficulty of attributing change directly to programming efforts when evaluating initiatives, programmes or projects. The effects of secondary and tertiary prevention interventions cannot be tested in isolated settings. This means that there is no possibility to set up control groups, as would be the case in other research endeavors. On the side of practical challenges, data availability and reliability are usually the most common obstacles for evaluating change (outcomes and impact). This paper will further explore these challenges in depth.