[authorbox authorid=”73″ title=”The Author”]
Looking back on the past 15 years, there is little doubt that the 9/11 attacks profoundly altered the landscape of international security. Over the last decade and a half, there has been a tremendous multiplication of counter terrorism measures, laws, policies at regional, national and international levels.
The events of 9/11 and -more recently- the phenomena of ISIS, “homegrown terrorism” and “foreign fighters”, set states and international bodies on an accelerated path of law making and institution-building.
Domestic and international counter terrorism initiatives include specific provisions to permit, require and fund technologically innovative measures to counter terrorism; thus smart borders, (smart) surveillance, data retention/collection and sharing, passenger name record exchange etc., feature heavily in the hyper technologized operational continuum of contemporary counter terrorism, where the production, retention, processing, and deployment of data has taken centre stage.
Although an array of difficult questions stems from the current risk prevention rationale regarding counter terrorism practices and the balancing act between security and democratic oversight, we will take a step back from the specifics of technological advancements (in relation to counter terrorism measures) to focus on a more structural, ethical and deontological aspect: effectiveness.
It is no secret that research conducted to evaluate the actual effectiveness of counter terrorism measures is worryingly insufficient or outright non-existent.
Taking into account that the adoption of certain counter terrorism practices can potentially have implications with regard to human rights, institutional liberties, core democratic principles and the Rule of Law, it is crucial that engaged and systematic efforts in researching the effectiveness of counter terrorism measures are implemented in a rigorous way.
The lack of empirical research is -by no means- a problem exclusive to counter terrorism but empirical evaluation is vital in clarifying and debunking prevailing beliefs regarding counter terrorism practices. It must be noted, however, that one can discern a number of challenges and technical difficulties when attempting to measure the effectiveness of counter terrorism.
Van Um and Pisoiu argue that a major point of contention is the lack of a universally accepted notion of what an effective counter terrorism policy is supposed to bring about. In other words, there is a need for a solid conceptual framework that could provide a critical overview of effectiveness -understood in terms of the extent to which objectives for the measures implemented have been achieved-.
Framing Context
It is perhaps trite to remark that terrorism –and counter terrorism for that matter- did not begin on 9/11; rather, terrorism has long been employed as a modus operandi by organizations and individuals, attempting to achieve sociopolitical and/or other goals.
—
For certain countries, violence associated with terrorism has long been a fact of life and many of these countries have had relevant anti terrorism legislation in place for years. Thus, in effect, laws, policies and regulations (occasionally with a plethora of implications for the social equilibrium) pursuant to a risk prevention/securitization rationale are nothing new at the domestic level.
It is also not the case that the concept of “transnational terrorism” is fundamentally or qualitatively new. Standard terrorist operating procedures -i.e. shopping center bombings, assassinations, suicide bombings, hijackings, civilian aircraft bombings, spree shootings etc. – were repeatedly observed far before 9/11.
Nevertheless, 9/11 did represent a significant, path-breaking moment in a broader historical trajectory, since it was collectively perceived as an unfathomable trauma, ushering in a qualitatively new time of fear and insecurity.
Yet, for all of that, it is safe to say that the September 11 attacks certainly marked the end of an era and the beginning of another, mainly because they sketched the contours of contemporary counter terrorism and shaped the context in which the notions of justification, effectiveness and impact (of counter terrorism measures) are analyzed.
Conceptualizing Effectiveness
The conceptualization of effectiveness (of counter terrorism measures) has been complicated by the variability and multidimensionality of the empirical phenomenon it seeks to describe. In other words, the understanding of effectiveness is largely dependent on the perspective, priorities and broader purposes of the person or entity making the assessment. In the abstract, effectiveness could be positivistically equated with “the achievement of general objectives”.
In the context of counter terrorism, however, this simple methodological continuum is complicated by three exogenous variables: i) the fact that particular counter terrorism measures are part of a broader operational framework and may have both meta objectives (relating to overall security matters) and specific objectives (relating to the measure in particular), ii) the fact that both the meta- and specific objectives may not be lucidly elaborated iii) the possibility of certain measures devised for one purpose being applied to other issues or having unexpected effects in relation to other areas.
As already suggested, a universally accepted definition or framework of counter terrorism effectiveness does not exist in the literature to date. The literature is currently limited to a hypothesized conceptual self evidence or specific indicators regarding stand-alone terms such as impact, success, effects, etc. Subject-matter expert Martin Mazé defines effectiveness based on whether the measures “actually make a difference” and particularly focuses on the societal impact as an effectiveness indicator.
He expressly notes that ex post facto evaluations rely heavily on outputs rather than impact per se; subsequently the effectiveness analysis risks excluding key societal indicators. It is worth noting at this point that the societal perception regarding the effectiveness of counter terrorism measures is also largely determined by three different arguments, central to the rationality of the Western political discourse: the exception argument, the balancing metaphor (between security and liberty) and the trade- off model.
These conceptual continua are the ideological backbone of the societal perception of CT effectiveness. This (societal) approach of CT effectiveness is often analyzed through an output-outcome-impact trichotomy, an epistemological framework originally developed by Young (2001).
Generally speaking, output effectiveness would refer to the adoption of a measure, policy instrument, or regulation in response to a threat; in this instance, effectiveness is assessed in relation to the behavior of the policy makers charged with the implementation.
Outcome effectiveness, contrarily, would examine the direct and measurable effects that a given policy framework has in real life; as in, it would also take into account the behavior of the targeted group (in addition to the policy-makers) in relation to the short-term objectives of the specific CT policy implemented, such as disrupting the financial flows or the recruitment process of a terrorist organization.
Finally, impact-effectiveness depends on the behavior/reaction of the targeted audience alone, with regard to the long term objective of the CT policy, namely its success or failure in reducing or stopping terrorism.
It is interesting to note that the effectiveness denominator is not something that most legal analyses tend to focus on, in general; they rather focus on the necessity, legality and proportionality of CT measures. Vermeulen, Deering and McCarthy argue that effectiveness is predominantly evaluated in a factual sense and is assessed by reference to the intended results a CT measure is supposed to produce, (e.g. “expedition of the prosecution of suspected terrorists or the freezing of suspected terrorists’ assets”).
The legal perspective may not directly make out a prima facie case of effectiveness but it does consider other variables such as whether a CT measure is compliant with democratic values and principles and/or whether it is “strictly required by the exigencies of the situation”; thus implicitly taking effectiveness into account as well.
Measuring Operational Effectiveness (from a Law-Enforcement Perspective)
A wide spectrum of different methods has been put to use in the study of CT operational effectiveness, with qualitative methods and impact effectiveness being the most prevalent epistemological approaches. This “clinical” analysis is somewhat understandable, since the ultimate objective of any CT measure or policy is to actually have an effect on terrorism and numbers tend to offer clearer policy-relevant indicators.
However, measuring operational effectiveness practically can be a frustratingly tedious task, given the large number of quantitative studies in the field and the difficulty to clearly establish the reliability and validity of the relevant indicators.
One basic factor that influences the overall validity and reliability of indicators and complicates the structured and systematic consideration of CT measures, is the fact that most studies focus only on individual measures -i.e. the increase in law enforcement powers or legislation, criminalizing acts as terrorism, racial profiling or data mining, or on a preferential selection thereof i.e. disrupting terrorist financing, roving patrols, security/defensive barriers, infiltration of terrorist organizations; arrests, sweeps/searches, confiscation of goods/funds, entering enemy territory, detaining or questioning, foiling attacks, sting operations, hostile actions (strikes/killings).
Along similar lines, Hewitt proposes a classification scheme albeit with largely unspecified criteria; from a law-enforcement stand point, he looks at anti-terrorist legislation, emergency powers, ceasefire and negotiations and the use of security forces. Occasionally, there is no specific mention of concrete counter terrorism measures; they are rather evaluated as a whole in cost-benefit assessments.
—
From a law enforcement perspective, operational effectiveness is -quite often- approached in the literature through the outcome-effectiveness basis. The study of outcome-effectiveness, in this sense, can be quite problematic due to the lack of systematization, convenient selection of indicators and a certain haziness as to whether certain elements should be considered indicators or side effects.
Most authors tend to systematize indicators through a dichotomous categorization; the root causes of terrorism and the operational ability of the targeted organization (resources, public support, recruitment, etc.).
Most studies fall into the second category and take one or more operational aspects into consideration. Della Porta, for instance, argues that “the number of terrorist…arrests (can be taken) as an indicator of state success” and therefore its effectiveness.
Byman focuses on the level of domestic support for counter terrorism operations, operational freedom and the disruption of the adversary’s command and control structure, while Perl suggests that operational effectiveness should be measured by reference to the degradation of terrorist capabilities and infrastructure.
Relatedly, Malvesti provides a comprehensive set of indicators and notes that one needs to take into account the “critical nodes” in a terrorist infrastructure, such as: the political base, financial networks, communication channels, sanctuary, intelligence network, weapons, cells and leadership.
Spencer proposes a more holistic approach, stating that the operational effectiveness of counter terrorism measures may also be purposively understood; if the implemented law enforcement measures manage to reduce the popular fear of terrorism, they can be considered effective.
In their report regarding the operational perspective, de Londras, Downing and Doody stress that operational effectiveness is directly proportional to “the satisfaction at operational level(s) of the original objectives that were to be achieved by the introduction of a CT policy or measure.
Nevertheless, they stress that a clear rationale for the policy and/or measure in question needs to be accurately defined along with an adequate monitoring mechanism so that effectiveness –for all intents and purposes- can actually be assessed.
It is worth noting that certain studies place a very high premium on impact effectiveness as well, with respect to the operational effectiveness of relevant (law enforcement) CT measures. Impact effectiveness is usually evaluated in relation to indicators associated with terrorist activity, i.e. a. number of attacks, recidivism rates for deradicalisation programmes of individuals and the degree to which group radicalization programmes are capable of preventing further terrorist activity.
In this context, Horgan and Braddock propose a more comprehensive assessment scheme, the Multi Attribute Utility Technology, consisting of two sets of indicators: a. the rate of terrorism, subordinated domestic terror rates and recidivism and b. the popular support for the government, boosted morale and political capital.
Noricks, however, notes a major difficulty within this category of effectiveness: the limited amount and reliability of data, such as the accurate reporting and tracking of re-arrests in the case of recidivism. Numerous scholars tend to use instead of impact, outcome indicators i.e. the number of killed/arrested terrorists and the destruction of the terrorist infrastructure.
It should be mentioned, nevertheless, that this particular indicator could lead to faulty assumptions, since operatives are easily replaceable, thus the arithmetic analogy does not necessarily reveal the potential or the size of a terrorist organization; therefore any assessments regarding the impact, dynamics or future attacks would -at best- be probabilistic.
Finally, a comparatively smaller body of literature uses output effectiveness for the evaluation of counter terrorism measures. This method is relatively unproblematic because its indicators are easily delineated. These include the production of strategies, organization reform and legislation, the establishment of authorities or the implementation of legislation.
De Londras observes that the introduction (and subsequently the effectiveness evaluation) of counter terrorism policies -particularly in the European Union- is generally structured around ex ante impact assessments, which basically can only ascertain predicted impacts and thus estimate the proportionality, effectiveness and appropriateness of the proposed technology or measure.
So, in essence, ex ante impact assessments are purely speculative and they cannot determine effectiveness as a matter of fact. She proposes instead an “ex post facto” review, (currently rare in the context of counter terrorism), of how the measure or technology is actually working, how it is widely perceived, what its real impact is, how much it is costing and so on, so that effectiveness can be assessed in the broad and vital sense.
In sum, academics have used a wide variety of methods to assess the overall effectiveness of counter terrorism responses and have reached very different conclusions regarding how effective counter terrorism policies actually are.
Conclusion
While effectiveness could be regarded as one of the most essential aspects of any counter terrorism policy, its study is cumbered by both epistemological underdevelopment and a lack of a canonical scientific basis. Conceptually, there is a wide variety of indicators/measures/techniques taken into consideration, and in numerous categorizations, from comprehensive strategies to a set of measures and/or individual ones.
The incremental methodological complexity is directly analogous to a series of validity and reliability issues, which explain the –partly- divergent results obtained. Another major hindrance is the data availability and quality as well as the lack of empirical data, the limited nature and/or manipulability of statistics and the impossibility of capturing some disruptive successes in statistical form.
The limitations of a statistical and/or qualitative evaluation should also be born in mind, as statistics are not necessarily objective and/or exhaustive indicators of effectiveness. In designing mechanisms for evaluating effectiveness, qualitative empirical research should be conducted, in order to identify potential conditions that could be operationalized in assessment variables.
Furthermore, policy makers must bear in mind that much of counter terrorism is -in effect- mutative in nature (shifting threats, priorities, geopolitical realities) and effectiveness itself may change over time, depending on the particular circumstances; hence, the effectiveness of CT measures requires constant re-evaluation, occasionally beyond a statistical and/or quantitative basis.
