Evaluation of interventions to prevent intimate partner violence
Evaluation of interventions to prevent intimate partner violence
Abstract and Keywords
Intimate partner violence (IPV) is a global problem, causing injury and mortality, as well a range of physical and mental health problems, including HIV infection, post-traumatic stress disorder, depression, substance abuse, and miscarriage. This chapter evaluates interventions to reduce men's use of violence and shows how mixed-method approaches are necessary to understand the full impact of the intervention on gender-based violence.
Intimate partner violence (IPV) is a global problem, causing injury and mortality, as well a range of physical and mental health problems, including HIV infection, post-traumatic stress disorder, depression, substance abuse, and miscarriage (Campbell 2002). Its roots lie in the dominant gender order of societies and changing this requires intervention at all levels: societal, community, family, and individual. Prevention is increasingly seen as an important public health activity, but most research has been on victim responses; that is, secondary prevention. Primary prevention has chiefly focused on individual- and community-level change. Relatively few interventions have been developed and evaluated, which clearly points to the tremendous importance and challenges of stimulating research in this area.
Types of intervention
Among the primary prevention interventions that have been evaluated are school-based interventions on violence (the best known of which is Safe Dates; Foshee et al. 2004); interventions that focus on men and boys and building gender equity (Barker et al. 2007); mass media activities that seek to raise awareness and change attitudes to violence and risk-factor-reduction activities such as reducing problem drinking or promoting anger management or non-violent conflict resolution. Recently evaluated interventions in developing countries have included a programme for women which combined a small loans scheme (microfinance) with a 10-session group intervention in (p.122) violence and promotion of community action (Pronyk et al. 2006) (see Box 9.1) and a participatory intervention known as Stepping Stones used with men and women (Jewkes et al. 2008) (see Box 9.2).
A secondary (and tertiary) prevention intervention that has been widely used and tested is often known as ‘screening’. It involves case identification through sensitive questioning, provision of a simple message about the non-acceptability of violence, and referral of cases for further support (Garcia Moreno 2002). Other secondary prevention interventions include legislative measures which criminalize gender-based violence and provide for protection orders and compulsory treatment of offenders; shelters for women; self-help groups; counselling and treatment for men who abuse women; and peer-education approaches which change community norms on the use of violence and gender relations.
These represent a wide range of interventions and each one has its own challenges in evaluation. The nature of many interventions makes the research complex. Determining and measuring appropriate outcomes, and attributing effects, are considerable challenges. Transposing interventions developed and tested in one setting to another is often seen as an efficient way to develop knowledge and a means to circumvent some of these problems. Even if an evaluation of an intervention appears to have strong external validity (see Chapter 4 for an explanation of external validity), the intervention should still be tested in each new setting to establish relevance and effect.
Although posited as the gold standard, randomized controlled trials (RCTs) should be considered a last step in a pathway, as in drug development (Phase III trials), rather than a first-line methodology. This statement does not undermine their potential to make a valuable contribution to knowledge, particularly attribution of effect, but rather, carefully positions it. RCTs that test the achievement of hard outcomes (e.g. violence reduction) and look for this to be sustained in the long term often have considerable economic and opportunity costs for research teams. Their real place is to test interventions that have been shown through other forms of evaluation to be promising (see Chapter 4). The evaluatory process needs to start well before the RCT is conceived, in the development and initial testing of an intervention.
Research into violence interventions needs to start from the first stage of development or when testing a promising intervention for a new setting. Here formative research is crucial and qualitative methods may be particularly useful. With new interventions, initial planning stages involve developing intervention goals (following appropriate theoretical frameworks of behaviour change), intervention approaches and mapping the content accordingly (see Chapter 3 for a discussion on incremental approaches to evaluation). These require a detailed understanding of the dynamics of violence in the target population.
A second stage involves the initial testing of the intervention with a limited subset of the target population. The focus at this point is not on behaviour or attitude change, but on process evaluation; that is, the more limited goals of testing feasibility of the intervention in the time allocated, coherence, acceptability, and the perceptions of the intervention by the target group. Feasibility and coherence are generally assessed by those delivering the intervention, and much can be learnt about an intervention by asking men or women experiencing it how they perceived it and what impact it had on them. This information can also be used later to shape a quantitative evaluation (see the chapters in Part IV of this book, which discuss involving users in evaluation).
Special challenges may arise in research and interventions addressing IPV. One challenge is the established gender order that position men as socially superior to women, and provides legitimacy for the use of violence to assert dominance and punish women. This is deeply embedded in the social practices of a society and receives legitimization from both men and women. It is very difficult to implement an intervention which seeks to build gender equity in relationships, or to respond sensitively to victims, if the implementers are unconvinced of the appropriateness of this or if they believe, for example, that women are beaten because they deserve it.
Interventions should be tested for efficacy (internal validity) in ‘ideal’ conditions before they are subject to wider roll out and evaluation in other settings. Part of the process of establishing ideal conditions should include giving special attention to the selection of implementers, including their ideas about gender relations. The programme for training implementers (whether health professionals or intervention facilitators) needs to expound in some detail the context and nature of the problem, and to challenge gender attitudes, as well as exploring and working through personal experiences of violence. This requires time. Some of the notoriously unsuccessful IPV interventions have completely neglected these aspects of the problem, imagining it stripped of its emotive elements and rendered a mere medical risk factor or injury cause. For an example of how these issues may be explored in a training intervention see Box 9.3.
Determining realistic impact and outcomes
Expectations of what impact can be achieved by an intervention should be realistic and commensurate with the nature of the intervention. Interventions that seek to change the general climate of gender relations by challenging norms and attitudes, such as mass-media interventions, can realistically be expected to do this in the exposed population, but to expect a limited-duration, general intervention to impact in a measurable way on behaviour may be unrealistic. Interventions that focus on participants in groups for long periods of (p.126) time (such as Stepping Stones; Jewkes et al. 2008) can certainly be expected to change behaviour in a measurable way in a well-designed study.
With some IPV interventions, determining what may be a realistic outcome is less straightforward. Listening to the women or men who experience the intervention as well as the people who implement is valuable. Screening interventions usually try to: validate the non-acceptability of violence; enable access to support for abused women; and promote safety planning and trigger other help-seeking practices. If sustained and successful they may change community norms, reduce violence and improve women’s health and wellbeing. These outcomes will only be visible in the longer term. Evaluation should initially concentrate on assessing the extent to which the more limited (p.127) proximal goals are achieved, as it is unlikely that longer-term changes will occur without substantial achievement of short-term goals (see Chapter 3 for a discussion on proximal and distal outcomes).
A first stage, proof of concept, evaluation is very useful. This includes a questionnaire which measures a range of possible intervention impacts. Ideally this stage has two study arms, random allocation is not necessary, and a formal sample-size calculation is not done. The focus is on assessing what changes (acknowledging that the design does not lead to any conclusive attribution of impact), how much it changes by (necessary for sample-size calculations for a trial), and showing initial promise that can justify further evaluation. Follow-up is short and thus there is a risk of rejecting interventions for not showing effect at this stage. Qualitative research undertaken with participants going through the intervention can shed considerable light on the processes of change (see Chapter 7 on process evaluation and Chapter 14 on feedback to participants). Only after successful proof-of-concept evaluation is an intervention well placed to be evaluated with an RCT of appropriate sample size and duration of follow-up.
Measuring experiences of intimate partner violence as victims and perpetrators
Quantitative evaluation of IPV is challenging, but has an important role in rigorous assessment of interventions alongside the use of qualitative methods. It is important to consider the range of potential abuse. Apart from physical violence, IPV also includes sexual and psychological abuse and controlling behaviours. Although IPV is often emphasized, controlling behaviours of intimate partners have been shown to be equally important in influencing certain health risks, notably HIV (Dunkle et al. 2004a). Initial work developing measurement instruments was done with a view to interviewing women as victims, but more recent research with men on perpetration has begun to show that the same measurement principles pertain.
Internationally, work to develop best practice in violence research (Ellsberg and Heise 2002) and to create instruments that measure (p.128) experiences of physical and sexual IPV in a way which is both locally valid and allows for comparison across settings has been undertaken. The World Health Organization has developed an instrument for its multi-country study on women’s health and gender-based violence, which has been tested in over a dozen countries (WHO Multi-Country Study Core Team 2000). They recommend that questions on physical and sexual abuse focus on discrete acts of violence rather than using broad and potentially charged, subjectively interpreted words. So, for example, a question should be asked about ‘kicking’ or ‘use of a weapon’ rather than a general one about ‘physical violence or abuse’.
The effects of an act of physical violence lingers after the injuries have healed and often women describe living in a pervasive atmosphere of fear in between the violent acts. There have been some attempts to develop measures of women’s subjective experiences of violence and control in relationships that extend beyond the measurement of violent acts. Two notable examples are the Sexual Relationship Power Scale (Pulerwitz et al. 2000) and the WEB Scale (Smith et al. 1995). Landenburger (1998) described a cycle of episodes of violence followed by periods of remorse accompanied by affectionate behaviour, but then followed by tension-building phases before violence is repeated. Most researchers agree that these elements are found in many abusive relationships, although the cycle is rather stylized. This poses a challenge for people who want to evaluate the effectiveness of interventions. Positive perceptions of change in a relationship may merely be a product of data capture during the partner’s remorseful (honeymoon) phase. Better practice in evaluation of IPV would include measures of the frequency of discrete violent acts and a measure of women’s subjective experiences.
Problems of recall
The period over which physically or sexually violent acts are measured is critical. If the period of recall is too long, events are forgotten and so there is greater inaccuracy; if too short it may lead to exaggeration of the frequency because episodes occurring outside the recall period are erroneously included. Frequency of abuse is problematic as an outcome measure because few women experience physical violence on a daily or even weekly basis. For many women it is much less common. (p.129) Research in South Africa suggests that approximately a third of women who had ever experienced physical or sexual violence from an intimate partner had only had one episode and that a similar proportion of those experiencing it in the past year have only experienced it once (Dunkle et al. 2004b).
Researchers recognize the limitations of reports of violence as an outcome because they are subjective. It is very difficult to validate them, and research has shown that memory of violence is not terribly reliable as it is influenced by the affective state of the relationship. Thus women in abusive relationships who feel their relationship is tolerable and decide to stay may recall violent episodes differently to those who do not (Eisikovits and Winstok 2002). In response to this some have suggested that interviews should be undertaken with couple validation. This is problematic for both ethical and practical reasons. One concern is for women’s safety, it could expose women to further abuse from their partners if men perceived that they were losing face due to disclosure of violence in the relationship. Secondly, research with couples shows that recall of violent acts differs between them and is critically related to meaning associated with the act, both at the time it occurred and the time of the interview, with both abusive men and abused women minimizing or ‘forgetting’ violence at times (Armstrong et al. 2001). Thus reports of IPV experiences are inherently imprecise. Taking efforts to minimize bias caused by this is important, for example by using standard assessment methodologies and random allocation between study arms in an RCT. However, the inability to blind participants to which intervention arm they are in may still lead to bias in reporting, especially in studies of interventions that are primarily IPV prevention interventions, rather than interventions to prevent IPV but with other goals such as promotion of safer sex.
Psychological abuse is an important dimension of IPV but is even more difficult to measure. Enumerating all acts of psychological abuse is probably impossible. It is not surprising that there is no substantial body of international opinion on how to define or measure psychological abuse. It is generally agreed that shouting, belittling, verbal abuse, and threats of violence are common manifestations, but in some countries other forms of abuse are also common. These may include taking a partner’s earnings, evicting her from the home, stalking her, (p.130) bringing home or boasting about other girlfriends, undermining self-esteem, failing to contribute to maintenance or the household, dictating what she wears, or trying to control her behaviour and movements. Moreover, measuring discrete acts of psychological abuse may fail to capture the pervasive atmosphere created by such acts.
Some IPV interventions carry a distinct possibility that the intervention may provoke further violence. Although some women find that abuse stops when they have the strength to stand up to their partner, others find that such action escalates abuse. This needs to be taken into consideration in intervention design and the evaluation. Many IPV interventions have been shown to have differing short- and long-term impacts (see Box 9.4).
Attributing change to interventions
Research on IPV can be an intervention in itself. The interview process and accompanying messages about the non-acceptability of violence, as well as the provision of referral information that must be (p.131) given to meet ethical standards (Ellsberg and Heise 2002), constitutes an intervention. Similarly, research asking men about rape perpetration has been shown to generate self-questioning of their violent behaviour (Sikweyiya et al. 2007). This is a particular problem in an evaluation of screening by health care workers, as these interventions often require them to ask just two or three screening questions and yet the assessment questionnaire may include 10 or more, which could significantly influence reflections on abuse experiences in the non-intervention study arm. At the very least, this might reduce the intervention effect size and so sample sizes need to be adjusted accordingly.
Problems of attributing a cause to an observed behaviour change pertain to interventions seeking to reduce IPV. Most women in violent relationships are not able to simply leave or stop the violence or they would probably have done so already. The process of taking action may be started by that initial contact with an intervention, a chance to talk with a doctor or nurse, or seeing a billboard message that women do not need to tolerate such behaviours. The impact of these, if seen at all, should be expected to evolve along a slow, spiralling, and convoluted pathway from there and will include exposure to multiple interventions. Experience of non-governmental organizations that help abused women is that clients often want to test the water when they make their first contact, talk about problems, and get reassurance about themselves and only later try forms of interventions such as leaving. Even then, they often go back to their partner multiple times. Some will try to get family members to intervene or they may get a temporary protection order several times, never going on to making it permanent, or they may engage with other legal and social interventions. If women leave a violent relationship, the process often takes several years. Many women do not ever choose to leave, what they seek is for the violence to stop or lessen and again over time this may happen.
A further problem stems from the fact that most interventions focus on the current relationship and yet it is known that women who enter new relationships after experiencing violence often have serially abusive relationships. Ideally evaluations should have long enough follow-up to capture this but in practice this is rarely made possible due to the limitation of funds and resources needed for such a long follow-up.
(p.132) In summary, research with IPV interventions must start at a formative stage and then ensure that staff have been appropriately selected and trained to implement the intervention. Only when this has been achieved can intervention fidelity be assessed and the participants’ perceptions of the intervention be assessed. Qualitative methods are most appropriate at this stage and good evaluations will listen carefully to the participants’ experiences. This process often takes one or more years, depending on the nature of the intervention, and may be iterative with false starts, dead-ends, and so forth. This must be adequately resourced if interventions are to be well developed before they are tested. When interventions have been shown to be robust they need to be tested in proof of concept studies to demonstrate promise. In most cases, it is only after proof-of-concept studies that RCTs should be contemplated, although the greater rigor and longer follow-up of an RCT may render visible effects which are otherwise not seen (see also Craig et al. 2008 on developing and evaluating complex interventions).
Evaluations must explore multiple forms of IPV and consider both subjective experiences as well as objective measures of frequency and severity. Outcomes are likely to be influenced by exposure to multiple interventions or to important life events. All this suggests the need for long-term experimental community-based research in live settings (see Chapter 10 for more information on this) to enable a better understanding of the natural history of gender-based violence and the impact of different interventions to be developed. An ultimate goal for evaluation would be to demonstrate whether concerted efforts to intervene can result in improvements in women’s mental and physical health and a reduction in IPV.
Armstrong, T.G., Heideman, G., Corcoran, K.J. et al. (2001) Disagreement about the occurrence of male-to-female intimate partner violence: a qualitative study. Family & Community Health 24, 55–75.
Barker, G., Ricardo, C., and Nascimento, M. (2007) Engaging Men and Boys to Transform Gender-Based Health Inequities: Is There Evidence of Impact? World Health Organization and Institute Promundo, Geneva.
Bott, S., Guedes, A., and Guezmes, A. (2005) The health service response to sexual violence: lessons from IPPF/WHR member associations in Latin America. In Jejeebhoy, S., Shah, I., and Thapa, S., eds, Sex Without Consent: Young People in Developing Countries, pp. 251–68. Zed Press, London.
Campbell, J.C. (2002) Health consequences of intimate partner violence. Lancet 359, 1331–6.
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., and Petticrew, M. (2008) Developing and evaluating complex interventions: the new Medical Research Council guidance. Medical Research Council Guidance. British Medical Journal 337, a1655.
Dunkle, K.L., Jewkes, R.K., Brown, H.C., Gray, G.E., McIntryre, J.A. and Harlow, S.D. (2004a) Gender-based violence, relationship power and risk of prevalent HIV infection among women attending antenatal clinics in Soweto, South Africa. Lancet 363, 1415–21.
Dunkle, K.L., Jewkes, R.K., Brown, H.C. et al. (2004b) Prevalence and patterns of gender-based violence and revictimization among women attending antenatal clinics in Soweto, South Africa. American Journal of Epidemiology 160, 230–9.
Eisikovits, Z. and Winstok, Z. (2002) Reconstructing intimate violence: the structure and content of recollections of violent events. Qualitative Health Research 12, 685–99.
Ellsberg, M. and Heise, L. (2002) Bearing witness: ethics of domestic violence research. Lancet 359, 1599–1604.
Foshee, V.A., Bauman, K.E., Ennett, S.T., Linder, G.F., Benefield, T., and Suchindran, C. (2004) Assessing the long term effects of the Safe Dates program and a booster in preventing and reducing adolescent dating violence victimisation and perpetration. American Journal of Public Health 94, 619–24.
Holt, V.L., Kernic, M.A., Lumley, T., Wolf, M.E., and Rivara, P. (2002) Civil protection orders and risk of subsequent police-reported violence. Journal of the American Medical Association 288, 585–94.
Jewkes, R., Nduna, M., Levin, J. et al. (2008) Impact of Stepping Stones on HIV, HSV-2 and sexual behaviour in rural South Africa: cluster randomised controlled trial. British Medical Journal 337, a506.
Landenburger, K.M. (1998) The dynamics of leaving and recovering from an abusive relationship. Journal of Obstetric, Gynaecologic and Neonatal Nursing 27, 700–6.
Pronyk, P., Hargreaves, J.R., Kim, J.C. et al. (2006) Effect of a structural intervention for the prevention of intimate partner violence and HIV in rural South Africa: a cluster randomised trial. Lancet 368, 1973–83.
Pulerwitz, J., Gortmaker, S., and De Jong W. (2000) Measuring sexual relationship power in HIV/STD research. Sex Roles 42, 637–60.
Sikweyiya, Y., Jewkes, R., and Morrell, R. (2007) Talking about rape: men’s responses to questions about rape in a research environment in South Africa. Agenda 74, 48–57.
Smith, P.H., Earp, J.A., and DeVellis, R. (1995) Measuring battering: Development of the Women’s Experience with Battering (WEB) scale. Women’s Health: Research on Gender, Behavior, and Policy 1(4), 273–88.
WHO Multi-Country Study Core Team (2000) WHO Multi-Country Study On Women’s Health And Life Events Questionnaire Version 9.9. World Health Organization, Geneva.