The effectiveness and cost-effectiveness of behavioural interventions for the prevention of sexually transmitted infections in young people aged 13-19: a systematic review and economic evaluation

Epidemiology and natural history of STIs

Rates of STIs continue to increase in the UK, particularly among young people. Increases in diagnoses may reflect greater ascertainment of cases through more testing and better diagnostic methods. They may also reflect an increase in unsafe sexual behaviour among young people. Most STIs are caused by either bacteria (e.g. chlamydia, gonorrhoea) or viruses [e.g. HIV, genital herpes, human papillomavirus (HPV)].

The impact of increases in the incidence and prevalence of STIs over recent years has placed great demand on health service resources for screening, treatment, and prevention of infections and their complications. Data from the British National Surveys of Sexual Attitudes and Lifestyles (NATSAL) show an increase in the number of individuals attending genitourinary medicine (GUM) services over a 10-year period. 4 Between 1990 and 2000 there was an increase of 4.3–7.6% in men and 3.3–6.6% among women. This may be explained not only by increases in rates of STIs, but also by improvements in access to clinic services, and increased asymptomatic screening. STIs can lead to a range of serious long-term complications, such as infertility, pelvic inflammatory disease (PID) (in women), and epididymitis (in men), particularly if undiagnosed and untreated. HIV, though treatable, currently remains incurable, and, if not successfully managed, is associated with faster disease progression to acquired immune deficiency syndrome (AIDS) and related complications, and to mortality.

More detailed descriptions of the characteristics of individual STIs, including their causes, symptoms, treatment, incidence, prevalence and complications, are provided in the following subsections.

Complications of STIs

Sexually transmitted infection prevention in the UK

Sexually transmitted infection prevention for young people in the UK is provided by a number of agencies, including schools and colleges, the NHS, the youth service and non-statutory organisations. Sometimes prevention interventions are undertaken through multi-agency collaboration, such as school health clinics that are run by health professionals. [The subject of a separate systematic review and economic evaluation funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme – see www.hta.ac.uk.]

Searching for primary evaluations

Sensitive search strategies were developed and tested by an experienced information scientist (see Appendix 2 for search strategies). The finalised strategies were applied to the following electronic bibliographic databases:

• MEDLINE (via Ovid)

• MEDLINE In-Process & Other Non-Indexed Citations (via Ovid)

• EMBASE (via Ovid)

• PsycINFO (via Ovid)

• Educational Resources Information Centre (ERIC) (via CSA)

• CINAHL (via Ovid)

• Cochrane Central Register of Controlled Trials (CCRCT)

• Trials Register of Promoting Health Interventions [TRoPHI – the Evidence for Informed Policy and Practice Information and Co-ordination Centre (EPPI-Centre) register of RCTs and non-randomised controlled trials]

• Applied Social Sciences Index and Abstracts (ASSIA) (via CSA)

• Sociological Abstracts (SOCABS) (via CSA)

• POPLINE (POPulation information online).

ASSIA was searched from 1987 to March 2008. All other databases were searched for the period 1985 to March 2008. It was considered that evaluations of behavioural interventions to prevent STIs, notably HIV, would have started to be published from the mid-1980s onwards, hence the choice of 1985 as the start of the search period.

The following websites were also searched to identify relevant studies:

• The UK National Library for Health (NLH) – www.library.nhs.uk/Default.aspx

• UNAIDS (Joint United Nations Programme on HIV/AIDS) – www.unaids.org/en/

• CDC (Centers for Disease Control and Prevention) – www.cdc.gov/

• Google Scholar – https://scholar.google.co.uk

Google Scholar was searched from 1989 to 2007; UNAIDS and CDC were searched for the period 1998 to 2007. The NLH database does not permit restriction of searches to particular dates.

In addition, relevant articles were sourced from bibliographies of the studies included in the systematic review. For all searches, studies were limited to English-language articles. Full papers were obtained from University libraries, the British Library, and from the internet (e.g. Google). Search results were uploaded into the eppi-reviewer database28 (specialist systematic reviewing software developed by the EPPI-Centre), de-duplicated and stored, ready for screening.

Searching for systematic reviews

A separate search was conducted to identify other systematic reviews of behavioural interventions to prevent STIs. The aim was to scan the reference lists of relevant systematic reviews to identify studies, supplemental to those identified by our own searches, which might meet the inclusion criteria of our review. The systematic reviews themselves were not subjected to data extraction and critical appraisal and their results were not assessed. However, some notable examples of these systematic reviews are discussed in relation to the findings of our review (see Chapter 8, Findings from other systematic reviews). The search for systematic reviews was based on those conducted for the Evidence briefings on STIs and HIV published by the (former) Health Development Agency (now the Centre for Public Health Excellence at NICE). 29,30 As these briefings searched up to 2003, we carried out further database searches for the period 2003 to March 2008 to identify more recent systematic reviews. The databases searched included: Cochrane Database of Systematic Reviews (CDSR); Centre for Reviews and Dissemination (CRD) (University of York) databases: DARE (Database of Abstracts of Reviews of Effectiveness) and HTA (Health Technology Assessment); MEDLINE (Ovid); MEDLINE In-Process & Other Non-Indexed Citations (Ovid); EMBASE (Ovid); CINAHL (Ovid); British Nursing Index; PsycINFO; ERIC (CSA); and SOCABS (CSA). The search identified 643 systematic reviews, of which 21 met the inclusion criteria [i.e. the review focused on behavioural interventions targeted at young people aged 13–19 years, and could be considered systematic (in that they provided evidence of: search strategy; application of inclusion criteria; and use of quality assessment)]. The 21 reviews yielded 76 references to studies that had not been identified from our search of bibliographic databases. These were screened for inclusion, along with all of the other references that had already identified from the databases.

Participants

The participants were young people aged 13–19 years.

If the study included a broader age range (e.g. 17–22 years), but the mean or median age was within the specified age range, then the study was included.

Study design

Outcome evaluations (RCTs or non-randomised controlled trials) were used.

These were defined as studies designed to establish whether the intervention changes the outcomes specified in the aims of the study. Only trials that compared the intervention with a control or comparison group were included. Outcome evaluations that included an integral cost-effectiveness analysis and/or process evaluation were also included. Studies reported in abstract form only (e.g. conference proceedings) were only included if they were published in or after 2005. It was assumed that, in general, abstracts from before this date would have been fully published and hence identified by our search.

Intervention

Behavioural interventions that aim to prevent STIs were used.

The commissioning brief did not define what a behavioural intervention was, so we adopted an inclusive strategy that defined it as ‘any activity to encourage young people to adopt sexual behaviours that will protect them from acquiring STIs’. Sex education studies were included provided there was some indication that prevention of STIs was addressed by the intervention.

Comparator

The commissioning brief specified ‘standard practice’ as the comparator. We therefore did not restrict inclusion of studies on the basis of any particular comparator. (See Chapter 4, Comparators, for a description of the various comparators used in the studies which were systematically reviewed.)

Outcomes

Studies that reported the impact of the intervention on a sexual behavioural outcome were included, for example:

• abstinence from sexual activity/delaying onset of sexual activity

• self-reported condom use (e.g. frequency of use)

• number of sexual partners

• age at first experience of sexual intercourse.

Studies that reported incidence/prevalence of STIs or pregnancy-related outcomes (e.g. rate of conceptions) were included, providing they also reported a sexual behaviour outcome. Since self-efficacy does not constitute a behaviour, studies that reported self-efficacy or sexual behavioural intentions were included only if they also reported a sexual behaviour outcome. Likewise, studies measuring variables considered to mediate behaviour change (e.g. knowledge, attitudes, beliefs, intentions) were included provided that a behavioural outcome was also measured.

Meta-analysis

Meta-analysis was conducted using eppi-reviewer. A fixed-effect model was used, with a random-effects model reserved to explore any significant statistical heterogeneity observed. Statistical heterogeneity was assessed using the chi-squared test, with a p-value greater than 0.10 indicating significant heterogeneity. 48 The I²-statistic was used to quantify the magnitude of statistical heterogeneity. This describes the percentage of the variability in effect estimates that is due to heterogeneity rather than sampling error (chance).

As will be reported in more detail in Chapter 4 (see Methodological quality of the outcome evaluations), 9 of the 12 methodologically sound outcome evaluations randomly allocated clusters rather than individuals to study groups. An estimate of the intracluster correlation coefficient (ICC) was therefore required. This is necessary to account for the tendency for individuals within a cluster to be similar, thus reducing the variability in the responses. 49 The ICC is expressed as a value of between 0 and 1. The closer the figure is to 1 the greater the within-cluster correlation. 49 Five of the nine cluster RCTs reported an ICC, and these were entered into the meta-analysis. For studies that did not report an ICC we imputed a value of 0.2, based on that used for the power calculation in one of the trials included in the systematic review [the RIPPLE (randomised intervention of pupil peer-led sex education) trial]. 50 This value was higher than ICC values reported by the other included studies. Therefore, the effects of imputing lower ICC values, and of omitting an ICC value when none was reported, were tested. It was found that they did not alter the pooled effect estimates (data not reported).

The mean cluster size was calculated by adding the total number of young people randomised to each study group and dividing this total by the number of clusters. The majority of outcome measures for which sufficient data were available were dichotomous. Odds ratios (ORs) were chosen as the summary effect estimate for these outcomes.

Behavioural outcomes were prioritised for meta-analysis because effect estimates of parameters such as condom use were required to inform the Bernoulli equation in our economic model (see Chapter 7, Methods for cost-effectiveness modelling of behavioural interventions for prevention of STIs in young people). The outcome measures reported by each of the RCTs judged methodologically sound in the systematic review were tabulated to determine whether sufficient data existed to permit meta-analysis. This was an intricate process, as there was much variability in the definition and reporting of behavioural outcomes, such as condom use. Once tabulated, specific behavioural outcomes were defined and data assembled into a format suitable for entry into the meta-analysis facility in eppi-reviewer. If the primary studies assessed outcomes at several time points (e.g. 6 months’ follow-up, 12 months’ follow-up), the data were entered for all of the reported follow-up time points.

Where possible, the outcome data were entered into the analysis based on an intention-to-intervene (ITI) analysis (i.e. based on all young people randomised). For some behavioural outcomes, data were reported for subgroups of young people who were sexually active/inactive. Where necessary, study authors were contacted for clarification on figures reported in their papers or to supply missing data. However, few of the authors responded, which meant that certain studies could not be included in the meta-analysis for some outcomes. It was decided that the study by Borgia and colleagues51 would not be included in the meta-analysis as it compared two very similar behavioural interventions (the only difference between them being that one was teacher-led, while the other was peer-led), whereas the other studies generally compared a behavioural intervention with either a control or with standard practice. (Note: The trial by Stephenson and colleagues,50 which also compared peer-led with teacher-led interventions, was included in the meta-analysis as the teacher-led intervention was standard practice – see Chapter 4, Synthesis of results of sound outcome evaluations.)

Where data allowed, we conducted separate meta-analyses for the subgroups based on gender (young men and young women). This was one of the subgroups specified in the protocol as a variable to be explored in the synthesis of outcome evaluations.

The feasibility of meta-analysis of behavioural mediators, such as knowledge and self-efficacy, was explored. Much of these data were measured using widely differing measurement scales, and in many cases no measure of variance was reported to allow meta-analysis of continuous data. Meta-analysis of these outcomes was therefore not considered feasible or appropriate.

Process evaluations

Narrative methods were used to synthesise findings from the process evaluations. These methods were based on those we have developed in previous work on the synthesis of process evaluations and qualitative research44,52–54 and from other groups working on methods for synthesising process evaluations. 55–57 Detailed evidence tables were prepared, describing the methodological quality of the process evaluation, contextual details (e.g. intervention content, setting) and the findings of process evaluation (Appendix 6). This was undertaken by one researcher (EBP), with contributions from the second reviewer (AH), based on the data extracted using the instrument described in Appendix 5.

Findings were assigned to one or more of eight predetermined categories (accessibility/programme reach; collaboration and partnerships; content of the intervention; intervention implementation; acceptability; quality of intervention materials; skills and training of intervention providers; and ‘other’).

Both researchers independently read and re-read the tabulated details and noted down their initial thoughts on the main themes to arise from the findings. The two researchers met to discuss and compare their individual themes and compiled a jointly agreed list (Figure 1). These themes were used to generate hypotheses about intervention effectiveness and to illuminate issues around intervention acceptability as well as barriers and facilitators to implementation.

FIGURE 1.

Initial and jointly agreed themes.

A narrative was written to describe and elaborate on these themes and this was reviewed and discussed by five members of the review team (AH, EBP, JS, JK and JP) to generate possible explanations for the effects of the outcome evaluations. The themes and notes from this meeting were reviewed again by two researchers (AH and EBP) who translated their initial themes across the process evaluations in order to answer two questions. Firstly, what factors facilitate or hinder the implementation of skills-based behavioural interventions in schools? Secondly, what factors impact on student engagement and intervention acceptability?

Intervention provider

Table 22 presents the number of studies using different types of intervention providers. Just over half of the studies involved teachers as intervention providers and nearly as many involved peer educators. Five were delivered by both teachers and peers (not shown in Table 22). 51,58,63,68,71 Two of the studies, Levy and colleagues65 and Coyle and colleagues,61 which fell into the ‘Other’ category, used specially trained health educators. One study, Jemmott and colleagues,43 used a specially trained ‘adult facilitator’. Stanton and colleagues,69 whose study took place in a number of rural locations, involved local people to deliver the intervention, including ministers, recreation centre directors, liaison officers, faculty members, graduate students and housewives.

Intervention setting

Table 23 presents the number of studies using various settings for the delivery of the intervention. All included studies took place in a school/college setting, in accordance with the inclusion criteria. Two studies, Stanton and colleagues69 and Jemmott and colleagues62 were also partly based in the community. In terms of country, eight interventions were delivered in the USA,43,58,61,62,65,66,69,71 three in Africa (South Africa,63 Tanzania64 and Namibia68), two in Europe (Italy51 and the Netherlands67) and two in the UK (Scotland70 and central southern England50).

Type of intervention component(s)

Table 24 presents the number of studies using various intervention components. As can be seen, the interventions featured more than one component. All provided skills training, in addition to information (a consequence of our inclusion criteria).

All interventions provided education/information and skills training. Fewer studies were classified as providing resources and services. Two interventions included a mass media component: Wight and colleagues70 included videos to trigger discussion, whilst Stanton and colleagues69 used a long-distance interactive televised format to deliver their intervention to some of the participants. Components in the ‘Other’ category included the use of games (Stanton and colleagues,68 Stephenson and colleagues50), a computer-based interactive program (Roberto and colleagues66) and the use of drama (Klepp and colleagues64).

Sexual health topics covered

Table 25 presents the sexual health topics that were addressed by the interventions. Most of the studies covered HIV, and just over half also covered other STIs. None targeted any one particular STI (other than HIV). The studies in the ‘Other’ category covered the prevention of unintended pregnancy, in addition to STIs. 58,61,66,70,71

Content of the interventions

The majority of the interventions encouraged risk reduction. Although not always clear from the descriptions provided, seven studies mentioned risk reduction only and a further five studies aimed at both risk reduction and delayed sexual intercourse. Only one study, Klepp and colleagues,64 focused solely on delayed initiation of sex. Two studies were not clear.

All studies aimed to increase knowledge of HIV and other STIs. Two-thirds of the studies covered negotiation/communication skills, one-third mentioned enhancing self-efficacy, and three studies mentioned attempting to alter peer norms regarding either the desirability of using protection during intercourse or of delaying intercourse. Most studies mentioned the promotion of condoms, and seven studies described actively modelling and practising condom use skills with the young people.

Three studies involved parents as part of the intervention, although the parents were not the official intervention providers. One further study aimed to increase young people’s communication with their parents. Only one study, Wight and colleagues70 mentioned including the promotion of local sexual health services (either by the young people visiting the service or by a member of staff visiting the school), although Coyle and colleagues61 arranged visits to local ‘AIDS service organisations’. None of the interventions featured the distribution of contraception or referrals to sexual health clinics.

Additionally, the two UK-based studies (Wight and colleagues70 and Stephenson and colleagues50) both aimed to improve the quality of young people’s sexual relationships.

None of the interventions were described as having included either referrals to local sexual health services or the provision of contraception, although one (Coyle and colleagues56) required pupils to gather information on local resources and sexual health services. Furthermore, the studies did not tend to mention whether the interventions followed an empowerment or a directive approach to health promotion.

Length/intensity of the intervention

Although some authors provided only minimal details, the length/intensity of the interventions varied widely across studies. The shortest/least intense intervention was the single 5-hour session of Jemmott and colleagues. 26 The lengthiest/most intense intervention appeared to be Wight and colleagues’ intervention,70 consisting of 20 sessions over 2 years. Not so lengthy, but also intense, were the studies of Coyle and colleagues,61 involving 26 hours of intervention over 5–7 weeks, and Stanton and colleagues,68 comprising 14 2-hour sessions over 7 weeks.

Theoretical basis

Table 26 presents the number of studies that used specified theories of health-related behaviour in the conception of the interventions. All except two of the studies were explicitly informed by a named theory. Stephenson and colleagues50 noted that the RIPPLE intervention was not based on any particular theory, but was designed to be a pragmatic approach to sex education. Zimmerman and colleagues71 mentioned that their intervention was theory driven, but did not cite any specific theoretical models. The theories used in the studies were generally sociopsychological in origin. They included Social Learning Theory and Social Cognitive Theory (often associated with the work of Bandura72–74), the Theory of Reasoned Action (associated with the work of Ajzen and Fishbein75), as well as its successor the Theory of Planned Behaviour (associated with Ajzen76). These models predict how health-related behaviour can be influenced in social settings through role modelling to foster positive attitudes, intentions and self-efficacy around negotiating safer sex. Most studies cited more than one theory in the development of the intervention, such as the trial by Karnell and colleagues,63 which included what the authors described as three overlapping theories (Social Learning Theory, Social Inoculation Theory and Cognitive Behaviour Theory).

A range of theories were classified as ‘Other’, including the Social Influences Model,77 Inoculation Theory,78 the Extended Parallel Process Model,79 Protection Motivation Theory,80 the Health Belief Model81 and the Transtheoretical Model (stages of behaviour change). 82 These theories vary in their tenets, but, in common with the theories mentioned above, generally seek to predict health-related behaviour through the role of attitudes, skills, self-efficacy and social influences.

Origin of the interventions

Some of the interventions were derivations of other interventions, some of which were included in our review. For example, ‘Safer Choices’, reported by Coyle and colleagues,58 was based on an intervention, in the USA, entitled ‘Reducing the Risk’ (originally evaluated by Kirby and colleagues,83 but did not meet the inclusion criteria for this systematic review). ‘Reducing the Risk’ was one of the Centers for Disease Control’s ‘programmes that work’84 and was subsequently replicated and modified by Zimmerman and colleagues71 (included in this systematic review) for ‘high sensation seeking and impulsive youth’. Similarly, the ‘Our Times, Our Choices’ intervention evaluated by Karnell and colleagues,63 which was delivered in South Africa, was also a modification of ‘Reducing the Risk’.

‘Safer Choices’, itself, subsequently became one of six effective interventions selected by the National Institutes of Health in the USA for adaptation and replication in different settings. 85 Evaluation of ‘Safer Choices 2’, this time in an ‘alternative’ rather than a mainstream high school setting (e.g. for students with behavioural problems, truancy, poor educational attainment or pregnancy), has been conducted and results are awaited. 86,87

Another of the six replication projects is the West Virginia Focus on Kids (FoK) intervention by Stanton and colleagues69 (included in our systematic review). This is an adaptation of the original FoK intervention, which was originally evaluated with urban, low-income, predominantly African-American young people in Baltimore, MD, USA. 88 (Note: The original intervention is not included in our review as the mean age of the young people, 11 years, did not meet our inclusion criteria.). It was adapted to be relevant to the predominantly rural white population of West Virginia, with low HIV seroprevalence. The original FoK intervention has also been adapted for use outside of the USA – in Namibia. 68 (Note: This study was included in the systematic review, but was judged not to be methodologically sound.)

Comparators

Table 27 presents a classification of the comparators used in the studies. The majority of studies used ‘standard’ sex education as their comparator. This generally consisted of information provision and tended not to include active participation, exercises, role play and skills development. Nearly as many studies used an ‘attention’ control, in which an intervention of similar length and intensity, focusing on a non-sexual health-related topic, was provided.

Age

Studies reported age in a wide variety of ways. Some gave a mean, some a median age, and some a breakdown across the age range of those in their sample (see Table 21, earlier). Some reported for intervention and control groups separately, some for the total sample. This makes it difficult to summarise data on age across the 15 included studies. However, studies largely targeted young people in their early- to mid-teens (i.e. up to, and including, 16-year-olds). The exceptions to this were the study of Borgia and colleagues51 – the mean for the total sample was 18.3 years [standard deviation (SD) = 1.1] – and Stanton and colleagues,68 who reported a median age of 17 years. The youngest to be targeted were 12 years old (for example, in Levy and colleagues65 and Stanton and colleagues69).

Gender

Study samples were predominantly of mixed sex and were equally balanced between the sexes. The exceptions to this were the samples of Coyle and colleagues,61 of which around two-thirds were male, and Jemmott and colleagues,43 whose intervention was designed only for young men.

Race and ethnicity

Ten studies reported the racial/ethnic profile of the sample, but the level of detail given varied considerably. 43,50,58,61–63,65,66,69,71 The ethnicity of the samples was largely a function of either the country or the aim of the study. In North American studies, the young people were generally ethnically diverse. The exceptions to this were the studies by Stanton and colleagues,69 whose sample was predominantly white (this study took place in a largely white, rural area of West Virginia, USA), and Jemmott and colleagues,62 who specifically targeted African-American young men. In the case of the two UK studies, Stephenson and colleagues’ sample was predominantly white,50 whereas Wight and colleagues70 did not report on the ethnic composition of their sample.

Socioeconomic status

Eight of the included studies reported on the SES of the sample, and again the level of detail given also varied considerably. 50,51,58,62,63,65,69,70 Typically, these samples were either representative of the general population (white, middle class, but including some young people of lower SES, for example studies by Borgia and colleagues51 and Wight and colleagues,70 or they specifically targeted young people of lower SES, for example studies by Jemmott and colleagues,62 and Levy and colleagues65). In four studies, low SES was implicit. 43,63,66,69 A number of studies provided brief details about participants’ education and occupation. Unlike measures of parental education and occupation, these are not commonly used as indicators of participants’ SES.

Sexual behaviour

There was variability across the studies in terms of sexual experience and participation in risk behaviour. All but one of the included studies indicated what proportion of their sample was sexually active at baseline, with some studies reporting additional detail on sexual behaviours. The lowest proportion of sexually active participants were recorded by the two UK-based studies, where less than one-fifth of participants had ever had sex at baseline [6.7% in both study groups of the RIPPLE trial,50 15% in the control group and 19% in the intervention group of the SHARE (sexual health and relationships education) trial70]. Neither of these studies reported any further detail of the young people’s sexual behaviour prior to the study, but it is likely that participants in these studies constituted relatively low-risk groups in terms of STIs.

In six studies between one- and two-fifths of the participants reported ever having sex. 58,63,65–67,71 Four of these studies also reported on condom use. 58,65,67,71 In one study,65 over 70% of participants had ever used a condom, two studies reported condom use at last sex58,67 of between 56.3% and 61%, and, in a further study,71 54.6% of participants reported always using a condom. The sexual behaviour of participants in these studies also appears to be fairly low risk.

In three studies, approximately half of the participants reported ever having sex. 51,62,68 In one of these studies68 almost three-quarters of participants had ever used a condom and nearly one-fifth had experienced more than two sexual partners. In another study,51 only about one-third of participants reported that they always used a condom, but only about 6% had experienced more than one partner. In the third study,62 17.7% of participants reported ever having anal sex, with 8.3% having had anal sex in the preceding 3 months. The participants in these three studies had baseline sexual behaviour that may have put them at some risk.

Coyle and colleagues61 reported the highest proportion of participants who had ever had sex (82.0% in the intervention group, 84.7% in the control group). Although over three-fifths of the participants reported using a condom at last sex, about 20% of participants had ever had anal intercourse, and of these over two-fifths had not used a condom the last time they had anal sex. This group of participants was highly sexually active and some participants took part in risky sexual behaviour.

One study64 did not report an overall figure for the proportion of the participants who were sexually active at baseline. Instead this was reported separately for boys and girls, which indicated a large difference between sexual experience according to gender (50.8% of boys had ever had sex versus 10.4% of girls). No other markers of sexual risk were reported by this study.

The study by Jemmott and colleagues43 was the only one not to provide an indication of what proportion of their study participants (who were all males) were sexually active, but other measures of baseline sexual behaviour were reported, which indicated that this was a group with relatively high-risk sexual behaviour. Only about 30% of participants always used a condom, and over 30% had experienced more than one partner in the preceding month. Almost 13% had taken part in heterosexual anal sex in the previous 3 months.

Other characteristics

Eight studies took place in an urban setting,43,51,58,61–63,65,71 two in a rural setting,66,69 and two in a mixture of urban and rural settings. 50,64 The locations of the remaining studies were either unclear67,68 or not stated. 70 Five studies collected data on drug use,43,51,61,62,65 but only one suggested levels of use to be problematic. 61 Four studies reported alcohol use, in terms of ever having used it or frequency of use. 51,63,65,68 Surprisingly, only three studies reported explicitly on sexual orientation: Roberto and colleagues reported their sample to be exclusively heterosexual,66 Jemmott and colleagues,26,62 reported low levels of homosexual experience in both of their studies. Religious affiliation was only reported by three studies, in which the samples comprised a mixture of Catholics, Protestants or Muslims. 64,67,68

Only one study reported whether or not the young people had ever been considered as offenders;61 in this study, 62% and 57% of the intervention and comparison groups, respectively, had offended. The respective figures for currently being on probation were 53% and 49%. None of the studies reported the HIV status of the young people or whether any currently had an STI, although Stanton and colleagues69 did mention that their study took place in a low HIV seroprevalence setting.

None of the studies provided any information on the following: levels of social capital, disability status, whether any of the sample were commercial sex workers, and whether any of the sample were looked-after young people.

Method of randomisation

Three trials43,62,68 randomised individuals to study groups. Stanton and colleauges68 indicated that a random number table had been used as part of the randomisation process, Jemmott and colleagues43,62 stated that the sample had been stratified before randomisation,43 and indicated that both stratification and a random number table had been involved in the randomisation process. 62

Twelve of the RCTs were cluster trials, 11 of which randomly allocated schools, with one allocating school districts. Six of the twelve studies did not provide details of the randomisation process,63–67,69 although it should be noted that Roberto and colleagues66 only involved two schools. Of the remaining six studies, the process of Zimmerman and colleagues71 involved pairing clusters, Coyle and colleagues58 ranked and paired clusters, whilst Coyle and colleagues,61 Stephenson and colleagues,50 and Borgia and colleagues51 had used processes that involved stratifying or matching clusters. Wight and colleagues70 stated that ‘balanced randomisation’ was used to allocate schools.

Allocation concealment/blinding

None of the studies stated whether allocation to intervention and comparison groups were concealed (blinded). It was also unclear in the majority (12 of the 15 studies) whether participants were aware or could have deduced which study group they had been allocated to. 43,51,58,61,63–69,71 Stephenson and colleagues50 and Jemmott and colleagues62 gave details that suggested either that the schools knew which group they had been allocated to50 or that young people had been given sufficient information about the programme content for each group to have been able to deduce which group they were in. 62 It was clear in Wight and colleagues’ study70 that participants would have known which group they were in. Ten of the fifteen studies did not state whether outcome assessors were blind to group allocation43,58,61,63–67,69,71 and in three studies it was not clear whether outcome assessors were aware of group allocation status. 50,51,68 Only Wight and colleagues70 and Jemmott and colleagues62 specifically stated that data checks70 or data collection62 had been carried out in a blinded fashion.

Comparability of study groups at baseline

More than half of the studies reported baseline sociodemographic and other sexual health-related characteristics for each study group, including all of the individuals who participated in the baseline assessment. 50,51,58,61,63,65,68–70 Wight and colleagues65 and Levy and colleagues70 also provided some baseline data for the individuals who remained in the study at follow-up. Roberto and colleagues66 and Klepp and colleagues64 only provided baseline data for the individuals who remained at follow-up, whilst Zimmerman and colleagues71 restricted reporting to characteristics that differed between the study groups. Three studies did not report baseline characteristics for the intervention and control groups. 43,62,67 The two studies by Jemmott and colleagues43,62 reported baseline values for the whole study population, but Schaalma and colleagues67 reported no numerical data on baseline variables at all.

In five trials, the majority of sexual health-related characteristics were balanced between the study groups. 50,61,64,66,70 Three of these trials, Coyle and colleagues,61 Stephenson and colleagues50 and Wight and colleagues,70 had reported data for all individuals in each study group at baseline. The other two trials, Roberto and colleagues66 and Klepp and colleagues,64 reported baseline data only for the individuals who remained at follow-up. Six studies reported that study groups were unbalanced for more than one of the reported characteristics – these were all studies that had reported baseline data for all individuals in each study group at baseline. 51,58,63,65,68,69 A further three trials did not report baseline data separately for intervention and comparison groups, so it was unclear whether the groups were balanced. 43,62,67 Zimmerman and colleagues71 reported some differences between the study groups, and implied that the imbalances were not significant, but then appeared to make a correction for imbalances in the analyses (although this was not clearly explained).

Of the six trials that reported that study groups were not balanced at baseline, five reported making a correction for this in the analyses,51,58,63,65,69 with Zimmerman and colleagues71 also appearing to make a correction for baseline differences as noted above. Only Stanton and colleagues68 reported imbalances between the groups at baseline, but did not report having corrected for this in the analyses.

Attrition

An overall attrition rate was reported by, or could be calculated from, data in 10 studies. 43,50,58,61–63,65,68,69,71 Attrition ranged from only 3.2% at 3 months in a study by Jemmott and colleagues62 to 55% at 2 years in a study reported by Levy and colleagues. 65 Where studies reported on outcomes at more than one time point, attrition was generally seen to increase with increasing length of follow-up,58,61,62,65,68,71 although there was one study in which attrition remained fairly constant at 3, 6 and 9 months. 69

Twelve of the fifteen studies reported the attrition rate separately according to study group,50,51,58,62–66,68,69–71 although Coyle and colleagues58 did so only for the first of their three follow-up periods. The remaining three studies did not report a numerical value for attrition rate separately according to study group. 43,61,67 Attrition ranged from less than 5% in one or more study groups to over 50%, with greater attrition tending to occur after longer periods of follow-up.

Only two studies clearly reported that there was no statistically significant difference in attrition rates between the arms of the study, 61,62 and one study described attrition between study arms as fairly even. 63 Six studies reported that attrition differed between study groups,50,51,64,68,69,71 with the differences being described as statistically significant in three studies,50,64,68 and statistically significant at some but not all study follow-up points in a further two studies. 68,71 The remaining study51 did not state whether the noted difference in attrition between the groups was statistically significant or not. Four studies did not report whether attrition differed between the study groups. 58,65,66,70

Nine studies51,58,61–65,68,71 commented on whether there were any differences between those participants who were lost to follow-up, and those who were retained. Only one study51 reported that they did not detect any statistically significant differences between participants lost to follow-up, and retained participants. The remaining studies commenting on this noted at least one statistically significant difference between lost and retained participants.

Selective reporting

Eight studies reported outcomes for all individuals/groups,43,50,58,61,63,65,68,70 although there were inevitably some outcomes that only applied to certain subgroups of young people (e.g. those who were sexually active at the start of the study, and those who became sexually active during the study). The remaining seven studies51,62,64,66,67,69,71 reported outcomes only for some individuals/groups. Often these studies reported outcomes only for the subset of participants who contributed both baseline and follow-up data,62,64,67,71 but in other cases it was not clear which individuals or groups were contributing outcome data,51,66 and one study combined three intervention groups together in the reporting of outcomes. 69 All but two studies reported data for all outcome measures. 51,63

Validation of outcome instruments

Outcome data were gathered, in all studies, by self-completion reports, diaries or questionnaires. In addition, one study also interviewed the young people to assess outcomes. 65 Seven of the fifteen studies reported that the questionnaire used to collect outcome data had been developed or tested in a pilot study50,51,61–64,70 and 11 studies reported a coefficient of reliability for some, or all, of the scales used within questionnaires. 43,58,61–67,69,71

Length of follow-up

Stephenson and colleagues50 assessed outcomes at four or more outcome assessment points. The remaining studies varied in reporting outcome data from either one,51,63,66,67,70 two43,64,65,71 or three58,61,62,68,69 assessment points. The ways the studies reported the timing of the outcome assessments were variable: some reported the time elapsed since the baseline assessment, some reported time elapsed since completion of the intervention, and some reported both. Ten of the fifteen studies followed up participants for less than a year,43,51,62,63,66–71 with the remaining five studies continuing to assess outcomes at 1–2 years,61,64,65 2–3 years,58 and 3–7 years50 after the intervention took place. However, it is worth noting that some studies reported high attrition at later time points (see below).

Unit of data analysis

The three trials43,62,68 that randomised individuals to study groups also used individuals as the unit of data analysis. The remaining 12 trials randomised clusters of individuals to groups, but eight of these analysed data at the unit of the individual. 50,58,61,63,65,69–71 In the studies by Roberto and colleagues66 and Borgia and colleagues,51 the unit of data analysis was unclear, but was most probably the individual. Schaalma and colleagues67 described a hierarchical linear model, analysing data at both the student (individual) and school (cluster) level. Stephenson and colleagues50 analysed the primary outcome by age 16 years at the school level (cluster) but the primary outcome at age 20 years, and all secondary outcomes were based on individual participant data. Only the cluster randomised trial reported by Klepp and colleagues64 used the cluster as the unit of data analysis for all of the study outcomes, with two others, Stephenson and colleagues,50 and Schaalma and colleagues,67 using the cluster as the unit of data analysis for some of the study outcomes.

Overall judgements on study quality

Five of the fifteen studies were considered to be methodologically sound (Table 28). 50,62,64,66,70 A further seven were judged to be methodologically sound despite a discrepancy with our quality assessment criteria (see Data extraction in the systematic review and Quality assessment, and Appendix 5), hereafter referred to as methodologically sound studies. 43,51,58,63,65,69,71 Of these seven studies, five failed due to high and/or unbalanced attrition at later time points65,71 or because attrition at later time points was not reported for each study group. 43,58,69 However, the earlier time point(s) provided data that could be considered sound according to our criteria and hence were used in our analysis (but data from the later time points were not). 58,65,69,71 One other study did not provide clear details about attrition but did state that an ITI analysis had been conducted. 51 We therefore considered that attrition bias was unlikely. Karnell and colleagues63 failed to report on all of the outcomes described in their methods; however, it appeared unlikely that reporting bias would have been introduced, as the majority of outcomes were reported and for many of these there were no statistically significant differences between the groups.

In summary then, the results of the systematic review of effectiveness as presented in the following sections are based on a total of 12 RCTs43,50,51,58,62–66,69–71 (of which nine were cluster RCTs), hereafter referred to as being methodologically sound. Three studies were not judged to be methodologically sound and their results are not analysed in this review. 61,67,68

Sexual behaviour

In accordance with our inclusion criteria all of the methodologically sound studies reported on at least one behavioural outcome (the tabulated numerical results for each study are presented in Appendix 8).

The behavioural aims of the studies varied, and about half described more than one behavioural aim (Table 29). Eight studies stated that the aim of the intervention was to impact on risk and/or protective behaviour. 43,50,62,63,65,69,70,71 One of these, Karnell and colleagues,63 was also one of five studies that assessed whether or not the intervention delayed sexual initiation. 50,58,63,64,66 The authors of three studies hoped to see both an increase in condom use and a decrease in the number of sexual partners. 51,58,66 The effect on rates of unintended pregnancy was assessed by two studies. 50,70

The extent to which the active comparators, i.e. standard sex education or teacher-led interventions, aimed to affect behaviour was generally not well described.

The majority of the studies reported more than one behavioural outcome, with some reporting on a wide range of behavioural outcomes. The study by Klepp and colleagues64 was the only one to report a single behavioural outcome (Table 30), and this was also the only study that did not report a condom use outcome. 64 All of the other included studies reported on at least one of nine different condom use outcomes: last intercourse with a condom,50,58,63,66,69,71 last intercourse without a condom,70 first intercourse with a condom,50,58 first intercourse without a condom,70 frequency of condom use,43,51,70 frequency of intercourse with a condom,69 frequency of intercourse without a condom,43,58,62 ever used a condom65 and condom use with foam. 7 As indicated in Table 30, other behavioural outcomes included initiation of sexual activity, episodes of sexual intercourse (primarily vaginal), use of contraception and/or pregnancy rates, and number of sexual partners.

Each of the main types of behavioural outcomes reported by the included studies are presented below.

Initiation of sexual intercourse

Seven studies reported this outcome. 50,58,64,66,69–71 However, Zimmerman and colleagues,71 and Stanton and colleagues69 only reported data on this outcome for their final time points (12–18 months and 9 months, respectively). These data were not included in our review because they did not meet our criteria for methodological soundness at these time points due to attrition bias. In the study by Stanton and colleagues69 the differential attrition rate between study groups was not clear, and in the study by Zimmerman and colleagues the attrition rate exceeded 70% (see Chapter 2, Quality assessment, and Appendix 5). Therefore, five studies contribute data to this outcome, which was assessed with reference to participants’ status at baseline50,58,64,66,70 (Appendix 8, Table 70). The time between baseline and the reported follow-up ranged between 5 and 18 months, but it should be noted that in two cases the intervention had not been completed70 or had been completed only 10 weeks before follow-up66 when data were collected.

Three of the five studies, Coyle and colleagues58 Wight and colleagues70 and Klepp and colleagues,64 found that there was no significant difference between the intervention and comparison group in the initiation of sexual activity among those who were virgins at baseline. The other two studies50,66 did report a statistically significant difference between groups. Roberto and colleagues66 found that young people in the control group were nearly three times more likely to have initiated sexual activity, in the 5 months since the pre-test questionnaire, than students in the intervention group. Stephenson and colleagues50 found that of those who were virgins at baseline, girls in the peer-led group were significantly less likely to report having had sex by age 16 years than were those in the comparison group, but there was no statistically significant difference for boys.

The impact of the interventions on sexual initiation is summarised in Table 31. Data were in a suitable format for meta-analysis in three of the five studies. 50,58,70 Roberto and colleagues66 did not provide the denominators for each group, but these were calculated (using the reported numerators and percentage values) in order to complete the 2 × 2 table of data so that this study could also be included in the meta-analysis. It should be noted, however, that there appeared to be a reporting error in the paper published by Roberto and colleagues:66 once denominator values had been calculated, it was apparent that they did not sum to the reported total number of participants in the study who were sexually naive at baseline. Data from Klepp and colleagues study64 could not be included in the meta-analysis because details were not provided that were necessary for calculating the numerators and denominators corresponding to the reported percentage values of those becoming sexually active in each group. 64

TABLE 31 - Summary of intervention effects on behavioural outcomes

US, unsound data reported but not data extracted.

For subgroup of sexually inexperienced (virgins) at baseline.

Reported for a subgroup of participants only.

Reported all outcomes separately for young men and young women.

Reported for both coitus and heterosexual anal sex.

◆, No significant difference between the study groups (assumed for studies not reporting significance or any p-value).

✓, Outcome favoured the intervention group.

‡, Outcome favoured the intervention for one subgroup of participants, but for one or more subgroups of participants there was no significant difference between the study groups.

	Sexual initiation	Condom first sexa	Condom last sex	Frequent condom use	Freq. sexual intercourse	Had sex	Abstinence	Use of drugs and/or alcohol	Contraception	Pregnancy (♀ only)	Number of sexual partners	Other
Intervention vs standard sex education
Coyle et al.58	◆	◆	✓b	✓b	◆			◆	✓		◆	◆ HIV/STI tests
Karnell et al.63			◆					‡
Levy et al.65				‡	‡						◆b
cWight et al.70	◆	◆	◆	◆					◆	◆		◆ Any unprotected sex
Zimmerman et al.71	US		◆			◆		◆
Intervention vs control (no intervention, delayed intervention, non-sex education intervention)
Jemmott et al.43				◆	◆d	◆d					◆d	✓ Risk behaviour
Jemmott et al.62				✓	✓b	✓b	◆				‡	✓ Risk behaviour
Klepp et al.64	◆
Roberto et al.66	✓		◆								◆
Stanton et al.69	US		◆	◆		◆			◆
Peer-led vs teacher-led interventions
Borgia et al.51				◆							◆
cStephenson et al.50	‡	◆	◆			◆			◆	◆		◆♀ Abortions by age 20

Two of the studies50,70 reported data separately for young women and young men. The data for young men and young women were therefore combined for each group to provide an overall study value for use in the meta-analysis. The fixed-effect pooled OR was 1.03 [95% confidence interval (CI) 0.74 to 1.43] indicating no significant difference between intervention and control (Figure 2). No statistically significant heterogeneity was detected (p = 0.929, I² = 0%).

FIGURE 2.

Meta-analysis forest plot for outcome: ‘delaying sexual initiation’ (OR). Note: The zero value for Coyle and colleagues’ sample size was due to slightly different data entry for this study. The sample size was 2565. Coyle and colleagues did not report data that could be entered into a 2 × 2 table, but did provide an effect size and its standard error, which were entered directly into the eppi-reviewer software for meta-analysis.

Condom use

All but one study, Klepp and colleagues,64 reported on condom use. Outcome measures included whether a condom was used at first sex,50,58,70 whether a condom was used at the most recent episode of sexual intercourse50,58,63,66,69–71 and frequency of condom use,43,51,58,62,69,70 and one study reported whether condoms had ever been used and the use of foam (the term foam was used for any product containing the spermicide nonoxynol-9) with condoms. 65 The numerical results are presented in Appendix 8, Table 71.

Statistically significant effects in favour of the intervention group over the comparison group were only reported by two of the studies. Coyle and colleagues58 found that participants in the intervention group significantly outperformed the comparison group, statistically, on the outcome of condom use at last sex58 (but there was no statistically significant difference in condom use at first sex) and frequency of condom use. 58 A statistically significant reduction in unprotected sex (i.e. increase in frequency of condom use) was also observed by Jemmott and colleagues. 62 This study was one of four studies with a control group that reported on condom use. Outcomes measurement that we considered methodologically sound took place at 6 months62 and 7 months,58 but, due to high attrition in the study by Coyle and colleagues,58 at later time points (19 and 31 months) these outcomes were considered unsound and therefore were not data extracted. Whether improvements made in condom use behaviour can be maintained long term is therefore unknown.

Studies commonly reported either that there was no significant difference in condom use outcomes between the groups, or did not report whether a statistical comparison had been made. Of the studies comparing a behavioural intervention to standard sex education, statistically significant differences were not observed for condom use at first sex by Coyle and colleagues58 as noted above, and this was also true for the study reported by Wight and colleagues. 70 The differences between the groups for outcomes of condom use at last sex,70,71 and frequency of condom use,70 were not significantly different, and Karnell and colleagues63 did not report a statistical comparison for condom use at last sex. Two of the four studies with a control group reporting condom use outcomes also reported that there was no difference between the groups for the outcomes of condom use at last intercourse66,69 and frequency of condom use. 69 The remaining study, Jemmott and colleagues,43 did not report a statistically significant effect for the intervention on either of the condom use outcomes reported (frequency of condom use, and number of days a condom was not used during coitus).

Of the two studies that compared peer-led with teacher-led interventions (Stephenson and colleagues50 and Borgia and colleagues51), neither demonstrated any statistically significant differences between the study groups for condom use at first sex,50 condom use at last sex50 or frequency of condom use. 51 However, statistically significant improvements between the pre-test and post-test were observed in both groups for the frequency of condom use outcome. 51

Levy and colleagues65 limited the reported findings on condom use to the subgroup of young people who had become sexually active in the period between the pre-test and first post-test measure. For one outcome, ever used condoms with foam, a statistically significant difference in favour of the intervention group over the comparison group was observed. But this was not the case for the remaining condom use outcomes of ever used condoms, used condoms in the past 30 days, and used condoms with foam in the past 30 days.

The effects of the interventions on condom use are summarised in Table 31. Although condom use was a commonly reported outcome, there were many different ways that this outcome could be measured. In order to obtain an overview of all possible condom use outcomes, a meta-analysis was conducted for the ‘general’ outcome of all condom use (shown below in Figure 3 – see also additional forest plots in Appendix 9 for some of the constituent condom use outcomes that were used in the general condom use outcome).

FIGURE 3.

Meta-analysis forest plot for the outcome: ‘all condom use’ (OR). Note: The zero value for Coyle and colleagues was due to slightly different data entry into the meta-analysis software for this study. The sample size was 1018 (see Appendix 12 for more detail).

The fixed-effect pooled OR was 1.07 (95% CI 0.88 to 1.30), with no statistically significant difference between intervention and comparator. No statistically significant heterogeneity was detected (p = 0.333, I² = 12.9%).

Skills and self-efficacy

It was a condition of inclusion in our review that interventions had a skills component. However, not all the sound studies reported on whether participants felt that they had gained skills or increased in self-efficacy (their belief in their abilities) to perform certain skills. Where studies did report self-efficacy outcomes these varied depending on the focus of the skills component of the intervention employed in the study. The scales used to quantify self-efficacy varied among studies. Within studies, scales were often different for the various self-efficacy outcomes (the tabulated numerical results for each study are presented in Appendix 8, Table 77).

All twelve sound studies included a skills component within their intervention. These skills components were described as focusing on communication and negotiation, decision-making, risk avoidance, sex refusal or abstinence, and condom use (Table 32). The most commonly described skill was a refusal skill or abstinence skill component, present in seven of the twelve studies; communication and/or negotiation skills training were included by six studies; and six studies included condom use skills training. Condom use skills training involved participants handling condoms in three studies,58,62,70 but it was not clear whether participants handled condoms in the other three studies that included condom skills training. 43,50,71 The intervention reported by Stanton and colleagues69 was to have included exercises relating to condom use. However, during intervention adaptation, communities or schools requested that this element should be removed and therefore over 90% of sites did not include this type of skills training. Decision-making skills were a component of three interventions51,65,69 and two studies taught risk avoidance skills. 63,71

Four studies had interventions that were described as having only one skill component,43,64,66,70 but most studies described an intervention that contained either two50,51,62,63 or three 58,65,69,71 skill components.

The self-efficacy outcomes reported by the included studies did not always correlate with the skills components included within the interventions (Table 33), and four studies did not report a self-efficacy outcome. 43,64,65,70 The most commonly reported self-efficacy measure was condom use self-efficacy, which was reported by seven of the eight studies that reported a self-efficacy outcome. 50,58,62,63,66,69,71

Six studies reported refusal or abstinence self-efficacy,50,58,63,66,69,71 four reported communication/negotiation self-efficacy,50,51,58,66 and two reported situational self-efficacy (the negotiation of potentially risky situations). 66,71 Five studies assessed self-efficacy at more than one time point,50,58,62,69,71 but the high attrition rate in two of these studies at the later time point meant that the data were not extracted. 58,71

Two of the three studies that compared a behavioural intervention with standard sex education found there were no statistically significant differences in self-efficacy outcomes between the groups. 63,71 Coyle and colleagues58 reported a statistically significant effect for condom use self-efficacy, but not for the other two self-efficacy items they assessed. Karnell and colleagues63 reported a statistically significant intervention effect for refusal self-efficacy, but only in one subgroup: young women in the intervention group had greater refusal self-efficacy than young women in the comparison group. Conversely, the three comparisons between behavioural intervention groups and control groups (receiving no intervention) demonstrated a greater number of statistically significant intervention effects. Jemmott and colleagues62 found that the intervention group had significantly greater condom self-efficacy, statistically, immediately after the intervention, and at 3 and 6 months post intervention, than the control group. Similarly, a statistically significant difference in favour of the intervention group 6-months’ post intervention (but not at 3 months post intervention) was reported by Stanton and colleagues69 for both condom self-efficacy and abstinence self-efficacy. Roberto and colleagues,66 who reported four different self-efficacy outcomes, found statistically significant benefits of the intervention on condom negotiation and situational self-efficacy, but did not report any statistically significant effects for the other two measures – condom use and refusal self-efficacy.

Of the two studies that compared peer- with teacher-led interventions, one (Borgia and colleagues51) reported a statistically significant improvement in prevention skills (including communication and negotiation skills) in both groups following the intervention in comparison to pre-intervention scores. However, there were no statistically significant differences between the groups. The other study to compare peer- with teacher-led interventions, Stephenson and colleagues,50 also reported that there were no differences between the groups for the communication measure (confidence about discussing sex and contraception with a partner). Statistically significant differences were found for girls, at the 18-month follow-up, who were less confident in the peer-led arm than those in the teacher-led arm about refusing to do something they did not want to do sexually, but who became more confident about using condoms.

Knowledge

All 12 sound studies attempted to measure participants’ knowledge, but the knowledge items that were measured varied depending on the focus of the educational component of the intervention employed in the study. The extent to which knowledge outcomes were reported also varied (the tabulated numerical results for each study are presented in Appendix 8, Table 76).

The majority of the studies tested participants’ knowledge of HIV51,64,65 or their knowledge of HIV and STIs more generally. 43,50,58,62,63,69 Zimmerman and colleagues71 also tested knowledge of pregnancy prevention as well as HIV and STIs. Wight and colleagues70 tested practical knowledge of sexual health, though this was not further defined. The knowledge tested in the remaining study, Roberto and colleagues,66 was not clear as no description was provided.

Knowledge was tested at the same time as other outcomes at periods, ranging from immediately after the intervention to 18 months following intervention. Seven studies assessed knowledge at more than one time point,43,50,58,62,65,69,71 but the high attrition rate in three of these studies at the later time point meant that only the data for earlier time points were extracted, whereas the later time points did not meet our quality assessment criteria for being methodologically sound. 58,65,71

Statistically significant effects in favour of the intervention group over the comparison/control group were found by all but two of the studies. Four of the five studies that made a comparison with standard sex education found that participants in the intervention group significantly outperformed the comparison group, statistically, in the knowledge test. 58,65,70,71 Only Karnell and colleagues63 reported that there was no intervention effect. Similarly, compared with control groups (e.g. no intervention, a delayed intervention or a non-sex education intervention), four of the five studies reported that the intervention group statistically significantly outperformed the control group in the knowledge test,43,62,64,66 whilst Stanton and colleagues69 did not report on whether there was any difference between the groups.

Both studies that compared peer- with teacher-led interventions50,51 reported that participants in the peer-led groups scored statistically significantly better on the knowledge outcomes than participants in the teacher-led groups.

The impact of the interventions on knowledge outcomes is summarised in Table 35.

Attitudes

Eight of the included studies had an assessment of participants’ attitudes among their outcomes. 43,50,51,58,63,64,66,71 Six of these investigated participants’ attitudes towards risky sexual behaviour or sexual intercourse,43,50,58,64,66,71 with two specifically focused on attitudes towards waiting to have sex. 66,71 Three studies reported on attitudes towards condom use50,58,63 – two on attitudes relating to people with AIDS51,64 and one included positive and negative attitudes towards alcohol63 (see Appendix 8, Table 78).

Jemmott and colleagues43 reported statistically significantly less favourable attitudes towards risky sexual behaviours among the intervention group immediately following the intervention, but 3 months after the intervention this effect had waned to a non-statistically significant trend in favour of the intervention group over the control group. Roberto and colleagues66 were the only authors to report a statistically significant effect in favour of the intervention group over the control group for participants’ attitude towards waiting to have sexual intercourse. The other study to focus specifically on attitudes to waiting to have sexual intercourse reported that there was no significant difference between the groups,71 and the other three studies that reported attitudes towards sexual intercourse also found no significant difference between the groups. 50,58,64 Similarly, only one of the three studies reporting on attitudes towards condom use (Coyle and colleagues58) found a statistically significant effect in favour of the intervention group, whereas the other two studies, Stephenson and colleagues50 and Karnell and colleagues,63 found no significant difference between the groups50 or did not report a statistical comparison. 63 Both of the studies reporting on attitudes of participants towards people with AIDS reported significant changes. Klepp and colleagues64 found a statistically significant effect in favour of the intervention group over the control group, whereas as Borgia and colleagues,51 assessing peer- and teacher-led interventions, found that whilst statistically significant improvements were observed in both groups between the pre-test and the post-test, a post-test difference between the two groups was not reported. The single study that assessed both positive and negative attitudes to alcohol did not report on any statistical comparison between the groups. 63

Behavioural intentions

Outcomes relating to participants’ intentions were reported by six studies,7,43,62–64,71 although one of these, by Levy and colleagues,65 only reported intention outcomes for the subgroup of participants who had become sexually active during the course of the study. Four of the six studies reported on participants’ intentions to have sex,63–65,71 one reported on intentions to engage in risky sexual behaviour,43 three reported on intentions to use condoms62,63,65 (one further study, by Zimmerman and colleagues,71 indicated that condom use intentions formed part of their participant questionnaire but did not report on this outcome), and the study by Levy and colleagues65 reported on participants’ intention to use condoms with foam (Appendix 8, Table 79).

Only one of the studies reporting on participants’ intention to have sex. Klepp and colleagues64 found a statistically significant difference in favour of the intervention group at follow-up. Participants in the intervention group had a reduced intention to have sex in comparison to the control group. The three other studies reporting on this outcome either reported no significant differences between the groups71 or did not report a statistical comparison. 63,65

The only study reporting intentions to engage in risky sexual behaviours found a statistically significant effect in favour of the intervention participants (participants had weaker intentions than the control group to engage in risky sex behaviours) both immediately after the intervention, and at the three month follow-up. 43 In terms of intention to use condoms, only one study reported a statistically significant difference between study groups in favour of the intervention group for the intention to use condoms. 62 Karnell and colleagues63 found that overall there was no difference between intervention and comparison groups, but there was a statistically significant difference in favour of the intervention group for the subgroup of participants who were already sexually active at baseline. The third study reporting on the intention to use condoms, by Levy and colleagues,65 did not report a statistical comparison for this outcome, but did report that the subgroup of participants who had become sexually active during the course of the study in the intervention group were more likely than participants in the comparison group to express an intention to use condoms with foam. 65

Infection rates

None of the included studies reported infection rates. Coyle and colleagues58 reported whether participants had had a HIV or STI test, but the outcome of these tests is not known.

Summary of the results of sound outcome evaluations

• Sexual behaviour:

  1. – Outcomes related to the included sexual behaviours were reported under the headings: initiation of sexual intercourse; condom use; sexual intercourse; contraception and pregnancy; sexual partners; and other behavioural outcomes.

  2. – Interventions resulted in few statistically significant effects on sexual behaviour. Statistically significant effects on at least one behavioural outcome were reported by seven of the twelve studies. However, in three cases the effect was limited to a subgroup of participants, and in all studies no statistically significant effect was observed for some or all of the other behavioural outcomes reported on.

  3. – Five of twelve studies did not report that the intervention had any statistically significant behavioural effects.

• Skills and self-efficacy:

  1. – Two-thirds of the included studies reported a self-efficacy outcome, although all studies included a skills component within their intervention.

  2. – Most studies reported on more than one self-efficacy outcome and reported mixed results, statistically significant effects for some, but not all participants, or statistically significant effects for some, but not all self-efficacy outcomes.

  3. – One study reporting on a single self-efficacy measure reported a statistically significant intervention effect.

  4. – Two studies, one of which reported a single measure, found that the intervention had no statistically significant effect on the reported self-efficacy measures.

• Knowledge:

  1. – Ten of the twelve included studies reported that the intervention had a statistically significant effect on increasing knowledge.

• Attitudes:

  1. – Nine studies reported attitude outcomes. Statistically significant effects of the intervention were reported by more of the studies that assessed an intervention in relation to a control group than those that assessed an intervention against standard sex education or teacher-led education.

• Behavioural intentions:

  1. – Half of the studies reported on behavioural intentions. As noted above for attitudes, a statistically significant intervention effect was more likely to be reported by studies with a control group rather than those with a comparison group.

• Infection rates:

  1. – None of the included studies reported on a biological outcome relating to infection with STIs. One study reported whether participants had undergone an HIV or STI test, but the outcome of the tests was not reported.

Fidelity of implementation

The fidelity of intervention implementation (i.e. the extent to which the intervention was delivered as intended) was assessed by all studies, but only seven reported findings on this. 50,51,63,65,66,70,71 As a number of these were multisite cluster RCTs they involved the implementation of a standardised programme across a number of schools.

Three studies found that implementation fidelity varied across and within schools. In the UK, Stephenson and colleagues50 found that overall, 84% of the intended recipients of RIPPLE reported that they had received at least some peer education, but this varied between 51% and 97% across schools. Wight and colleagues70 found that although the Scottish-based teacher-delivered SHARE programme was implemented in all 12 experimental schools, there was variation in the extent to which all 20 sessions of the programme were covered (from 38% to 88% across schools). In the USA, Zimmerman and colleagues71 also found that implementation of the 16–17 session teacher- and peer-delivered ‘Reducing the Risk’ programme varied somewhat across schools and classrooms.

The factors influencing the implementation of the intervention are discussed in detail below, but a particular challenge was consistent implementation of some of the interactive and ‘novel’ elements of the programmes. In the RIPPLE trial,50 whilst most students reported that the intervention had covered topics such as HIV, STIs, where to get contraception/condoms and medical advice, and had looked at condoms, fewer students reported that they had practiced putting a condom on (76%), discussed how to use condoms with a partner (39%) or practiced resisting pressure to have sex (55%). (However, greater numbers of students in the experimental schools reported that they had received skill-based activities than in the control schools.) In the SHARE trial70 teachers reported making ‘considerable changes’ to five of the 20 lessons in the programme. These tended to be those lessons that involved role playing to develop communication and resistance skills and one lesson in which young people got the chance to visit a family planning clinic. Parental involvement was a novel element of the US-based Youth AIDS Prevention Project (YAPP) programme, but Levy and colleagues65 reported that, despite substantial cost and effort, it was not possible to get parents to attend on-site school activities.

From the synthesis, a hierarchy emerged (Figure 4), incorporating a number of factors that may impact on fidelity of implementation.

FIGURE 4.

Factors impacting upon fidelity of implementation.

Student engagement and intervention acceptability

The interventions evaluated by the studies included in this review shared two common features. Firstly, all interventions were designed to engage young people actively in their own learning through interactive exercises, such as role plays, discussions and small group work. Secondly, all interventions were designed to be relevant and appealing to young people in general or particular groups of young people. The latter was attempted in various ways including the use of peer leaders to deliver the intervention and attempts to make curriculum materials interesting, fun and relevant through, for example, the use of fictional teenage characters. Although there was evidence from the process evaluations to confirm that interventions did engage and appeal to many of the young people involved, this was not always the case.

Six of the process evaluations contributed findings that illuminated issues of student engagement and intervention acceptability. 50,51,63,66,70,71 A number of factors emerged as influences on this issue (Figure 5).

FIGURE 5.

Factors related to student engagement and intervention acceptability.

Summary of process synthesis findings

• Fidelity of implementation Three of the process evaluations (including the two most extensive evaluations – Stephenson and colleagues50 and Wight and colleagues70) reported variation in fidelity of implementation. In some cases, fidelity was greatly compromised.

• School culture School culture (the involvement and commitment of key stakeholders, management support, the prioritisation of PSHE, overall morale) was vital in providing an accommodating context for the implementation of the intervention. Again, this was found to vary widely across schools.

• School administration Staff absence and turnover, timetabling issues and shortage of time acted as important barriers to fidelity of implementation.

• Teachers – enthusiasm, expertise, autonomy Wight and colleagues70 found that enthusiasm, expertise and autonomy of teachers were vital to the delivery of the intervention. Despite thorough and highly acceptable training, teachers often failed to engage with the more interactive elements of the SHARE intervention and with the theory that underpinned it.

• Peers Selection of peers with the qualities that make an effective leader was important; this did not always happen. Some of the peer leaders were found to lack enthusiasm, organisation and the skills to manage and engage participants.

• Parents It is feasible to involve parents in homework tasks, but less so in attendance of on-site activities.

• Health educators/facilitators The studies did not report on the impact on fidelity of implementation of employing health educators or other facilitators.

• Computer The one study66 that used a computer to deliver the intervention reported high fidelity of implementation.

• Content appeal It is both feasible and important to develop sexual health interventions with content that is highly acceptable to young people. However, acceptability alone does not guarantee effectiveness.

• Provider qualities It is vital to the success of any skills-based intervention that providers, whether teachers, peers or other facilitators, have sufficient expertise to deliver the intervention effectively. Process evaluations showed that this was not the case in many of the interventions reported on.

• Meeting needs It is less likely that interventions will impact on behaviour if they do not meet young people’s self-identified needs. According to Stephenson and colleagues,50 in the case of sex education these include sexual feelings, emotions and relationships, which were often addressed only partially or not at all.

• Gendered norms Gendered norms, especially the pressure on boys to conform to accepted notions of masculinity, inhibit and disrupt discussion of sexual issues in the classroom. Mixed-sex groups were more successful for boys than single-sex groups.

• Age/timing Some of the reviewed interventions were not age appropriate, either because they were felt to be more suitable for a younger age group or – due to the sexual inexperience of the participants – an older one.

• Discomfort Pupils can experience discomfort in engaging with interactive interventions in the classroom relating to sexual behaviour.

Search strategy

A systematic literature search was undertaken to identify economic evaluations for behavioural interventions for sexually transmitted interventions for young people. Sensitive search strategies were developed and tested by an experienced information scientist [see Appendix 10 for the MEDLINE (Ovid) search strategy]. These strategies were used to search the following electronic bibliographic databases:

• The Cochrane Central Register of Controlled Trials (CCRCT) (Issue 1, 2008)

• MEDLINE (via Ovid)

• MEDLINE In-Process & Other Non-Indexed Citations (via Ovid)

• EMBASE (via Ovid)

• Science Citation Index

• NHS Economic Evaluation Database (NHS EED, via NIHR CRD)

• Health Technology Assessment Database (via Cochrane Library)

• Database of Abstracts of Reviews of Effectiveness (DARE) (via Cochrane Library)

• Econlit

Searches were limited to the period 1990 to February 2008.

Inclusion criteria

Titles and (where available) abstracts of references identified by the search strategy were assessed for potential eligibility against inclusion criteria (Table 43) by a health economist. Full papers of those which appeared relevant on title or abstract were retrieved and independently screened by two health economists. Any differences in judgement were resolved through discussion.

The quality of these economic evaluations has been assessed using a standard checklist adapted from Drummond and Jefferson93 (Table 44).

Estimation of outcomes within economic evaluations

Four of the studies used a model developed by Weinstein and colleagues. 101 The Bernoulli model of HIV transmission is a cumulative probability equation that describes the probability of HIV infection based upon HIV prevalence (π), single act transmission probability (α), condom effectiveness (e) and condom use (f), number of sexual episodes (n) and number of sexual partners (m). For example, the estimated probability of an uninfected person becoming infected is P,

The model estimates the probability of becoming infected for the intervention and comparator groups according to changes in parameters that may be affected by the intervention, i.e. condom use, number of sexual partners and number of sexual episodes. The number of cases averted is estimated by multiplying the results by the number of people who receive the intervention.

Wang and colleagues59 estimated the probability of infection during a 3-month period and then converted this 3-month probability into a 1-year probability. They used the following equation to calculate the total number of cases averted for primary transmission:

where X is the total number of cases averted, N is the number of uninfected students, and P_i and P_c are the probability of infection for the intervention and control groups, respectively.

In addition, the number of secondary transmissions is estimated. Secondary transmissions are those transmissions from infected intervention students to non-infected sexual partners. In this case the probability of infection, P, is

where α is single act transmission probability, e is condom effectiveness, f is frequency of condom use and n is number of sexual episodes.

Pinkerton and colleagues95 used similar expressions to estimate the number of primary and secondary infections associated with anal intercourse. Cohen and colleagues60 used this model to estimate the cost-effectiveness of a range of behavioural interventions. In addition to HIV infections, Wang and colleagues used the equations presented to estimate the cases of other STI infections avoided, i.e. chlamydia, gonorrhoea and pelvic inflammatory disease. They also developed a pregnancy model to translate contraceptive use into cases of pregnancy averted.

Hogan and colleagues96 developed a mathematical model of HIV/AIDS, based on the Bernoulli model described above, combined with the progression from HIV to AIDS and AIDS to death and transmission of other STIs.

Tao and Remafedi94 used a dynamic mathematical model to estimate the cost-effectiveness of HIV prevention for gay and bisexual young people. They modelled HIV transmission by calculating changes in the number of risky partners (i.e. unprotected) and then projected quarterly prevalence of HIV in the target population for a control and intervention population over 10 years. They assumed that without intervention, risky behaviour in the target population would not change during the 10-year period. Participants maintained less risky behaviour for only 1 year before relapsing to the previous level of risk. The target population was classified by participants, non-participants, infected and uninfected subgroups. The model calculated HIV prevalence after 10 years and used parameters for the number of risky partners, initial HIV prevalence, the probability of HIV transmission in each infected–uninfected partnership, and the percentage of gay and bisexual adolescents recruited into the intervention.

The studies used a range of values for the input parameters. HIV prevalence rates varied between 0.1% and 0.6% for US heterosexual adolescents, and 2% for gay and bisexual adolescents in the USA. The probability of transmission of HIV varied between 0.1% to 1.6% for US heterosexual adolescents and 6% for gay and bisexual adolescents. The increase in condom use varied between 11% and 24% for heterosexual adolescents, with a 50% increase in gay and bisexual individuals. Wang and colleagues59 did not report any change in the number of sexual partners and episodes, whilst Pinkerton and colleagues95 found intervention participants engaged in vaginal intercourse on less than half the number of days and with half the number of partners.

Estimation of quality-adjusted life-years

Tao and Remafedi94 and Pinkerton and colleagues95 estimated that 16.9 QALYs and 14 QALYs, respectively, would be lost by an HIV-infected young person based on previous work by Holtgrave and colleagues. 102 These estimates were based upon the person receiving standard HIV-related medical treatment (including antiretroviral treatment). Hogan and colleagues96 used a life expectancy estimate of between 8.5 and 16 years with HIV for males, depending on whether or not they received treatment.

Estimation of costs

Each of the studies estimated the intervention costs, including staff training, renting space, condoms, administration and monetary incentives for the participants. Wang and colleagues estimated costs of US$26 per participant, Pinkerton and their colleagues £89. Tao and Remafedi94 reported much higher costs of over US$440 per participant. The costs for the group counselling and youth supervision programmes in the study by Cohen and colleagues60 were US$300 and US$57 per participant, respectively. Hogan and colleagues96 reported annual costs of school-based education from US$58M to US$77M for 50–95% coverage in the sub-Saharan Africa region. The differences in the costs were mainly due to staffing costs. For example, Tao and Remafedi94 employed several staff, including a full-time case manager, two directors, a secretary and field worker at a total cost of US$184,006, in order to provide an intervention for about 500 participants. On the other hand, Wang and colleagues included costs for only teaching, teacher training and site co-ordination costs at a total cost of US$85,599 for almost 4000 participants.

Three of the studies included the cost of future medical care for HIV in the cost-effectiveness analyses. Wang and colleagues59 and Pinkerton and colleagues95 applied discount rates of 5% and 3%, respectively, to estimate the cost of medical care based on a previous study by Holtgrave and colleagues. The cost used for future medical HIV care varied between US$78,425 and US$195,188 per person with HIV. Wang and colleagues59 also estimated the medical costs for chlamydia and gonorrhoea treatments, based upon costs from the public sector analysis from the California Medicaid Program. Wang and colleagues estimated the social cost of HIV infection in terms of lost productivity or foregone wages of US$430,000 per patient. It included the costs of earning-related outcomes, public assistance and other consequences.

Comparison of results

The base-case results of the cost-effectiveness studies are presented in Table 46. A comparison of the studies shows a wide range of cost-effectiveness estimates depending upon the assumptions and parameter values used. For example, Cohen and colleagues60 found one of the interventions (‘Safer Choices’) to be not cost-effective, with a cost per case averted > US$39M. In contrast, Wang and colleagues59 found the same intervention to be cost saving. Wang and colleagues included the effect of other STIs and unintended pregnancy. In particular, unintended pregnancy had a large effect on the results, with the medical and social costs for pregnancy comprising over half the total averted costs. However, the effects of the intervention in preventing pregnancy appear to have been overestimated. They estimated that 18.5 pregnancies were prevented in 345 ninth-grade students (aged 14–15), but this is likely to be higher than the conception rate in this age group. The under-16 conception rate for England in 2005 was 7.7 per 1000 girls aged 13–15. Furthermore, there were differences between the studies for the values for the probability of HIV transmission from 0.001 to 0.016.

Several of the studies conducted sensitivity analyses on the main model parameters. The most influential parameters were found to be baseline HIV prevalence, baseline number of risky partners, cost per person reached by the intervention, duration of the effect of the intervention and the single sex act transmission rate.

Published economic evaluations – summary of methods

A systematic review of cost-effectiveness studies identified five economic evaluations of behavioural interventions for prevention of HIV, published between 1998 and 2005. Only one study evaluated other STIs in addition to HIV. All studies used mathematical models extrapolating the changes in sexual behaviour to number of cases of HIV averted. With the exception of one study for developing countries, all studies evaluated the interventions on a US population. All interventions were effective at encouraging safer sexual behaviour and were estimated to avert cases of HIV transmission through modelling. There was a range in assumptions and parameter values used in the mathematical models, and this led to substantial differences in the estimated cost-effectiveness of the behavioural interventions. Therefore, no existing model was appropriate for this study and so we developed a de novo model.

Static versus dynamic modelling

In the epidemiology of infectious diseases, the force of infection is the rate at which susceptible individuals become infected. Vaccination against infectious disease not only reduces the incidence of disease in those immunised, but also indirectly protects non-immunised susceptible people,109 a concept known as herd immunity. Static models assume a constant force of infection and cannot take into account reinfection. On the other hand, dynamic models are able to capture herd immunity effects.

Barham and colleagues26 systematically reviewed and critically appraised the economic evaluation of one-to-one interventions to reduce STIs and teenage conceptions. The review was conducted to underpin NICE’s guidance in this area. 22 The majority of studies examined the cost-effectiveness of chlamydia and HIV screening programmes in various settings. There were fewer published studies of other STIs, and of behavioural interventions. The vast majority of studies used static modelling techniques. However, three studies (out of 55) employed dynamic approaches for the cost-effectiveness of chlamydia screening.

Welte and colleagues110 compared a dynamic (stochastic network simulation model) with a static (decision–analysis) model for evaluating an opportunistic screening programme for chlamydia. The dynamic model required much more detailed data about sexual behaviour and the infectious disease than the static model, such as duration of partnerships, frequency of sexual intercourse in partnerships and transmission probability per sexual episode. It produced results that were more favourable in terms of cost-effectiveness than the static model.

Welte and colleagues110 suggest that dynamic, rather than static, models are appropriate for the economic evaluation of chlamydia screening programmes that might affect the force of infection. They state that static models have frequently been applied in the past, and these did not include the risk of reinfection resulting from failed partner referral (Table 47). The only reasons against using the dynamic approach were its higher complexity, data demand, time and monetary costs and need of mathematical modelling expertise. On the other hand, static models might be the preferred option for estimating the cost-effectiveness of screening programmes that have no impact on the force of infection.

Kretzschmar and colleagues111 developed a stochastic simulation model for chlamydia screening in the Netherlands. They made several assumptions to derive data for the model and had limited knowledge about disease-specific parameters, such as the probability of transmission per sexual episode, the average duration of the infectious period, and the fraction of asymptomatically infected persons. They suggested that their results should be interpreted mainly in a qualitative sense and not too much importance should be attached to the absolute numbers due to the uncertainties in parameter values.

Roberts and colleagues,103 funded by the NIHR HTA programme, reviewed economic evaluations of various forms of chlamydia screening. They found that the majority of studies, all since 2000, had used static models. They suggested that this reflects the view that the simplicity of static models outweighs the limitations of violating the assumption of independence. They recommended the use of either DES or system dynamics models which provide more realistic representations of complex systems. These models ‘can take into account the full economic consequences of interpersonal interactions’ (p. 198). These interactions mean that the risk of infection depends upon the background STI prevalence, and that screened and treated individuals will not transmit infection but are susceptible to re-infection. Screened partners that remain untreated can also continue to transmit infection.

Turner and colleagues112 developed a stochastic, individual-based network model to evaluate the effectiveness of opportunistic chlamydia screening (‘The National Chlamydia Screening Programme’). They chose to use an individual-based dynamic model because population-based models ‘fail to capture important individual level effects in the sexual network. For example, reinfection is dependent on the infection and treatment status of current partners, not the average level of infection in the community’ (p. 4). For many of the model parameters the values were highly variable, the parameters of interest could not be measured directly or few data were available at all. They estimated some of these parameters by fitting the model to data. Several parameters were unknown: the proportion of individuals desiring short partnerships, the proportion of individuals changing from wanting short partnerships to long partnerships each year, the average duration of long partnerships, the annual increase in preferred partnership duration and the average gap between partnerships. Other parameters that were subject to high uncertainty were duration of chlamydial infection and transmission probability, the proportion seeking treatment and the level of partner notification.

Low and colleagues25 developed a dynamic model to estimate the cost-effectiveness of active population screening for chlamydia. The model took into account the effects of ageing and replacement (i.e. new young people entering the model), partnership formation and dissolution, chlamydia transmission and progression, testing and treatment, and the complications from chlamydia. The authors stated there were no data available on the activity groups to estimate the propensity to form new partnerships and the mixing between activity groups. Instead of direct incorporation of data as input parameters, it was necessary to adjust the input parameters until a reasonable fit to the available data were obtained.

The two models developed for chlamydia screening provided very different estimations of cost-effectiveness, even though both used similar modelling methods (£1350112 versus £26,00025 per major outcome avoided). The differences between these models were due to differences in assumptions and data values, and these differences far exceed the difference shown comparing the results from static and dynamic models (Welte and colleagues110).

Discussion and key points

Modelling guidelines suggest that DES is suitable for modelling disease process when interaction between individuals is important. However, almost all previously published studies have used a static approach. Dynamic models are considerably more complicated, time consuming and require more data, which is often not available. Much of the discussion on type of modelling has focused upon chlamydia screening, as opposed to primary prevention, and even in these studies there were few data for key model parameters. The additional time, expense, and complexity involved in dynamic modelling may not actually provide a gain in precision.

In view of these points, and the lack of impact of school-based behavioural interventions on sexual behaviour (and therefore the low probability of altering rates of infection) demonstrated in our systematic review, we chose to construct a static economic model to explore the potential cost-effectiveness of behavioural interventions under various assumptions.

Overview

A comparison of the costs and benefits of behavioural interventions for the prevention of STIs in young people was made using decision-analytic models.

The cost-effectiveness of two types of school-based behavioural intervention were assessed:

1. A teacher-led curriculum, spread over 20 sessions (based on the Scottish SHARE trial70).

2. A brief peer-led classroom curriculum, spread over three sessions (based on the RIPPLE trial in Central and Southern England50).

Both interventions provide factual information about STIs in addition to the teaching of skills associated with the practice of safer sex. These two interventions were considered to be broadly representative of the interventions included in our systematic review. However, they were prioritised for economic modelling because, in terms of costs and resources, they were considered to be more reflective of UK practice than many of the other (mostly US) studies in our review. This is particularly the case for the SHARE programme, which has been implemented by NHS Health Scotland.

The comparator was standard sexual health education, which is generally provided by teachers in schools as part of the PSHE curriculum. In the studies included in our systematic review of effectiveness (Chapter 4), standard sexual health education tended to provide basic information on STIs and sexual health, but did not teach skills. It is therefore the teaching of safer sex skills and other broader activities that distinguishes between the behavioural intervention and standard education.

Models were constructed in Microsoft excel according to standard modelling methods (NICE113). To identify data to populate the model, systematic searches were conducted to locate studies on the natural history and epidemiology of STIs, sexual behaviour and lifestyles, HRQoL, and costs. Various websites of relevant organisations were also searched (e.g. HPA) and contact made with experts to identify data.

The quality of data used for the model varied. Generally, there were few data for children aged under 18 years old, and, where there were no data, assumptions were made from existing data. The baseline clinical data were estimated from administrative databases for the UK, and prospective studies. The HRQoL data have been taken from previous utility studies using validated tools for groups of patients with the condition of interest.

Costs were derived from published studies (where available), and from national and local NHS unit costs. Only direct NHS and PSS costs were included and hence the model was from the perspective of the NHS and PSS.

In the model, the intervention effects last for 1 year, on the basis that the majority of the trials included in our systematic review assessed outcomes up to 1 year (see Chapter 4, Length of follow-up). The model estimates the lifelong costs and benefits from averted STI cases. The intervention effect (condom use) was derived from the systematic review of effectiveness reported in Chapter 4. The economic evaluation focused on estimating the number of cases of STI and associated complications averted. The outcome is reported as cost per QALY gained.

Model structure

We adapted the Bernoulli model, previously developed by Weinstein and colleagues,101 as it estimates the effect of changes in sexual behaviour in terms of STIs averted. This model has been described earlier in Chapter 6 (see Results). The Bernoulli model of HIV transmission is a cumulative probability equation that describes the probability of HIV infection based upon HIV prevalence (π), single act transmission probability (α), condom effectiveness (e) and condom use (f), number of sexual episodes (n), and number of sexual partners (m). The equation described in Chapter 6 is an approximation to the original equation, but this approximation is only close when the proportion of condom use is either very low or extremely high, or when the infectivity is minimal (α < 0.001). Consequently, that equation is not appropriate for chlamydia, gonorrhoea and genital warts. For this reason we split the equation for condom users (c) and non-condom users (nc) so that, the estimated probability of an uninfected person becoming infected is P = fP_c + (1 – f)P_nc, where risk for a condom user P_c is:Pc=1−{(1−π)+π|1−α(1−e)|n}m and risk for a non-condom user, P_nc is,Pnc=1−{(1−π)+π|1−α|n}m

The model estimates the probability of becoming infected for the intervention and comparator groups according to changes in parameters that may be affected by the intervention, i.e. condom use, number of sexual partners and number of sexual episodes. The number of cases averted is estimated by multiplying by the number of people who receive the intervention.

These cases averted, in turn, would have infected further individuals, i.e. through secondary transmission. The estimated probability of an uninfected person becoming infected is P = fP_c + (1 – f)P_nc, where risk for a condom user Pc is,

and risk for a non-condom user, P_nc is,

The number of cases averted through secondary transmission is estimated by multiplying the risk of becoming infected by the number of cases averted through primary transmission.

The model estimates the number of STI cases averted for HIV and also for chlamydia, gonorrhoea and genital warts, according to the risk of infection as shown above and the proportion of sexually active individuals who receive the intervention. For each STI case averted there was a HRQoL loss and resource use cost associated due to complications, such as PID or infertility. The data and assumptions used to derive the model parameters are described in the following sections.

The total number of STI cases averted, and consequent QALY gain, cost of the intervention and the saving in medical costs is estimated for males and females for all STIs for 1 year. Thus the cost-effectiveness is calculated,

The cost-effectiveness results are shown later (see Results of the modelling, below). For the base-case analysis, a cohort of children aged 15 years old receive the teacher-led intervention.

Assessment of uncertainty

The evaluation of the cost-effectiveness of behavioural interventions for preventing STIs is based on uncertain information about variables, such as clinical effect, HRQoL and resource use. This uncertainty was evaluated using deterministic and probabilistic sensitivity analyses. One-way deterministic sensitivity analyses were conducted to evaluate the influence of individual parameters on the model results and test the robustness of the cost-effectiveness results to variations in the structural assumptions and parameter inputs (see Sensitivity analysis).

Multiparameter uncertainty in the model was addressed using probabilistic sensitivity analysis (PSA) (see Probabilistic sensitivity analysis). 114 In PSA, probability distributions are assigned to the point estimates used in the base-case analysis. The model is run for a number of iterations (in this case 1000) according to a different set of parameter values, using Monte Carlo simulation methods, to give a range of results. The main parameters were varied according to the ranges used in the one-way deterministic sensitivity analysis. For each iteration, parameter values are sampled at random from their probability distributions. The uncertainty surrounding the cost-effectiveness of the behavioural intervention is represented on a cost-effectiveness acceptability curve (CEAC) according to the probability that the intervention will be cost-effective at a particular willingness-to-pay threshold. Appendix 11 reports the parameters included in the PSA, the form of distribution used for sampling each parameter, and the upper and lower limits assumed for each variable.

Finally, value of information analyses were conducted to investigate the expected pay-off from further research (see Expected value of perfect information for the prevention of STIs). 114,115 When decisions are based on imperfect or uncertain information, there is a risk that a wrong decision will be made and there will be a loss in costs and health benefits. The expected value of perfect information (EVPI) shows the value of reducing the uncertainty around the decision of whether or not to adopt the intervention. The EVPI is estimated for different cost-effectiveness thresholds and this gives an indication of an upper bound for further research at these thresholds. In order to determine where further research would have the most effect in reducing uncertainty, a partial EVPI (EVPPI) is conducted. In this analysis, the uncertainty around particular input parameters in the model is investigated.

Data synthesis

Sensitivity analysis

The parameters in the STI model were varied in a series of sensitivity analyses for the base case and the results are shown in Table 62 and Figure 6.

FIGURE 6.

Tornado plot for sensitivity analyses for the STI model. BC, base case; IE intervention effect.

Where possible, the parameters were varied according to the ranges of the CIs of these parameters, otherwise a suitable range was chosen. Parameter values for all the STIs were altered by the same magnitude, but, for simplicity, only the input parameters for chlamydia are shown in Table 62 and input parameters for the other STIs are shown in Tables 63–66. The results were most sensitive to the intervention effect, the transmission probability and the number of sexual partners.

The parameters are investigated in more detail in Tables 63–67. The model results are most sensitive to changes to the model parameters for chlamydia, whilst changes to the model parameters for the other STIs produce only small changes in the model results.

Table 67 shows the effect of changing parameter values for STI complications on the results. These indicate that the sensitive parameters are those related to tubal infertility, whilst those for the other complications have little effect on the results.

Probabilistic sensitivity analysis

The PSA was conducted for the base-case analysis for the teacher-led intervention (i.e. based on the SHARE trial). The main parameters were varied according to the ranges used in the deterministic sensitivity analysis. The PSA parameter values were sampled from appropriate distributions for these parameters as shown in Appendix 11. The model was run for 1000 iterations.

The teacher-led intervention was typically associated with increased costs (Figure 7 – scatter plot of the incremental cost and QALYs), although there is a small proportion (1%) of simulations where costs are lower (associated with positive incremental outcomes). The percentile-based 95% CI for incremental cost is £341 to £12,060. There is a wide range for incremental QALYs, as shown in Figure 7, from –2.0 to 2.5, with poorer incremental outcomes typically being associated with higher incremental costs. In 23% of the simulations the teacher-led intervention was associated with a QALY loss. The percentile-based 95% CI for incremental QALYs is –0.7 to 2.

FIGURE 7.

Scatter plot of PSA results for 1000 iterations for the teacher-led intervention.

In addition to graphing the incremental cost and incremental QALYs for the teacher-led intervention, a CEAC was derived, representing the proportion of simulations where the teacher-led intervention is cost-effective for a range of willingness-to-pay thresholds, up to £100,000 (Figure 8). In this analysis the teacher-led intervention had a probability of being cost-effective of 46% at a willingness-to-pay threshold of £20,000, and 54% at a willingness-to-pay threshold of £30,000.

FIGURE 8.

CEAC of results for 1000 iterations for the teacher- and peer-led interventions.

Expected value of perfect information for the prevention of STIs

Decision-makers may be interested in whether it would be worthwhile conducting further research to gain more precise information on uncertain parameters in the model. For example, in our STI model, we may be interested in whether it would be better to derive more precise estimates of the effect of behavioural interventions on condom use or other outcomes (by conducting a further trial), better estimates of sexual activity in younger people, especially those under 16 years of age (using a sexual health survey) and whether either of these would be worthwhile, considering what the likely health gain for the population would be. Value of information analysis attempts to answer these questions by analysing the hypothetical case for which perfect information could be obtained through further research. 114 The EVPI is the price that one would be willing to pay in order to gain access to perfect information through further research, and represents an upper bound on the value of conducting further research. The EVPI varies according to the cost-effectiveness threshold. For interventions in which there is large uncertainty around whether they should be adopted, for example when the ICER is close to the cost-effectiveness threshold, the health-care system may be prepared to pay more for research which informs decisions on these interventions.

For the STI model, the individual EVPI was calculated as the difference between the expected net benefit with perfect and with current information. The total EVPI is estimated for the total current and future population of people who may benefit from this intervention. We assumed that the effective lifetime of the intervention would be 10 years. The effective lifetime represents the time until a new intervention supersedes or replaces it, rather than the duration of effect. There are 640,000 children of 15 years of age in England and Wales in 2008 (ONS). 157 The effective population over the lifetime of the intervention is 5.2 million (with a discount rate of 3.5%). The population EVPI is shown in Figure 9.

FIGURE 9.

Population EVPI curve for the teacher-led intervention.

At a threshold for cost-effectiveness of £20,000 per QALY, the population EVPI is £12.5M, assuming a 10-year lifetime for the intervention, suggesting a upper limit to expenditure on further research to reduce uncertainty. However this overall EVPI estimate gives no indication which of the uncertain parameters has an impact on the decision whether the new technology is cost effective or not. As a result, the global EVPI does not help to answer the question posed at the beginning of this section – whether it would be better to fund an RCT (to improve the precision of the effect estimate) epidemiological research (on the prevalence of STIs and their complications in this population) or surveys of sexual lifestyles. The following section extends the analysis of EVPI to groups of parameters to identify the priorities for research to reduce decision uncertainty in the model.

Expected value of perfect information for individual parameters

The effect of individual parameters on the population EVPI was investigated by running an EVPPI analysis. 114 This shows the value of reducing the uncertainty surrounding particular input parameters in the decision model, by conducting further research on these parameters in order to obtain perfect information.

The PSA was run for 1000 iterations, whilst keeping the parameter value(s) of interest constant. This process was repeated for 100 different parameter values. The EVPIs associated with the inputs of the model are illustrated in Figure 10 for a threshold of £20,000 per QALY and a 10-year effective lifetime for the intervention. The value of information associated with the intervention effect for condom use is £10.2M, which is substantially higher than any other model inputs and accounts for over half of the decision EVPI. Furthermore, the EVPI for STI prevalence and condom effectiveness is zero and the EVPPI for baseline condom use, sexual behaviour and intervention cost is low, indicating that future research for these parameters would not reduce the decision uncertainty in the model.

FIGURE 10.

EVPI for the individual parameters for teacher-led intervention.

Principal findings of the systematic review

The role of theory

The vast majority of the interventions were theory-based, and the theories tended to be drawn from sociopsychological models of health-related behaviour change. At their simplest, these theories propose that interventions which foster favourable attitudes and intentions will, in turn, encourage the adoption and maintenance of health-promoting behaviours. Whilst many texts on the application of theory to explain health-related behaviour acknowledge the role of knowledge, attitudes and behaviours, it is also acknowledged that these cannot necessarily bring about behaviour change without structural and environmental reinforcements (e.g. service provision, policy, legislation, etc.). 165

The role of peer norms and role models were also considered important. For example, a key tenet of Social Learning Theory72–74 is that young people will be influenced by the attitudes and behaviour of role models, and will, in turn, copy them. Accordingly, some of the studies included in this review used peers (sometimes chosen for their influence) to deliver interventions. The studies varied in the extent to which they discussed the conceptual mechanisms through which they expected the interventions to operate. Some made only passing reference to theory (e.g. Coyle and colleagues58), whilst others (e.g. Karnell and colleagues63) were more explicit about how the theory chosen might encourage favourable sexual behaviour. Of course, no one theoretical model is without limitations, and it was therefore encouraging that many of the interventions were based on a range of complementary models.

Despite the predominance of theory-based interventions, one of the potential explanations for the lack of impact on behaviour in the studies included in this review is that the interventions were poorly conceived. That is, the theories chosen did not adequately guide the intervention in helping young people to develop the knowledge, attitudes and skills necessary to engage in safer sexual behaviour. However, it appears that the theories employed were generally appropriate to the aims of the interventions. For example, the Theory of Planned Behaviour76 (used in three of the studies) recognises that, despite favourable attitudes and intentions, external factors, such as the influence of sexual partners, can limit the extent to which intentions can be translated into behaviour. To account for this many of the interventions drew on the concept of self-efficacy to underpin the teaching of skills to use condoms correctly and to communicate with partners. The aim was to bridge the gap between intention and behaviour, particularly in situations when sexual partners might favour unprotected sex.

The ‘Youth AIDS Prevention Project’ evaluated by Levy and Colleagues65 had a particular focus on teaching young people decision-making and resistance/negotiation skills. They distinguished between ‘use self-efficacy’ (the perceived ability to obtain and use contraception) and ‘refusal self-efficacy’ (the perceived ability to refuse to engage in high-risk behaviours). This distinction is reflective of the range of different skills taught by some of the studies.

As reported in Chapter 4, under Skills and self-efficacy, many of the studies demonstrated statistically significant differences between study groups in terms of condom use self-efficacy, suggesting that the theories were only partially successful in achieving behaviour change goals (though the influence of non-theory-related factors, such as the length of follow-up and the appropriateness of specific types of behavioural outcome measures, may have also influenced the results, as mentioned earlier). There were fewer significant differences for other measures such as sex refusal self-efficacy and communication self-efficacy. The reasons for this disparity are not entirely clear, though it is possible that the young people felt more confident about using condoms correctly (e.g. by practising on anatomical models) than they did in terms of communication (e.g. in role play scenarios). This explanation is supported by the results of the synthesis of the process evaluation results (Chapter 5), which found that there were barriers to the teaching of sexual communication skills. This was a particular issue in the SHARE trial,70 in which, despite the fact that teachers found their training to be beneficial and acceptable, they felt inhibited in facilitating role play and other skills-building exercises in the classroom. It should therefore be acknowledged that whilst an intervention might be well conceived, it may not be delivered as intended, and, consequently, not remain faithful to its theoretical principles.

Another issue to bear in mind is the fact that interventions that foster self-efficacy and teach sexual health negotiation skills may not necessarily be appropriate for all young people. These models assume that young people are committed to protecting themselves from STIs and that, once armed with all the necessary knowledge, skills and confidence, will always choose to have safer sex. However, this is an unrealistic goal as some young people will make a conscious decision to participate in unprotected sex, either episodically or routinely. The theories underpinning the interventions in this systematic review seemed generally more geared to enabling young people to recognise risks and protect themselves, rather than to help those who already have the skills and confidence to negotiate safer sex, but choose not to. It is important that any future interventions with this group should be based on a theoretical perspective that accounts for this.

Finally, this systematic review restricted inclusion to behavioural interventions in which skills were taught within the context of sexual health. It should be acknowledged that there are broader school-based approaches to fostering young people’s health (of which sexual health is one outcome), which teach broader everyday life skills that can be applied in a variety of contexts, including personal and sexual relationships. 166 The effectiveness of such interventions might be a suitable topic for future evidence syntheses.

Findings from other systematic reviews

The results of other relatively recent systematic reviews in this area have been mixed, but generally show that behavioural interventions can encourage safer sexual behaviours amongst young people, to varying degrees.

The Cochrane review of ‘abstinence plus’ interventions (i.e. promotion of abstinence from sexual activity, but also of condom use and other safer sex practices), included 39 randomised or quasi-randomised trials, of which 10 were based in schools (Underhill and colleagues167). The mean age of the participants varied between 11 to 19 years, and all were based in the USA, Canada or the Bahamas. A meta-analysis was not performed due to the heterogeneous nature of the interventions, and lack of data. Of the 39 trials, 24 reported a significantly protective intervention effect on any sexual risk behaviour or biological outcomes. The number of trials reporting statistically significant results in favour of the intervention varied according to different behavioural outcomes: self-reported frequency of unprotected vaginal sex (6/12 trials); incidence and frequency of all sex (5/21 trials); number of partners (4/13 trials); condom use (14/26 trials); and sexual initiation (4/19 trials). Statistically significant effects on knowledge in favour of the intervention were reported in many studies. It was concluded that many abstinence-plus programmes reduce short- and long-term HIV risk behaviour.

However, a review of abstinence only interventions in high-income countries by the same authors came to less optimistic conclusions (Underhill and colleagues168). Of the 13 randomised or quasi-randomised trials included, seven were school based. There was no consistent effect on unprotected vaginal intercourse, frequency of vaginal sex, number of partners, sexual initiation or condom use.

Robin and colleagues2 systematically reviewed behavioural interventions to prevent HIV/STIs and pregnancy amongst young people, and published in the 1990s. Of the 20 included studies, nine were based in schools. Nine studies were classified as having a ‘positive’ effect (defined as a positive effect on at least one behavioural or biological outcome and no negative effects relative to the control group), five studies had ‘null’ effects (defined as no differences among groups for any of the behavioural or biological outcomes), and three had ‘negative’ effects (the intervention had any negative impact on one or more of the behavioural or biological outcomes, regardless of any positive findings).

Johnson and colleagues3 included randomised and quasi-experimental trials of HIV prevention interventions in young people aged 11–18 years. A total of 56 interventions were reviewed, the majority of which were delivered in North America, and in school or community settings. Statistically significant differences in favour of the intervention were found for frequency of sex, condom use, skills for condom use negotiations, and skills for condom use, and communications with a sex partner (based on a meta-analysis of effect sizes). Factors cited as influencing condom use included interventions with condom information and skills training, and studies where the comparison group received less HIV skills training.

Sales and colleagues169 systematically reviewed STI and HIV interventions published in peer-reviewed journals between 1994 and 2004, delivered in a range of settings for young people aged between 11 and 22 years. Of the 39 interventions included, 13 were conducted in schools. Of these three-quarters reported ‘some behaviour change’ with reducing the frequency of unprotected sexual intercourse was the most frequent outcome. Several studies reported a delay in initiation of intercourse and/or a delay in frequency of intercourse. Effective interventions were noted to be theory-based, implemented by trained teachers or health educators, and including skills and knowledge-building activities.

In summary, other published systematic reviews of interventions to prevent STIs with young people provide a more optimistic picture of the ability of behavioural interventions to influence sexual behaviour than this systematic review. However, these reviews are not wholly comparable with our review due to differences in scope and inclusion criteria, and, particularly, the fact that they were not restricted to school-based interventions.

Methodological quality of the outcome and process evaluations

The RCTs included in our systematic review were of reasonable methodological quality. However, poor reporting of factors such as randomisation procedure, concealment of allocation and attrition prohibited a thorough assessment of quality.

It is encouraging that a high proportion of the 15 outcome evaluations also conducted a process evaluation. Process evaluation is essential to explain what factors may have contributed to success or failure of the intervention, to assess its acceptability to the stakeholders involved, the determine the fidelity of the intervention delivery, and to establish the generalisability and replicability of an intervention. 170 However, the extensiveness of the process evaluations included in this review varied from cursory monitoring of intervention activities to larger scale, comprehensive quantitative and qualitative evaluation of multiple aspects of the intervention. The two UK-based studies included in the review both conducted comprehensive process evaluations, with extensive data reported in a number of publications50,70 (see Appendix 7 for a bibliography of these).

Improving the quality and extent of process evaluations would add much value for assessments of the effectiveness of complex interventions, such as the ones under consideration in this review. Process evaluations of experimental interventions should be more than superficial monitoring exercises. Ideally, process evaluations should collect a range of qualitative and quantitative data on intervention fidelity, programme reach, and provider and user perspectives on the intervention. The collection of qualitative data is particularly important for understanding how contextual factors affect implementation and acceptability and for an assessment of the potential generalisability of an intervention.

The wider evidence base

The descriptive map reported in Chapter 3 provides a useful context within which to discuss the results of the systematic review. The majority of studies included in the map, and the systematic review itself, were conducted in the USA. This has been the case in health promotion systematic reviews in other areas, such as healthy eating,33 and reflects the strong tradition of experimental evaluation of social interventions in the USA over the decades. 171,172

Many of the interventions mapped were delivered in schools by teachers and/or peers. Although there were some examples of outreach interventions located in community settings, the predominance of school-based activities raises the question of whether the needs of the most vulnerable young people are being met. School activities are not likely to reach those persistently absent from school, young people who have left school early, or young offenders. They may not necessarily appeal to young people who are disaffected by school and the education system in general,173 although the influence of credible peer educators may help to overcome this. As explained earlier, school-based interventions were prioritised for our systematic review because, in consultation with stakeholders, it was considered that they were likely to reach the greatest number of young people. The systematic review potentially could have focused on a different subset of studies from the map that specifically included young people classified as being ‘at risk’ (refer back to Chapter 3, Table 9). However, this would have resulted in a systematic review of a more diverse set of interventions, in terms of factors such as setting and provider, which would have made comparison between studies more problematic. This does not, though, preclude such a systematic review in the future. One of the advantages of this kind of descriptive map is that it is a resource that can be used to identify and prioritise different topics for future evidence syntheses in accordance with policy needs (see Chapter 9, Recommendations for research).

Systematic review of effectiveness

One of the strengths of this review is its adherence to rigorous systematic review methods. We conducted exhaustive searches, applied explicit inclusion criteria to search results, critically appraised included studies and used transparent methods to synthesise study findings. A further strength is our incorporation of two relatively new and innovative review elements. Firstly, we conducted the review in two stages with an initial mapping stage followed by a systematic review of a subset of studies. This process facilitated the involvement of end-users in the design of the review. After our map was completed we consulted with our advisory group to ensure that any subset of studies we focused on was relevant and coherent in terms of policy and practice. Secondly we included, quality assessed and synthesised process evaluations. This enabled us to explore, in a systematic way, factors influencing the implementation of the intervention and to generate an explanation for the outcomes reported.

Despite its strengths the review had limitations. It was preferable for the subset of studies in the systematic review to be homogeneous in terms of intervention characteristics (e.g. provider, setting, materials, length/intensity, etc.) to ensure that their aggregation in a quantitative meta-analysis was meaningful and appropriate (i.e. that ‘like was being compared with like’). Whilst prioritising what appears on face value to be a fairly standardised behavioural intervention (i.e. school-based education and skills development sexual health promotion) there was, nonetheless, some degree of variation between the studies. As mentioned above, some interventions were relatively brief whilst others more extensive. Some were curriculum focused, whilst others were delivered in the context of wider school sexual health promotion initiatives, and supplemented by activities in the home and community. Furthermore, although all were included because they included an element of skills development, the type of skills taught, and the extent to which this was a focus of the intervention, varied. Some were theory-based interventions, designed specifically to improve sexual negotiation skills, whilst others provided relatively little rationale for skills training. This variability should be kept in mind in the interpretation of both the narrative synthesis and the meta-analysis.

The potential to perform meta-analysis was limited primarily by the lack of suitable data from the study publications. Common limitations included failure to report a measure of variance (e.g. SD) to allow continuous outcomes to be combined, or poor reporting of the number of young people with a given outcome (e.g. using a condom) to allow binary outcomes to be pooled. Our attempts to contact the authors to kindly request missing data were generally unsuccessful, as many did not reply to our e-mails. Although some of the studies were carried out some years ago, many of the authors still appear to be actively researching this area. Consequently, only a small proportion of the studies were able to be meta-analysed. Whilst one of the strengths of the meta-analysis was that it was only based on studies judged to be methodologically sound, not all of these studies were able to be included in it due to poor reporting of data.

Another issue to bear in mind is the fact that, despite a relatively narrow age range (13–19 years), it cannot be assumed that teenagers are a homogeneous group. There are likely to be numerous differences between younger and older teenagers in terms of social and sexual development. We have endeavoured, where possible, to take these differences into account in the analysis and interpretation of our results, although data on age subgroups are limited. It is also important to acknowledge that a number of studies that targeted young people aged up to their early to mid-twenties were excluded from our descriptive map (and hence our systematic review). These were not included as the commissioning brief for this project specified inclusion of young people aged 13–19 years. We also noted that much of the epidemiological and sexual lifestyle data relating to young people is for the 16–25 age group. Further evidence synthesis could therefore examine the clinical effectiveness and cost-effectiveness of behavioural interventions in this older group.

Economic evaluation

Our economic evaluation is one of the few published examples of an assessment of the cost-effectiveness of school-based behavioural interventions, particularly in the UK. It was informed by a systematic review of effectiveness and cost-effectiveness studies, and systematic searches for input parameter data.

Despite these strengths it is subject to limitations. As reported in Chapter 4 (see Sexual behaviour), our meta-analysis did not show a statistically significant intervention effect for behavioural outcomes. Absence of a statistically significant difference between behavioural intervention and standard sex education does not necessarily indicate that they are equivalent in terms of effect. However, rather than present a cost-minimisation analysis, which would be commonly be performed in this situation, we chose to report how a simple static model could provide illustrative estimates of the likely cost-effectiveness of two types of school-based behavioural intervention. A quantitative analysis of the impact of uncertainty on these illustrative results was undertaken using probabilistic sensitivity analysis, along with an indication of the likely cost and benefit of future primary research, via expected value of perfect information analysis. The static nature of the model does not take into account the dynamic nature of infectious diseases. However, a more pragmatic approach was considered appropriate, given the absence of key data for parameters needed to develop, calibrate and validate a robust dynamic model.

Primary research

If a further RCT of school-based sexual health education were to be commissioned in the UK it would be useful to evaluate the approach taken in one of the more effective studies included in our review, such as an adaptation of an approach similar to the Safer Choices intervention evaluated in the USA.

The intervention should comprise a range of components, including curriculum activities spread over at least two school years (single-sex lessons where necessary and feasible), accompanied by wider school health promotion activities, plus involvement of parents and, where necessary, health services (e.g. to encourage STI testing) and other relevant stakeholders (e.g. youth workers). A possible approach for evaluation would be for teachers and/or peers to provide factual information about STIs, with specialist trainers brought into schools to teach skills development. A pilot phase using qualitative research should assess the cultural and social relevance to the UK, and adaptations made accordingly.

The effectiveness of ‘booster’ sessions for young people progressing to further education and for those leaving full-time education should be explored. The aim would be to encourage sexually active young people to maintain protective behaviours, and to support those beginning to have sex in the adoption of safer behaviours.

All future evaluations should be rigorously designed and executed, with long-term follow-up of a range of outcomes, including sexual behaviour (e.g. condom use), conceptions and abortions, use of health services (e.g. STI testing) and rates of infections. Evaluations need to adequately measure the complexity of some young people’s risk-reduction strategies, assessing use of negotiated safety strategies with partners.

Outcome evaluations should be accompanied by extensive process evaluation, and have an integral cost-effectiveness evaluation.

Research commissioners, journal editors and other relevant research stakeholders should encourage researchers to undertake analyses to assess the impact of interventions on sexual health inequalities. They should further encourage complete reporting of data and methods of analyses that investigate the impact of an intervention on the health outcomes of different sociodemographic groups.

Journal editors should subscribe to the revised Consolidated Standards of Reporting Trials (CONSORT) statement, which is intended to improve the overall reporting of RCTs,174 and the Transparent Reporting of Evaluations with Nonrandomised Designs (TREND) statement for improving the reporting of non-randomised evaluations of behavioural and public health interventions. 175 These include guidance on appropriate reporting of outcome data and subgroup analyses.

Evidence synthesis

Further evidence synthesis would be appropriate to identify a subset of studies from our descriptive map with a particular focus on interventions aimed at young people classified as being vulnerable and at risk. It should be acknowledged that this would likely result in a more heterogeneous set of interventions in terms of setting, provider and message.

The effectiveness of skills development interventions in other (non-school) settings is also an important subject for evidence synthesis, complementary to this systematic review.

The effectiveness of broader school-based approaches to fostering young people’s health, which teach everyday life skills that can be applied in a variety of contexts, including personal and sexual relationships, might be a suitable topic for future evidence syntheses.

Evidence synthesis could also examine the clinical effectiveness and cost-effectiveness of behavioural interventions in the 16–25 age group.

Cost-effectiveness

As specified above, any future primary research conducted should be accompanied by detailed economic evaluation to assess cost-effectiveness. Detailed data of the costs of mounting the intervention should be collected, and the impact on HRQoL should be assessed to enable cost–utility analysis. The analysis should extend over a time horizon that is long enough to capture all of the intended costs and consequences (e.g. until young people are in their early twenties).

For many of the parameters used to inform our economic model there were few or no available data for the < 16-year-old age group, necessitating assumptions to be based on extrapolated data from older age groups. For example, there were few data on self-reported sexual behaviour of under-16s (e.g. number of sexual partners or episodes). The third National Survey of Sexual Attitudes and Lifestyles will commence in 2010, although as yet it does not appear that data on under-16s will be collected. There is a need for surveys such as this to collect data on younger teenagers, where possible, to inform research and enable the effective planning of services.

There is also a need for prospective cohort studies to inform the parameters used in economic modelling (e.g. transmission probabilities of STIs).

In order to reduce the uncertainty in the economic evaluation, analyses indicated that it is likely to cost up to £12.5M in order to obtain perfect information and that research would be best funded to establish the effectiveness of the intervention in terms of condom use, as this parameter has the largest effect on the uncertainty of the model results.